The autonomous vehicle market has gathered much interest and investment in recent years, but it has also caused much speculation and public opinion after news stories questioned the safety and viability of self-driving cars. The trend is clear that the automotive industry is driving to higher levels of autonomy and many cities are now allowing autonomous prototypes to share our roads. However, the world is really seeking answers to when these prototypes will become a deployable reality, what will they look like? How much will they cost? Can we trust them?
At Arm TechCon last year, I gathered some of my colleagues from around the automotive industry to discuss the future of autonomous systems and what will be necessary to make them viable and deployable.
The discussions generated lots of illuminating insight around the challenges facing autonomous systems, the technologies and solutions which will enable them and the timescales in which we might see this exciting future made real. I wanted to share this discussion with a broader audience who were not able to attend the session live. I hope you enjoy the discussion!
Phil: This is a grey area, lots of car manufacturers are saying they will skip Level 3. The obvious reason being the difficult problem of getting the driver to take back control of the vehicle, especially if the driver is distracted.
Sasha: You can only have Level 3 autonomy with a driver monitoring system, otherwise the transition from automated to driver operated is difficult. Waymo is public on starting at Level 4, but there is a huge leap from Level 3 to Level 4.
Lakshmi: There are Level 3 cars already, like Audi A8, but this capability is turned off in certain markets because of government regulation. I think it is likely that you as a consumer could have access to a Level 3 car but it is not clear that consumers will be able to buy Level 4 and 5 cars because of their potential high cost. There could be confusion for drivers knowing if they are driving or not. Most accidents are due to human error and Level 3 systems could increase the probability of that.
Mark: I 100% agree with Phil, the reengagement of the driver is the biggest challenge. How do you get the driver re-engaged? I view Level 3 as a legal challenge as well as a technical one. For Level 3 consumer deployments, it's case closed as far as I’m concerned. Level 3 systems are good from a research and learning standpoint though.
Sasha: Waymo says they will launch a self-driving service by the end of the year, these cars will look like mini vans. Cruise have a Chevy Bolt without steering wheels and pedals that they say will launch by the end of next year. Robotaxis seem to be imminent but consumer cars are further off. The fully self-driving car could be 20-30 years from now.
Lakshmi: To think literally about the look of the cars, it will be about your consumer experience when you’re sitting inside. It could be like a living room for long journeys, or an office with a table for commuters, or a bar, as my husband and I talk about. There are lots of interesting options for customizing cars, and we need to work out the use cases. These use cases will affect the look of the cars.
Mark: Context is key, so much has to be considered, take the downtown cab for example. Is there vandalism happening? How much monitoring is necessary? There is lots to consider from an internal and external context. I think that internal experiences will be a lot more interactive. I’d love to see Jonny Cab, but in reality, there could be a lot more “Big Brother” internal monitoring internally.
Phil: I think there will be architectural differences to the car. For example in terms of powertrain, there could be improvements to the steering to help maneuverability and maybe the car has the capability to move backwards and forwards. Higher levels of usage of cars could have an affect also, with cars in use more of the time.
Sasha: In the short term, you need to deal with the “junk in the trunk”, the data center in the car. How do you cost that down, make it robust, reliable, make it run the software stack with low latency? You also need to cost down all the sensors. Machine learning can solve a lot of problems, but it is kind of a black box and you can’t tell why it took the action it took. A neural network is not a lot of code, but it is unknown code. How do you verify and validate machine learning?
Lakshmi: To add to that, the compute can’t take up the whole trunk or the user function will be degraded. Also, the compute power has to use less energy than the prototypes. A watt that you spend on compute is one you don’t spend on range, while the weight of the compute hardware equipment can reduce range.
Mark: How do you make AI comply with ISO26262? How do you verify it, that’s huge! One approach is adding a safety gate to monitor the system which is verifiable and that can deal with catastrophic issues. One other point, time is safety. The faster you can do computation, the more options you have for safety.
Phil: These trunks can get really hot! You can cook your lunch, or worse, your components! We’ve had problem after problem after problem with our vehicles needing extra cooling. A couple of years ago the potential of using of AI in a production vehicle was unheard of and went against all traditional thought, but now any major OEM is working on AI based solutions, so the topic has come a long way. AI based systems really have the best chance of handling all those unique situations you will come across. There has been discussion of using two different software stacks, having a guardian system looking after the main autonomous system.
Lakshmi: This is not about one company, it is about cooperation. Historically, automotive was a closed sourced software environment, with a few people developing code that was real time or constrained in an IVI system. Now, new innovative work on perception is happening in small companies, like DeepScale, or open source communities or universities. A lot of open source innovation is happening, and companies are using it for prototyping activities. There are community efforts like Linaro working with Autoware, and also the Baidu Apollo community. We also need to consider the issues when you move to a production vehicle which is on the road for 7 – 10 years. Software is the bigger challenge for autonomy, not the hardware.
Mark: 100% agree. We learn about new markets from trickle down from other markets. In the 90s I worked with systems which needed full factorial walk-through of all the decision points in the code. It is the same with gambling systems inspected by the Navada Gaming Commission. They need to check there are no backdoors, and you have to go through all possible combinations of that code. I don’t know if we’re going to end up in a domain where we show all possible paths are covered. That’s crazy stuff for autonomous software. An alternative is having a safety gate, another system watching over the main one. This safety gate cannot be an AI system until that AI system can be verifiable.
Phil: We have 55,000 lines of code in a Level 2+ system but that doesn’t include libraries. In terms of an autonomous stack you would separate the systems. Best practice would mean you implement isolation and separation of systems.
Sasha: I am going to disagree but only on short term. In the short term, vertical integration wins. Don’t collaborate with others, collaboration is the enemy of speed and being first to market. If you look at the leaders in this space, they do everything themselves. It is the only way to move fast. Eventually we will get to modularization stage, where leaders will focus on their core competency and outsource other things, which will be standardized and modularized. There are no standards just now, the autonomous vehicle stack is super complex. There are no APIs from one layer to the next. We will get there; each component will be well specified but that’s years away.
Mark: That’s one to talk about! People have only recently started to take security seriously and customers are not currently viewing safety and security simultaneously. They have to be viewed from the ground up and that evolves with the hardware. A key emerging concept is that of a fully verifiable hypervisor which can lock down the different domains to keep things from becoming convolved. The more you convolve, the more you run into problems. Security must start from the ground up. Many people in bay area are transitioning from a research oriented to manufacturing approach. They talk about security, which sounds good, but do they have everything hardened? Issues of isolation and monitoring are huge. A hypervisor can span across SoCs and keep the domain preserved, but that interferes with safety so there are trade-offs. You also need to think about over-the-air (OTA) updates.
Phil: I couldn’t agree more, especially when it comes to OTA updates. If you look at a Tesla, they are doing updates all the time. The car would die without its connectivity. Data going into and out of the vehicle. It astonishes me that no other OEM has come anywhere near close to what Tesla has done with OTA. Look at what it enables them to do! You want self-driving, give them your credit card and they deliver it over-the-air.
Sasha: I hired the Jeep hackers at Cruise as soon as we learned about this and they are still there. This shows how GM and Cruise think about security, they need the experts. It goes back to hardware architecture, software architecture, training engineers to write secure code. From my days at Nvidia, this is something you need to take seriously. You need to train your workforce to introduce security from the start. Make code that is secure, readable, testable.
Lakshmi: Some things we are doing with Platform Security Architecture (PSA), figuring out threat models and bringing security down to the smallest components will help. The amount of electronics in cars vastly increases the attack surface. Trusted execution of software is key. Using Arm TrustZone on our Armv8-M microcontrollers can enable a level of confidence that you’re running what you’re expecting. When we think about it, we have to think from smallest microcontroller to big iron Cortex-A. You also need to enable security efficiently and finally, every component in car has to have hardware security.
I want to thank the panellists for sparing some of their valuable time with us and providing their knowledgeable insight into this popular topic. In the short time we had together at Arm TechCon we did get some quite consistent views on some of the key questions posed to the panel. There is definitely a belief that autonomous cars will happen, with some ideas as to the likely timeframe, but all the panellists agreed that there are still some difficult challenges to overcome, especially around the safety and security of these vehicles.
At Arm we are introducing automotive solutions and technologies that we believe can help automakers and mobility service providers overcome the challenges described above and get the industry to the point that they can deploy autonomous vehicles safely.
Visit our Automotive solutions page for more information on autonomous systems.
[CTAToken URL = "https://www.arm.com/solutions/automotive"_blank" text="Arm Automotive solutions" class ="green"]