Autonomous vehicles are moving closer to becoming a commercial reality. Industry experts predict that by 2060 around three quarters of the vehicles on our roads will be fully autonomous. In the UK, the government has ambitions for the country to play a leading role in autonomous vehicles and believes the UK industry will be worth £28bn in the next 17 years. As a result, codes of practice for testing have been created and countless trials are taking place.
However, to make the vision of AVs a reality, OEMs and Tier 1 suppliers are applying cutting-edge technology to solve some of the biggest problems in computing, engineering, software development and algorithm design today. For test engineers, this same innovation is proving extremely challenging. All these evolving ideas mean testing against unknowns: unknown regulation, unknown technology, unknown architectures, and even unknown algorithms created by neural networks instead of line-by-line code developed by software engineers.
Algorithms are a black box
The only way autonomous vehicles can become a reality is through the application of machine learning. There are an infinite number of possible scenarios that a vehicle could encounter; however, it’s impossible to hard code the algorithms to successfully negotiate all of them. Instead, massive data sets are being recorded along with humans’ reactions to the driving scenarios. These are then fed into neural networks.
Overall, this makes the test engineer’s job much harder. Algorithms are now a black box. This requires more extensive testing because you don’t have an understanding of the code that can be used to generate test scenarios. Instead, engineers need to test against almost every conceivable scenario to ensure the algorithms function properly.
Additionally, as training data sets change, everything needs to be retested. You don’t have the benefit of being able to isolate sections of code, because the code is dynamic and evolves as data sets are updated.
Test organizations also have even less time to run all this testing. Automotive companies are in a race to launch products to market and so testing timelines have decreased. To address this, test engineers are turning to machine learning to define the test cases that will give them maximum test coverage in the limited testing time available. They are even using it to identify sections of code that are most likely to contain errors.
Test organizations can maximize this effectively by pairing machine learning with software-defined test racks that can be controlled and configured remotely. This enables teams around the world to access the same test resources and run testing 24 hours a day, seven days a week.
Competing vehicle architectures
As technology evolves and becomes more cost competitive, design engineers are constantly updating their autonomous systems to include new and different sensor types. Adding to the complexity is the role cloud computing will play as 5G radio technology is rolled out and will finally offer the bandwidth requirements needed for the massive data streams coming off sensor systems.
What this is creating for test engineers is the need for flexibility, as no-one knows how many sensors will be on autonomous vehicles in the future. Test systems need to be adaptable to change, so they can add more cameras or radar sensors, and tack on new sensor types like lidar.
Choosing a test architecture based on commercial off-the-shelf platforms is one way to handle these changing vehicle topologies. With a platform-based system, the test vendor integrates new technology and I/O as they become available to the market. These new capabilities can then easily be integrated into existing system designs.
Collaboration drives innovation
Automotive companies can no longer only be experts in mechanical and electrical engineering. They now need to be experts in radar, machine vision, motor controls, battery chemistries, big data analysis, and machine learning. Therefore, the industry needs collaboration across automotive suppliers, OEMs, integrators, governments and test vendors alike to truly understand and propose solutions.
However, collaboration brings challenges. For instance, when and if OEMs are legally responsible for total vehicle safety, how much they can trust the testing their suppliers did, and how much they will need to redo. And beyond the legal responsibility, OEMs will always have to worry about their brand image and quality perception, even if their suppliers are legally at fault; again begging the question, how much redundant testing will actually need to take place?
Vehicle architectures are evolving and changing constantly, forcing test engineers to be adaptable to new I/O and test techniques best suited to each architecture
Written by Doug Farrell, transportation solutions marketing manager, National Instruments