How is simulation testing currently developing in the field of autonomous driving?

As the autonomous driving competition enters the second half, promoting commercial applications has become the focus of all companies. According to research by the Rand Corporation of the United States, if autonomous driving algorithms want to reach the level of human drivers, they need at least 17.7 billion kilometers of driving data to improve the algorithms.

If a fleet of 100 self-driving test vehicles were deployed to conduct road tests 24 hours a day at an average speed of 25 miles (40 kilometers) per hour, it would take more than 500 years to complete the target mileage. The time and cost consumed during this period would undoubtedly be unbearable.

Using simulation technology for testing is considered the key to reducing costs and improving efficiency. At present, how is the development of simulation testing in the field of autonomous driving? On October 12, the China Electric Vehicle 100 Forum, Tencent Autonomous Driving, and China Automotive Data Co., Ltd. jointly released the "2020 China Autonomous Driving Simulation Blue Book", which details the current status and challenges of technology application.

Trend: In the future, 99.9% of tests may use simulation platforms

Simulation testing based on scenario libraries is an important way to solve the lack of autonomous driving road test data. Simulation testing mainly builds a virtual scenario library to achieve closed-loop simulation testing of autonomous driving perception, decision-making planning, control and other algorithms to meet the requirements of autonomous driving testing.

The scenario library is the basis of autonomous driving simulation testing. The higher the coverage of the scenario library in the real world, the more realistic the simulation test results. The requirements for the scenario library vary at different stages of autonomous driving vehicle development, and the scenario library is required to implement different test functions.

In the development process of autonomous driving, the development process of pure model simulation - software-in-the-loop simulation - semi-physical simulation - closed-field road testing - open road testing is the most economical and efficient development process.

At present, autonomous driving simulation has been widely accepted by the industry. For example, Carcraft, a simulation platform owned by Waymo, a leading American autonomous driving company, drives about 20 million miles on virtual roads every day, which is equivalent to 10 years of driving in the real world. As of May 2020, Waymo has simulated 15 billion miles, compared with 10 billion miles in June last year.

In addition to Waymo, domestic and foreign autonomous driving solution providers such as General Motors' Cruise, AutoX, and Pony.ai are also conducting a large number of simulation tests to improve their autonomous driving systems. Simulation testing has become the most important test for commercial use of autonomous driving.

According to current data, about 90% of autonomous driving algorithm testing is completed through simulation platforms, 9% is completed in test fields, and 1% is completed through actual road testing. With the improvement of simulation technology and the popularization of its application, the industry hopes to complete 99.9% of the testing volume through simulation platforms, 0.09% through closed field testing, and the last 0.01% on the actual road. In this way, the development of autonomous driving vehicles will be more efficient and economical.

Current situation: Participants in each track are actively deploying

At present, the main participants in the autonomous driving simulation market include: technology companies, car companies, autonomous driving solution providers, simulation software companies, universities and research institutions, and intelligent network test demonstration areas. Since each market player has a different technical foundation in autonomous driving simulation, the research and development and cooperation methods in promoting autonomous driving simulation present different modes.

Technology companies started relatively late in simulation and have less experience in exploring automotive functions, but they have advantages in big data and strong software development capabilities.

Compared with traditional cars, self-driving cars have a greater demand for software. Technology companies are exploring simulation software in order to enter the huge automotive industry, build a larger data platform, and form a new business growth point. Currently, the main self-driving simulation technology companies include Tencent, Baidu, Huawei, Alibaba, etc.

Foreign technology companies such as Microsoft, NVIDIA and LG mainly focus on the research and development of autonomous driving simulation software. They have established an autonomous driving research and development ecosystem by cooperating with companies in the industrial chain and have become important participants in autonomous driving simulation.

For vehicle manufacturers, the best option is to conduct road testing and simulation testing simultaneously. Before autonomous vehicles can be put into use, they must undergo numerous functional and safety tests, of which road testing is one. Due to the low efficiency of road testing, many car companies currently prefer to combine autonomous driving simulation testing with road testing to complete pre-launch testing.

Autonomous driving solution providers mainly develop customized simulation software for their own needs and rarely provide simulation services to the outside world. However, with sufficient funds, talent concentration and their own R&D driving force, they have strong innovation capabilities in autonomous driving simulation. Leading autonomous driving solution providers have their own simulation test software, such as Waymo, Cruise, Pony.ai, AutoX, etc.

Simulation software companies can be divided into two categories: traditional simulation software companies and start-ups. Traditional simulation software companies have inherent advantages in entering autonomous driving simulation due to their deep technical accumulation, and they have more partners and advantages in secondary development. Start-ups started late and have weak technical accumulation. There is a big gap between domestic and foreign companies, but relying on strong funds and talent aggregation, start-ups are expected to rise rapidly in the research and development of autonomous driving simulation software.

Universities and research institutes mainly use autonomous driving simulation software for forward-looking and basic research, but it is difficult to form mature commercial products. Domestic universities and research institutes engaged in autonomous driving simulation research mainly include: Tsinghua University, Tongji University, Beijing University of Aeronautics and Astronautics, Jilin University, Tianjin University, Chang'an University, Nanjing University of Aeronautics and Astronautics, Wuhan University of Technology, etc.

The construction of intelligent network test demonstration zones has reached a certain scale. At present, there are more than 10 national and several provincial intelligent network test demonstration zones across the country. They mainly provide system testing services for autonomous driving, network communication suppliers, etc. through the deployment and application of new technologies such as 5G, V2X vehicle-road collaboration, simulation, and Internet of Vehicles, and promote the establishment of a comprehensive standard system for automobiles, information communications, road facilities, etc. In order to promote the simulation testing of intelligent network vehicles, some enterprises and intelligent network demonstration zones have carried out tests combining road testing with virtual simulation.

Challenge: Lack of standardization of test evaluation system

At present, the autonomous driving simulation test has initially formed a complete industrial chain system, forming an upstream simulation software provider mainly composed of technology companies, autonomous driving solution providers, and simulation software companies, and a downstream simulation software application provider mainly composed of car companies and autonomous driving test institutions. From the perspective of the industrial chain, there are still many problems in the current autonomous driving simulation test.

On the one hand, the construction of simulation scenario library and cooperation mechanism need to be improved.

The construction of scene libraries is inefficient and costly. Currently, the construction of scene libraries still relies on a large number of manual labor for data collection and labeling, and then scene analysis and mining, testing and verification. The entire process is inefficient and costly. Currently, the global annual cost of manual labeling is in the order of $1 billion.

The scale of the scenario library is not large enough, and the diversity, coverage, and scalability are not strong. The existing scenario library is not enough to cover common traffic scenarios, and with limited resource investment, it cannot effectively cover the diversity of the real world. Since changes in different elements in the scene can be expanded into different scenes, the current scenario scalability is not enough to meet the requirements of simulation testing.

The effectiveness of the scenarios needs to be improved. The existing scenarios are based on real-time data collection and cannot meet the requirements of dynamic changes in autonomous driving scenarios. In the scenarios, dynamic and static elements such as people, vehicles, roads, and driving environments are coupled. Changes in one element will cause changes in other elements, and different traffic participants have their own behavioral logic. For example, if the vehicle behavior and trajectory are changed, the behavior of surrounding vehicles and pedestrians will also change accordingly.

The collection format and storage issues of scene data. The existing test scene collection is based on different vehicle and sensor configurations and cannot be applied to the research and development and testing of various vehicle models and technical routes. The format of high-precision maps is also a focus of industry attention. The unification of the data format of the scene library, such as system architecture, data interface, database management system, etc., is also an issue that needs to be focused on.