The implementation of end-to-end autonomous driving in China has achieved significant results!

Recently, a research team led by Academician Li Keqiang and Professor Li Shengbo from the School of Automotive Engineering at Tsinghua University completed the country's first open road test of a full-stack end-to-end autonomous driving system.

Relying on the vehicle-road-cloud integrated intelligent connected driving architecture, the end-to-end autonomous driving system developed by the team covers the entire chain of "perception-prediction-decision-making-planning-control". In January this year, it took the lead in launching open road verification in urban conditions. After nearly 4 months of internal testing, it completed a comprehensive evaluation of various performances. This work has laid a solid foundation for the implementation of L3 and above high-level autonomous driving systems.

Full-link end-to-end autonomous driving system from perception to control (schematic diagram)

At present, L1 and L2 intelligent driving systems mainly rely on the design concept of "module decomposition". Although some modules (such as perception, prediction, etc.) have been initially neuralized, modules such as decision-making, planning, and control still rely heavily on manual rules and online optimization, and lack the ability to use data for closed-loop iteration, which leads to insufficient intelligence in the driving process.

At the same time, there is inevitably a large amount of information loss between modules, and there are certain conflicts in the optimization objectives of each module, which is not conducive to improving the overall performance of the autonomous driving process.

In comparison, the "end-to-end" autonomous driving system characterized by full-module neural network can rely on high-dimensional feature vectors for information transmission between modules, and the neural network has sufficient training freedom, which minimizes the information loss between sensors and actuators. This enables the full-stack module to have the ability to quickly update using data closed loops, which provides a new technical path for improving the intelligence of high-level autonomous driving.

In response to this technological development trend, the research team led by Academician Li Keqiang and Professor Li Shengbo has been focusing on the field of end-to-end autonomous driving since 2018, with a focus on breaking through the difficulties in neural network design and training in the areas of decision-making, planning and control.

The team has successively proposed an integrated decision-making and control (IDC) development framework for high-level autonomous driving, developed a data-driven reinforcement learning algorithm (DSAC) with internationally leading comprehensive performance, pioneered a spatiotemporal separation of traffic participant behavior prediction model (SEPT), designed a control neural network architecture (LipsNet) with smooth motion characteristics, and developed an optimal control strategy approximation solver (GOPS) with independent intellectual property rights. With the spirit of ants moving house, they have solved a series of core problems faced by end-to-end autonomous driving one by one.

Based on this, at the beginning of this year, the team successfully developed the first full-stack neural network autonomous driving system from sensor raw data to actuator control instructions, and was the first to complete actual vehicle testing and verification on open roads in urban conditions.

This research work has received full support from two high-tech companies, Zhixingzhe and Shengqi Technology, forming a joint research team with close cooperation and collaboration between the school and the enterprise. The main work of the Zhixingzhe team is the construction and pre-training of the environmental perception model, and the main work of the Shengqi Technology team is the development of the autonomous driving simulation platform. The three units jointly completed the later tasks such as system function integration and performance evaluation iteration. This research is supported by the National "14th Five-Year Plan" Key R&D Program, the National Natural Science Foundation, and Tsinghua University's Independent Research Program.