Intelligent driving and intelligent assisted driving simulation and verification framework

Autonomous vehicles (AVs) integrate complex perception and localization components to create a model of the world around them, which is then used to safely navigate the vehicle. Machine learning (ML) based models are commonly used in these components to extract object information from noisy sensor data. The requirements for these components are mainly to achieve the highest possible accuracy. Since modern cars deploy many sensors (vision, radar, and lidar), processing all the data in real time causes engineers to make trade-offs, which can result in suboptimal systems in certain driving situations.

Modular testing and validation also becomes challenging due to the lack of precise requirements for individual components.

The problem of the accuracy of abstract world models required to derive safe AV behavior from top-level driving scenario simulations is currently being raised in the industry. This is computationally expensive because the world model may contain many objects with multiple properties, and the AV extracts a world model at each time step during the simulation.

The field of autonomous driving is advancing rapidly with the advancement of sensor and computing technologies. Establishing the safety of autonomous vehicles is a challenging task due to the various conditions in which autonomous vehicles must operate and the complexity of their system implementation. The localization and perception components in the AV absorb sensor and map information to create a world model to capture the environment around the AV. This world model is then passed to the planning module to create a safe trajectory based on its goals. Vision- and lidar-based perception components increasingly use ML models to achieve 2D and 3D object detection. It is difficult to reason about the safety requirements of ML-based perception because it is unclear whether (and how) inaccurate perception will violate the highest level safety goals.

In practice, the requirements for different AV components are driven by experts in the autonomous driving industry and are mainly based on experience. In addition, these requirements are set conservatively and are universal across different driving conditions and operational design domains (ODDs). For example, the localization component should be relatively more accurate at a busy intersection compared to a sparse rural road.

Similarly, the perception component should have high recall and precision on highways, but only high recall is required in pedestrian areas. Ideally, one would like to use many high-resolution (e.g., 24-megapixel) cameras running at high frame rates (e.g., 120 FPS) and employ multiple high-precision, complex DNN models to perceive everything around the vehicle as accurately as possible. Since AVs run on resource-limited platforms, system designers have to make trade-offs and design a system that is accurate enough (e.g., using 2-8 megapixel cameras, 30 FPS, and optimized/quantized DNN models with slightly lower accuracy). This solution based on general requirements may result in a less safe system in certain situations where high-precision perception is required in certain areas around the AV (e.g., objects approaching quickly from the side at an intersection may require enhanced tracking).

Hardware-in-the-loop (HIL) and software-in-the-loop (SIL) simulations provide an effective end-to-end testing approach for AV systems. HIL testing uses automotive hardware, sensors, and possibly actuators for system verification and validation. Software-in-the-loop (SIL) simulations are used during the design phase as well as in unit and integration testing, where the inputs to a unit or component are automatically generated or handcrafted to simulate the effectiveness of different input parameters.

To enable the design of AV systems to better utilize resources for safer driving, a simulation-driven approach is proposed to compute the world model accuracy requirements for safe AV behavior.