Autonomous driving simulation, what exactly is it imitating?

When using smart driving, you must be concerned about the safety of the system, right? How is the algorithm tested and verified before it is installed on the actual vehicle?

A few months ago, we made a video about game engines entering the smart car industry. A few days ago, we exchanged an interesting topic with a company that makes virtual worlds - intelligent driving simulation.

Before the intelligent driving system is officially mass-produced and put into cars, the simulation system is the first examination room for testing and verification.

What exactly is driving simulation?

Welcome to Smart Car Planet. Today, we will talk about the hero behind the continuous iteration of smart driving technology - smart driving simulation.

Before talking about simulation, let us first talk about another concept-digital twin.

What does it mean? Wikipedia's explanation is not obscure. It is more simple and crude to understand. Digital twin is to create a digital version of a "clone" based on a device or system.

For example, the classic game "The Legend of Zelda" can be regarded as a digital twin world.

There are many versions about the origin of digital twins. According to available information, the U.S. Air Force Research Laboratory and NASA are both the proposers of the concept of digital twins. The Air Force Laboratory hopes to solve the health diagnosis and prediction problems of old aircraft, reduce aircraft operation and maintenance costs, and improve utilization rates. NASA hopes to solve the problem of health diagnosis and prediction of space exploration vehicles.

The fields are different, but there are two key similarities -

1. The proposal of digital twins is problem-driven, not technology-driven.

2. The purpose is to significantly and effectively reduce costs.

This is easy to understand. After all, if you want to verify whether a newly designed rocket can launch smoothly, you cannot first create several repeated launch verifications. In that case, not only will the development cycle be too long, but the cost will be ridiculously high, even if you dare to scale up. Musk doesn’t dare to create fireworks so casually.

Looking back at smart driving, there are not only verification cost issues, but also personal safety issues. You must try your best to ensure your own safety and not cause harm to other people on the road. Therefore, it is first verified in a virtual verification environment. It's very important.

So, what core technologies are needed to realize digital twins?

The first is modeling and rendering. The three-dimensional model is processed to be closer to real life and present a texture similar to the real thing. Then comes simulation. Yes, we finally come to the protagonist of this video. It is actually the core of realizing digital twins. One of the technologies.

If the digital twin is the virtual world built by "The Legend of Zelda", simulation is the various adventure activities that players carry out in the game. Players can choose different paths, adopt different strategies, interact with characters in the game world, and complete various tasks and challenges. The simulation process simulates behaviors and scenarios in the real world. Players can achieve success in the game world through experiments, observations, and decisions in the game.

Specific to intelligent driving simulation, it is to establish a vehicle model and digitally restore the vehicle driving scene to establish a system model as close as possible to the real world, so that intelligent driving systems and algorithms can be tested through software simulation.

The simulation process is roughly like this. An intelligent driving vehicle equipped with the initial version of the algorithm is driving in the simulation system. The system will simulate various road conditions. When the algorithm response does not meet expectations, the system will save the log at that time, and the R&D personnel will target these Problems are analyzed and solved, and reproduced and verified with the support of simulation, so as to iterate the algorithm and improve reliability.

So what exactly is meant by simulation? What kind of input should be constructed?

The first is to restore static traffic elements that are consistent with the real world, such as roads, traffic signs, guardrails, trees, buildings, etc. Currently, most intelligent driving simulation software or platforms use 3D modeling software to create a "material library", use vectorized graphics of high-precision maps to reconstruct road elements, and then use professional software to add buildings, trees, terrain and other elements. Static elements.

△Picture from 51Sim

With a static environment, there are also dynamic scenes.

The generation of dynamic scenes includes two aspects: one is microscopic pedestrians, vehicles, and weather; the other is the construction of macroscopic traffic flow scenes.

How to automatically generate realistic and logical traffic participants is a difficult point.

At the same time, dynamic elements such as weather changes and lighting changes need to strictly follow the physical laws of the real world. Here, the game engine can play an important role. Its ability to simulate real scenes such as scene rendering and physics engines is better than traditional simulation. The software is much more powerful.

△Picture from 51Sim

Therefore, many of the latest intelligent driving simulation platforms are developed based on game engines, such as Microsoft AirSim, Tencent TAD Sim, 51sim, etc., which are all developed based on the Unreal engine. Friends who are interested in game engines, please click: When a game engine is "rolled" into an electric car, something wonderful happens | Smart Car Planet, we have also put the video link at the end of the article, welcome to watch.

After talking about the static and dynamic environment modules, let’s take a look at the sensor module.

As the "eyes" of intelligent driving vehicles, sensors are used to sense the external environment, discover and classify obstacles, predict speed, and assist in accurately locating the environment around the vehicle.

From a simulation perspective, no matter what kind of sensor, it can theoretically be simulated at three different levels: simulating the physical signal, simulating the original signal, and simulating the sensor target.

The camera directly simulates the optical signal detected by the camera, and the radar directly simulates the sound wave and electromagnetic wave signal, which is physical signal simulation. The original signal simulation is to remove the sensor detection unit, because there is a special digital processing chip in the control electronic control embedded system, which can Directly simulate the input unit of a digital processing chip. Sensor target simulation, that is, if sensor perception and decision-making are divided into two different levels of chips, then the ideal target detected by the sensor can be directly simulated to the input end of the decision-making layer algorithm. Generally speaking, it is relatively easy to achieve target-level simulation and provide true values ​​through software simulation. However, the simulation of original signals, especially physical signals, requires the use of a large number of simulation equipment, which is relatively complicated.

The earliest and most complete part of vehicle simulation is the dynamics module. During the intelligent driving simulation process, it is necessary to use the vehicle dynamics model to objectively evaluate the decision-making and control algorithms.

Traditional commercial simulation software is very mature in this field. Generally, the body model, tire model, braking system model, transmission system model, etc. of the actual test vehicle are parameterized, and appropriate parameters are configured according to the vehicle's dynamics module to simulate Calculate the vehicle's capabilities and limits under the control of the intelligent driving system.

In addition to the main modules of intelligent driving simulation mentioned above, there are also modules such as high-precision maps and simulation interfaces. Each module has some key simulation issues that need to be solved.

In general, how to make simulation data have the authenticity and richness of the real world is the biggest challenge. At present, the substitutability of intelligent driving simulation for real testing still faces some doubts. I believe that with the improvement of system standards and technological advancement, Iteration and simulation will bring greater assistance to the advancement of intelligent driving.

We will also launch relevant content in the future to talk about the establishment of standards for intelligent driving simulation, representative companies, etc. If you have other points of interest, please leave a message and tell us.