NOP+: Pursuing a continuous point-to-point driving experience

The era of intelligent driving cannot do without "regulation and control", but for users, "regulation and control" may be a term that is both unfamiliar and has a strong sense of technology.

In this Tech Talk, Blake FAN and Eason QIN from the regulatory control team of NIO’s autonomous driving R&D department, and Simon WANG from the global intelligent driving experience team, will explain to us the development logic of “regulatory control” and reveal how NOP+ enhanced pilot assistance “drives”.

What is a decision planning control algorithm?

The full name of regulation and control is decision-making planning control algorithm, which is referred to as "regulation and control" externally. It is one of the core components of the intelligent driving system.

Aquila Weilai super-sensing system,

It has ultra-long-range high-precision laser radar

33 high-performance sensing hardware including

If the Aquila system is likened to the eyes of the intelligent driving system, then the regulation and control is like the brain of the intelligent driving system. Its responsibility is to drive the vehicle safely and smoothly, allowing users to enjoy a relaxed and pleasant travel experience.

Simply put, the control algorithm determines when the vehicle should give way to oncoming vehicles, when to change lanes, and when to enter and exit ramps; these driving behavior instructions are transmitted to the vehicle control end to achieve delicate, millisecond-level steering wheel angle and acceleration and deceleration control.

Therefore, the intelligence of regulation and control also determines the balance between comfort and traffic efficiency of the intelligent driving system.

How to become an excellent regulatory control algorithm?

For an intelligent driving system to be able to drive as smoothly as a human, at least two conditions must be met.

The first is whether it can obtain information about the surrounding environment like humans; the second is to build human-like thinking and driving behaviors.

NOP+ enhanced pilot assistance function is turned on

Regarding the perception of the environment around the car by the intelligent driving system, you can find the answer in the article "How special is the world in NIO's eyes?" However, in order to build a human-like way of thinking, it is necessary to invest in research and development efforts and achieve at least the following two points.

First, understand what good driving behavior is.

Excellent driving behavior consists of many factors, the most important of which is safety. Vehicles in intelligent driving state need to maintain safe interaction with other traffic participants in complex environments, drive as smoothly as possible, and reach the destination efficiently. Therefore, safety, stability and efficiency have become the development goals of the intelligent driving control module.

Second, build a human-like way of thinking about driving.

This is particularly important. Intelligent driving systems need to imitate the way human drivers think, such as thinking in a hierarchical manner after knowing the driving destination information (navigation), the current location (positioning), and the surrounding environment information (map, perception).

This thinking process can be divided into three stages, such as the decision-making and planning stage of thinking about whether to overtake, give way, or change lanes; the stage of planning a safe, comfortable, and efficient operating trajectory; and the control stage of how the vehicle executes instructions.

Intelligent driving system control workflow

Based on the above three levels of thinking, let’s talk about it in detail.

First, let's break down Behavior Planning. NOP+'s decision planning includes not only the prediction of the behavior of other traffic participants around, but also the impact of the vehicle's behavior on the surrounding environment. The overall modeling of all these factors, both self- and other factors, constitutes NOP+'s decision prediction integration.

It is like a chess game. We cannot predict the opponent's next move with 100% accuracy, but by deducing the current situation of the game, good chess players know how to make a move to get a higher chance of winning. Therefore, the decision-making goal of the intelligent driving system is to judge the "chessboard's 'momentum'" more carefully and deeply, that is, the situation and benefits.

In order to achieve a higher winning rate, smart driving vehicles need to accurately identify and recognize the surrounding environment - similar to the placement of chess pieces on a chessboard; as well as the game relationship between obstacles - how the opponent will make a move after making this move; after thinking is completed, it needs to choose the appropriate response behavior within a limited time - the strategy of playing chess.

In the lane-changing scenario below, after identifying the slow vehicle ahead and the vehicles and environmental information on the left, right, and rear, the NOP+ algorithm will "observe the situation" and deduce the possible consequences of executing different driving behaviors.

This is "game theory". After weighing the pros and cons, the lane change is finally completed in a safe and comfortable manner.

NOP+'s decision-making algorithm needs to consider all obstacles in the environment that may interact. When the environment changes, the factors that need to be considered in interactive decisions will increase exponentially.

For example, at the current moment, there are 10 obstacles that need to be considered for avoidance, so the complexity of the decision is 2^10=1,024; when considering whether to avoid each obstacle at each moment in the next (1 second, 2 seconds...5 seconds), the complexity will become 1,024^5, which is approximately 10^15.

Good drivers are often able to reduce risks and improve traffic efficiency and comfort through predictive driving. Therefore, the deeper the thinking, the better NOP+'s driving behavior prediction.

An excellent chess player will play 5-8 rounds of "game" before making each move, while intelligent driving needs to complete more complex deductions in less than 0.1 seconds. In order to quickly find the optimal solution among more than 10^15 strategies, in addition to stronger hardware computing power, the design and improvement of NOP+ regulation and control algorithms are more important.

In fact, NOP+'s decision-making algorithm uses millions of autonomous driving data for continuous training and learning, summarizes and records "patterns", and thus achieves fast and accurate search in the decision tree. We call this process: Deep Learning Policy Network, which means learning the judgment logic of excellent drivers for driving interactions through deep learning networks.

With the improvement of system hardware capabilities and the accumulation of data, the factors considered in the NOP+ decision-making system will become more refined, so that the best decision can be found in an ever-changing world.

After breaking down decision planning, let's take a look at how motion planning works.

When the vehicle gets the optimal decision, the motion trajectory planning module needs to convert it into a driving path that the vehicle can execute. A good driving path needs to optimize the safety, smoothness and traffic efficiency of the path as much as possible under safe conditions.

For example, in the following congestion and parking scenarios, when planning a reasonable braking force, the vehicle must not only consider the distance to the vehicle in front, but also optimize the change in deceleration as much as possible to achieve safe, smooth and comfortable braking and starting (such as the smooth blue acceleration and deceleration curve in the lower left corner of the Gif image).

Good driving is to strike a balance between efficiency and comfort while maintaining safety and following traffic rules. However, the balance between efficiency and comfort is different in different scenarios.

For example, roads in Beijing and Shanghai are very different from rural roads. In a congested environment, ordinary urban roads are very different from ring roads and elevated roads. Therefore, it is necessary to use data analysis to gain a deeper understanding of the surrounding environment and adaptively adjust the balance among many factors.

For example, in the congested car-following scenario of Banyan 2.0.0, the system learns the driving style of experienced drivers through a large amount of data, thereby obtaining a safe, smooth and comfortable driving experience. As the data accumulates, the experience can continue to evolve.

The last level is the control of the command. After the system makes a motion trajectory plan, it needs to control the vehicle to complete the corresponding action according to the prediction. "Control" will systematically consider the vehicle's motion state in a closed loop to better execute the command and achieve "hand-brain consistency".

NOP+: Pursuing a continuous point-to-point driving experience

Traditional assisted driving research and development often takes a single function as the starting point (such as the familiar ACC adaptive cruise control, LKA lane keeping assist, LCC lane centering assist, etc.) to achieve assisted driving in specific scenarios.

However, does it mean that smart driving can be achieved if every single function is done well? The answer is no. The reality is much more difficult than imagined!

First, real-world scenarios are often more complex and require a combination of multiple single-point functions. Imagine human driving behavior. When changing lanes, if there are vehicles in the target lane, you must not only consider whether there is enough space for changing lanes, but also consider the future movement trajectory and speed changes of the vehicles in front and behind.