Pure vision vs. LiDAR, the game of Tesla's autonomous driving

Xiaopeng NGP officially pushed NGP (Navigation Guided Pilot) high-speed autonomous navigation driving technology through OTA, which is similar to Tesla's NoA and Weilai's NOP automatic navigation assisted driving function. It is also the third car in China that has the ability to actively change lanes and enter and exit ramps in addition to autonomous cruising. In addition, the traditional car company Great Wall WEY also announced that it will launch a Great Wall version of a similar function NOH in March. This automatic navigation assisted driving function is also regarded as one of the important signs of surpassing the current mainstream L2.

Tesla, which first launched similar features, launched a beta version of its autonomous driving system FSD Beta to a small number of users last year, showing us a lot of capabilities that have not been pushed to a large number of users. Just as Tesla's competitors began to follow suit, FSD Beta has quietly evolved for several months.

Tesla's NOA is secretly doing Beta testing

Since FSD Beta is still only available to a small number of car owners, close to 1,000 people, mainly in North America, we can only witness the capabilities of FSD through their experience videos. At present, FSD Beta has been upgraded to the latest version 2020.48.35.7 through OTA. As soon as it appeared, it surpassed the current L2 autonomous driving capabilities and hardcore visual style, making its every upgrade attract much attention.

Let's first briefly list some of the things that FSD Beta can do that we currently rarely see in China:

- If there is a vehicle parking sideways ahead, the system will slow down and wait;

- Avoid bollards, parked vehicles occupying the roadside, and vehicles from the opposite lane that are encroaching on your lane;

- If a vehicle is approaching from the side or rear lane and is too close, the car will actively dodge slightly to the other side within the lane;

- If a lane is closed due to construction ahead, the car can automatically change to another lane even if there is no obvious sign, and change back after the construction area is over;

- In the city, the car will change lanes about 1 mile in advance to prepare for a turn, and will pull over in advance even if there is no clear lane line;

- If there are two turning lanes, the inner lane is used for smaller turns and the outer lane is used for larger turns, without affecting vehicles in adjacent lanes;

- In addition to automatic stopping at red lights, automatic stopping/starting at stop lines, automatic observation and turning at intersections (with prompts on the screen), etc., it can also give way to pedestrians who run red lights, and turning vehicles that violate the rules by not giving way to straight-moving vehicles;

- In a single lane, tentatively turn left to observe the road conditions of the oncoming lane and overtake quickly;

- The car can drive autonomously on mountain roads without relying on lanes or the vehicle in front. Even at night when there are no street lights, it can correctly choose irregular intersections, flexibly avoid parked vehicles on both sides of the road, and automatically turn right after passing a parked vehicle;

- When encountering a speed bump, although it is not displayed on the screen, the vehicle will significantly reduce its speed to pass through it. However, it will not be able to slow down like other vehicles around it due to the obvious bumps on the road.

These are some very daily but very human-dependent scenarios. When Tesla FSD handles various evasive actions, steering, acceleration and deceleration, they are smoother and not slower at all. They are basically highly consistent with our own driving behavior and are almost unaffected by weather conditions such as rain and night. In the United States, there are very clear orders and rules for crossing intersections. Sometimes we ourselves may not observe them properly, not carefully observing vehicles in other directions and not seeing the stop sign. NOA "sees" vehicles in other directions at the intersection for the car owner and gives way.

In addition, in various scenarios, the left side of the central control screen will display the scene observed by the camera in real time, and the display area occupies most of the area, including all vehicles traveling in front, behind, left and right, vehicles and pedestrians parked on the roadside, as well as trash cans, pile barrels, shoulders and other roadside objects that can be detected, and it can automatically or manually zoom in/out to allow you to understand its perspective simultaneously.

Therefore, from our current perspective, the comprehensive strength of FSD Beta is much stronger than everyone else. But the reason why it is a Beta version is that there must be a test version with problems, and it is definitely not perfect. For example, at intersections without traffic lights or even stop signs, the system will be too cautious and will not dare to turn or pass. It may even be stuck at the intersection for several minutes due to dense traffic. For example, for pedestrians or bicycles that appear illegally at intersections, the system will often be very careful to slow down or stop and wait.

It is of course necessary to consider safety, but it may still have some impact on other traffic participants on the road.

Two approaches, two games

The powerful capabilities of FSD Beta have made many people look at Tesla's pure vision + algorithm approach differently, which actually reflects the difference between Tesla and other automakers in the way of autonomous driving. In China, everyone generally adopts another approach. For example, independent brands and new forces have gradually begun to adopt a comprehensive solution of camera + high-precision map + lidar.

Tesla is obviously still using a pure vision solution. For example, the Model 3 now has three front-view cameras, four side-view cameras, and a total of eight rear-view cameras. The benefits of this are obvious. The cost of vehicle autonomous driving hardware can be controlled lower, and Tesla's huge database, precise algorithms, and powerful chips can also have strong capabilities. Musk has also expressed his dislike for lidar more than once in the past.

At present, FSD Beta already has all the very powerful capabilities we mentioned above. One of the reasons why it relies only on pure visual solutions + software algorithms is that Tesla has always used real road data for training. Nothing is more effective than practicing in a real environment, which is something that almost all other autonomous driving R&D companies cannot compare to. The reason is simple. No matter how realistic the simulation is, even if the test car is driven directly on the public road, it is still a test situation, which is always different from the real situation.

In daily driving, we may encounter various unpredictable emergencies, such as pedestrians suddenly appearing, fire hydrants suddenly exploding, and trucks overturning and goods scattering all over the ground. Just like in the movie "The Bankers", when imitating Italian Job, the sudden appearance of a bicycle disrupted the entire plan. Although Tesla's FSD, including ourselves, cannot encounter all unexpected situations, the experience and handling capabilities gained from these real cases and unexpected situations that have been encountered are unmatched by others. This is also one of the reasons why even though FSD is not omnipotent, it is almost the best performer.

However, because it is only a Beta test version, domestic car owners cannot experience it yet, so the current domestic NoA does not represent Tesla's strongest capabilities. However, according to the Q4 conference call, it is expected that FSD can be pushed in China this year, and it is stated that the potential of FSD has not yet been fully developed, and the computing power fully supports fully autonomous driving.

On the other hand, the focus of Tesla Autopilot's research and development forces, including researchers, road testing, and data analysis, is still in North America. No targeted testing has been conducted on the more unique and complex conditions of domestic roads. Just as the scenarios mentioned earlier where FSD performs poorly are very common on domestic roads, the level that Tesla FSD can ultimately achieve in China still needs to be developed.

On the other hand, NIO's latest ET7, Xpeng's new P5 and BAIC's new HBT have all announced that they will be equipped with lidar, which is the comprehensive solution of camera + high-precision map + lidar mentioned above.

The biggest limiting factor for this more advanced sensor solution was cost. However, as car companies have begun to deploy lidar on mass-produced cars this year, this year is likely to be the first year for lidar to be installed in cars in the domestic market. This also means that the cost of lidar has gradually dropped to an acceptable range. This is bad news for Musk's pure vision solution, as its cost advantage is not so obvious.

With the support of LiDAR, in theory, as long as the algorithm and logic keep up, the effect will be more perfect than pure vision. More advanced sensors can also make up for the shortcomings of software algorithms to a large extent, or reduce the requirements and standards of algorithms. However, this solution has high requirements for vehicle computing power and energy consumption control, which may affect the endurance of an electric vehicle. It may even require adjustments to the battery layout and three-electric system from the beginning of research and development. These are also issues that need to be considered after installing LiDAR.

But at least for now, Tesla still believes in its cameras. Not long ago, Musk replied to netizens on Twitter that they are gradually covering all neural network algorithms on the surround view images of 8 cameras to achieve autonomous driving capabilities far beyond human capabilities. Although it is very complicated to integrate the entire large amount of labeled and inferred data into the surround view vision, the advantage is that no hardware needs to be replaced. In other words, this may be another capability upgrade that can be achieved through OTA, and Tesla's visual solution is still evolving rapidly.