Will lidar replace cameras and become the "eyes" of cars?

Perhaps, there is no need to be led astray by Tesla’s “stubbornness”.

2021 is considered the first year of mass production of LiDAR. The reason is simple, because a batch of mass-produced models equipped with LiDAR will be launched this year.

Great Wall Motors' WEY brand Mocha model will be equipped with a fully solid-state laser radar and is expected to be officially launched in March this year.

As early as when the P7 Pengyi version was launched, Xiaopeng Motors announced that it would equip its third mass-produced model to be launched in 2021 with laser radar.

NIO's first sedan ET7 will also be equipped with lidar, and SAIC Zhiji Auto will also reserve an interface for lidar. Multinational automakers including BMW, Daimler, and Volvo all have plans to equip their vehicles with lidar in 2021.

In contrast, Tesla is out of place. Musk once compared LiDAR to "a normal person using a cane", which is really unnecessary.

Of course, it is completely understandable that Musk puts LiDAR under the consideration of cost priority. It is not ruled out that when the cost of LiDAR is reduced, Tesla will also choose the auxiliary solution of LiDAR.

From a business perspective, solving problems at the best cost is the main point of contradiction that Tesla is used to grasping.

This actually confirms the two prerequisites for the mass production of lidar: 1. The technology must be mature; 2. The cost must be reasonable.

However, as of now, Musk still "hates" LiDAR. As a result, a "two-party dispute" has emerged in the route to L3 and above autonomous driving.

The first group is led by Tesla, and is almost the only "visual group" that insists that cameras are sufficient to solve perception problems.

The other group, mainly the laser radar group of many new car manufacturers, insists that it is very necessary to add safety redundancy by equipping with laser radar, and to push the "weak perception" of autonomous driving that relies solely on vision to "strong perception".

The dispute between the two factions is actually a "dispute of perception", which is about who will be the "eyes" of the car.

What are the limits of visual perception?

The visual perception logic is close to that of the human eye. It uses cameras to capture images and identify road conditions.

But it is obvious that the picture captured by the camera is actually a 2D image, and we need to use algorithms to solve the problem of "what is captured".

To describe it in more professional terms, it is necessary to segment the image, classify the objects, perform target tracking, and calibrate the model in order to realize the recognition and matching of obstacles.

最显著的优势在于可获取的信息丰富，颜色、纹路皆可保留。在实际场景中，车道线、指示牌、红绿灯等均是其识别强项，并可以对物体做出清晰分类。

However, visual solutions are highly dependent on sample data and require large amounts of data for training, continuous optimization, and continuous learning.

It’s like a kindergarten kid. Every time you teach him something new, he will remember it and be able to recognize it the next time he encounters it. He will gradually move up to elementary school, junior high school, and then mature step by step.

Moreover, since the camera captures 2D images, but the real scene should be 3D data, it is necessary to use image processing algorithms and high-computing chips to complete the conversion. However, there are certain errors in the algorithm conversion, which leads to the spatial distance measurement of the visual solution not necessarily being accurate.

A complementary solution is to use binocular or multi-cameras, that is, to use the difference in viewing angles of the cameras to restore 3D data, similar to our two eyes' perception of the surrounding space.

However, the relevant technology still needs to be further improved, because the algorithm error of a monocular camera may be doubled on a multi-camera, resulting in a greater deviation from the actual distance.

The visual solution also needs to overcome another challenge. The camera is also highly dependent on the lighting environment. In environments with poor lighting and bad weather, the perception ability will be greatly reduced.

Given that cameras have so many disadvantages, why is Tesla so determined to take the pure vision route?

One reason is low cost. The cost of a monocular camera is between 150-600 yuan, while the cost of a more complex tri-camera can be controlled within 1,000 yuan. For Tesla, which prioritizes cost, cameras are definitely the first choice.

Another reason is that Tesla is very confident in its own algorithms and can create a very high technical barrier with "easy-to-obtain cameras and hard-to-get algorithms."

More importantly, Tesla has built a complete set of data collection and learning cycles at the beginning of product design.

Every Tesla on public roads is a "data collector". Each car can collect all-round data about the surrounding road conditions through its onboard camera, and then upload it to Tesla's cloud, eventually forming a huge and real data pool.

In addition, Tesla develops its own chips and algorithms, and its hardware and software can be optimized simultaneously. Compared with car companies that do not have the ability to develop their own chips, Tesla has greater potential for exploring chip computing capabilities and re-optimizing algorithms.

Moreover, the learning cycle requires a "closed loop", which needs to be completed by the "shadow mode". Tesla can continuously collect road conditions information and user driving behavior and compare it with its own decisions.

When it is found that the user's actual driving behavior is inconsistent with the system's judgment, the scene will be transmitted back to the cloud for algorithm correction. This cycle continues to improve the accuracy of the algorithm's decision-making.

Is LiDAR indispensable?

What do visual designers worry about the most? Corner cases. Corner cases may appear completely different in different places.

For example, the traffic conditions in China and the United States are very different, including roadside signs, traffic light styles, lane lines and driving rules. The algorithm optimized based on the US road conditions may not be suitable for the Chinese market.

Moreover, even in the Chinese market, the regional traffic environment is different between the south and the north, and between the east and the west. For example, on some county-level roads, you may encounter slow-moving low-speed electric vehicles and trucks carrying saplings. This places higher demands on the richness of data and localized adaptability.

LiDAR is a powerful sensor, and its principle is easy to understand. By emitting laser light and calculating the time it takes to reflect back, it can measure the distance and speed of obstacles and build a 3D model of the three-dimensional space. In very advanced LiDAR technology, it can even analyze the composition of materials, and it is no problem to use it to identify lane lines.

Waymo self-driving cars equipped with lidar

Since it can accurately measure distance, the introduction of LiDAR can reduce the difficulty of algorithm analysis in visual solutions. Even in some corner cases, obstacle avoidance can be completed, which actually adds another "lock" for safety.

LiDAR point cloud image

There is another question. It is also a radar. Why doesn’t the relatively popular millimeter-wave radar work?

The biggest problem is that the millimeter-wave radar's ranging accuracy is not enough. On the highway, millimeter-wave radar has difficulty detecting static objects, including relatively small objects.

Moreover, millimeter-wave radar lacks the ability to detect in the longitudinal space and cannot obtain a three-dimensional ranging space unless there is a breakthrough in 4D high-resolution millimeter-wave radar technology.

Millimeter wave radar diagram

Looking at the autonomous driving solutions of many domestic car companies, most of them choose Mobileye Q4 chip. Mobileye currently still mainly relies on the perception method of "camera + millimeter wave radar".

Even if LiDAR is introduced in the future, cameras will not be abandoned for a long time. Moreover, LiDAR is not omnipotent. In bad weather such as heavy rain, noise removal is also a problem.

LiDAR, camera, and millimeter-wave radar should be integrated into one another. LiDAR enhances perception, which in turn enhances safety redundancy, and is actually a supplement to visual perception.

Schematic diagram of the camera and lidar fusion process

At present, there are three major challenges facing LiDAR: cost, size, and automotive grade, with cost being the most acute. 2020 is the first year of mass production of LiDAR, also because of the reduction in cost.

As Huawei once revealed, it plans to develop 100 lines of products in the short term and reduce the cost to US$200 or even US$100 in the future.

DJI also announced in August last year that it could mass-produce automotive-grade lidars priced in the thousand-yuan range. Its products, Horizon and Tele-15, are priced at 6,499 and 9,000 yuan respectively.

The era when laser radars cost tens of thousands or even hundreds of thousands of yuan is probably gone forever.

Technology integration will eventually become a major trend

The industry has undergone dramatic changes. The new decade has just begun, and the frequency of mentioning electrification and intelligence is increasing. As the economy develops and the population ages, the entire society's demand for automation will also increase day by day. And autonomous driving will be an important sector.

However, autonomous driving must follow two principles: safety and experience. Consumers will be very direct in their evaluation of autonomous driving. At least, autonomous driving should be better and safer than human driving.

If we think more deeply, the battle between cameras and lidar is actually just autonomous driving technology at the vehicle level. In the future, more diverse intelligent technologies beyond vehicles will be integrated into it, such as high-precision maps, high-precision positioning, V2X, and more emphasis on vehicle-road collaboration, which is the coordination and unification of the entire transportation system.