Millimeter wave radar, launching an attack on high-resolution imaging

With the advent of the era of intelligent driving, automotive sensors have entered a period of rapid development, among which automotive millimeter wave radar has almost become the standard for intelligent driving at L2 and above. Statistics show that in 2020, the number of front-mounted angle radars installed in the Chinese market was 4.1428 million, a year-on-year increase of 72.53%; the number of forward millimeter wave radars online was 5.3572 million, a year-on-year increase of 38.43%.

From the perspective of physical properties, compared with optical sensors such as lidar and cameras, millimeter-wave radar is not restricted by light and can work normally in bad weather such as rain, fog, and snow, winning the title of "all-day and all-weather" in the field of vehicle-mounted sensors. reputation for work.

Image source: Xingyidao

However, in the face of high-level autonomous driving's demand for high-precision perception, traditional millimeter-wave radars are beginning to appear to be somewhat "inadequate". In May 2021, Tesla, the representative car company in the field of autonomous driving, announced that it would cancel the millimeter-wave radar of Model 3 and Model Y manufactured for the North American market and replace it with Tesla Vision (a pure vision solution based on cameras) to implement driving assistance functions such as Autopilot. This incident caused a lot of controversy for millimeter wave radar.

Since then, a lot of public information has shown that the millimeter-wave radar Tesla chose at the time had performance problems, and the main reason for the problem was the lower resolution of traditional millimeter-wave radar.

"Intelligent driving requires better millimeter-wave radar." Zhao Jie, CEO and CTO of Xingyidao, said at the GTM2022 Global Technology Travel Summit.

In the past, millimeter-wave radar was considered an auxiliary sensor, and its product positioning and main function was to achieve collision warning. Now, as autonomous driving evolves to higher levels, the performance of millimeter-wave radar also needs to be improved, such as elevation resolution capabilities. And high resolution, longer range, easy to integrate with vision, and can use radar data sets to train AI. The improvement of these four capabilities means that millimeter wave radar can, like vision, have the ability to accurately detect, track, identify and other imaging systems around it.

High-precision imaging components of millimeter-wave radar

So, what is the imaging system composition of millimeter wave radar? How does it achieve high-precision imaging?

Zhao Jie introduced that in addition to the basic theory, the imaging system of millimeter wave radar mainly includes three parts: visual semantic constrained imaging, scattering mechanism constrained imaging, and combined neural network constrained imaging. On this basis, millimeter wave independent imaging can be achieved. and in-depth integration with other sensors.

Regarding the definition of "vision", in 1982, the founder of computer vision pointed out in the book Vision: "Vision is to determine what is where by looking." Visual semantic constrained imaging of vehicle-mounted millimeter wave radar is "determining what is where from millimeter wave images."

Specific to the application scenario, vehicle-mounted millimeter wave radar needs to solve two things: one is the scene, object, category, and their relationship, that is, whether it can distinguish cars, motorcycles, bicycles, pedestrians and their similar types; the other is It is where they are, that is, about the geometric primitives and relationships of the scene, including roads, bridges, lines and surfaces in space, and their positions and orientations.

Zhao Jie said that due to insufficient angular resolution, traditional vehicle-mounted millimeter wave radar does not achieve complete visual semantic interpretation and does not constitute a vehicle-mounted millimeter wave radar imaging system. This is the fundamental reason why traditional vehicle-mounted millimeter wave radar has not entered the field of high-end sensor fusion technology. .

Scattering mechanism constrained imaging is a way for radar to explain targets and results through the interpretation of electromagnetic waves and electromagnetic fields. That is, by constructing the scattering mechanism, a sparse representation model of the scattering coefficient in frequency, angle, and space is established. When the vehicle-mounted millimeter wave radar signal is transmitted to the subsequent processing process through analog-to-digital conversion, the signal can be detected, tracked, and processed.

Zhao Jie specifically pointed out that in order to form an imaging system for vehicle-mounted millimeter wave radar, a semantic segmentation algorithm must be added. This is an additional part of the existing radar support system and is also a higher requirement for vehicle-mounted millimeter wave radar put forward by intelligent driving technology.

In addition to the above three parts, supporting an imaging system also requires the construction of a test and verification system. "In addition to the radar itself, the test and verification system must also combine many factors. Because it is a vehicle-mounted radar, it must meet a variety of regulations and requirements to better support the testing and verification of the imaging system itself," Zhao Jie said. arrive.

4D millimeter wave radar is accelerating its implementation

At present, the industry's progress in vehicle-mounted millimeter-wave radar imaging technology is mainly divided into two types. One is commonly known as 4D imaging (that is, 4D imaging based on the FMCW system that adds height-dimensional information), and the other is vehicle-mounted SAR imaging. . Xingyidao uses these two types of technologies at the same time and has achieved certain results.

According to Zhao Jie, Xingyidao 4D radar is more used in forward imaging, while SAR radar is used in lateral imaging. "For 360-degree imaging around the car, we have a variety of technologies. In principle, 4D can cover 360 degrees, but we also propose SAR imaging from the side, because SAR imaging will be relatively high in terms of effect, cost, and cost-effectiveness."

It is understood that the main principle of the SAR imaging system is to use the motion of the car to form a large antenna aperture in the space to achieve high resolution. Xingyidao is the first company in China to propose vehicle-mounted SAR radar. It started research and development in 2016. In 2019, it has realized an engineering prototype of SAR radar. In 2020, two papers related to vehicle-mounted SAR applications were accepted by IEEE. In 2022, four papers have been accepted. Patent authorization for five algorithms.

"SAR radar continues to receive a lot of recognition. We have an overseas customer and a large domestic passenger car customer, and they are all in the process of testing and productization." Zhao Jie said.

In terms of 4D millimeter wave radar, it is reported that in 2020, Xingyidao undertook a research and development project of the Beijing Municipal Science and Technology Commission, and the test results at the end of 2022 were satisfactory. In the same year of 2022, Xingyidao also launched a new product - Galileo 300, which is a 4D mid-range radar for L2 and L3 intelligent driving applications that realizes altitude information output. It adopts industry-leading super-resolution algorithms and supports Radar has outstanding advantages in real-time judgment.

It is worth mentioning that when solving forward imaging, especially when the industry is concerned about improving angular resolution, it is difficult to implement due to the size and cost limitations of millimeter wave radar. Xingyidao proposed a compressed sensing super-resolution algorithm without increasing hardware cost and size, providing an effective method for constructing millimeter wave imaging systems.

Zhao Jie explained that compared with beam forming and subspace algorithms, the biggest feature of the compressed sensing algorithm is that it breaks through the limitations of the traditional sampling theorem and performs fuzzy-free reconstruction, that is, under limited hardware constraints, there are fewer sampling points. The computing speed is faster and takes up less hardware resources.

Judging from the product parameters of Xingyidao's self-developed 4D millimeter wave radar, by adding a compressed sensing algorithm, the angular resolution of the 4D radar can be increased to 0.6×0.8 degrees (engineered), which is at the leading level in the industry. Through simulation calculations, the next generation compressed sensing algorithm can achieve an angular resolution of 0.1×0.1 degrees.

"Judging from some preliminary technical feasibility that has been done so far, millimeter wave radar can achieve the same performance as lidar in terms of angular resolution. This is the technical trend of 4D millimeter wave radar in the future. Looking at this technology from a global perspective It is accelerating its implementation.” According to Zhao Jie, the 4D radar product ALRR300 it launched has obtained overseas designation and will be delivered this year.

Finally, Zhao Jie said frankly that the anxiety behind the issue of which lidar or millimeter-wave radar is replacing which is currently focused on in the industry is actually the issue of resolution. Therefore, for millimeter wave radar, whether it is 4D radar or other new technology routes such as SAR radar, high-resolution imaging must be achieved as soon as possible in order to occupy a place in the market.