Automotive 3D radar is gaining popularity, so why is 4D imaging coming out?

On the evening of February 12, a Tesla Model S suddenly lost control again and crashed into a semi-underground balcony of a residential building at a speed of 80km/h in a residential area near Beijing Aerospace Bridge. Elon Musk must have a way to deal with these "invisible" behemoths, right? This article will mention his method below.

It is common to ignore large objects

It’s an exciting time for radar in automotive applications. Of course, the radar here mainly refers to 3D millimeter wave imaging radar. In 2019, its installation volume increased by 44% year-on-year, eroding the market for LiDAR and ultrasonic technologies in ultra-short range due to its use in more situations. The development of China’s automotive millimeter wave radar market is also entering the fast lane.

Technological development is always endless. Some manufacturers have begun to release new automotive  4D imaging radars , but there are very few products that can be mass-produced. People may ask, what 4D can do, can't the current 3D radar and camera already do it? For example, applications such as cockpit monitoring perception. Little do people know that 4D has its own reasons. Let's first look at the differences between 3D radar and 4D radar, and then look at the industry's expectations for market development.

1. The evolution of automotive radar

Semiconductors used in automotive radars have evolved over several generations and are still evolving. The first generation used GaAs chips mounted directly on a circuit board and connected with wires. The next generation is SiGe chips, which integrate more functions on-chip. Many companies are now developing silicon CMOS and SOI technologies. Many companies use a 40nm technology node, but some companies have reduced it to less than 28nm. Small nodes combined with CMOS technology allow for higher functional integration within the chip.

Currently, the latest generation of radar chips integrate not only transceivers and chirps, but also microcontrollers and digital signal processing (DSP) units. This points the way for single-chip radar solutions that support MIMO antennas. Turning to silicon technology will also better maintain a cost reduction route, especially more suitable for mass production.

Packaging and board technology are also evolving. The first generation of devices consisted of multiple die mounted directly on a circuit board and connected via wire bonds. These radar modules also had two separate boards: one for RF and the other for digital functions.

As packaging has advanced, various forms of wafer-level products have emerged. Boards have also evolved to blend with top RF layers composed of special low-insertion-loss materials such as ceramic-filled PTFE or similar. In cases where small antenna arrays are sufficient, antenna-in-package (AiP) designs have begun to be adopted, and some have proven suitable for short-range automotive applications.

Traditional radar has the ability to measure in two dimensions. 2D radar uses a dedicated rotating antenna to listen to the echo signal, which can obtain two coordinates and determine the location of the target. Then 3D radar appeared, which rotates like 2D radar, but after each scanning rotation, the elevation angle of the antenna will change in preparation for the next detection. In this way, 3D radar can detect three dimensions: azimuth, elevation and speed.

3D radar principle description 

2.  What are the changes in 4D imaging radar ?

In principle, 4D imaging radar is very different from traditional radar and LiDAR. From the perspective of theoretical physics, time is considered the fourth dimension. When it is applied to radar, it becomes the Doppler frequency, showing whether an object is moving towards you or away.

Some argue that 4D imaging radar is a misnomer and marketing ploy, as these radars are not truly mapping time, but rather using time to understand the 3D environment; moreover, modern reconnaissance radars already have the ability to detect Doppler shift, so the technology is nothing new.

However, 4D radar integrates the fourth dimension of measurement into 3D radar to better understand and map the environment. Even if the specific technology is not new, its integration is novel and worth studying.

Main functions of 4D imaging radar

Phil Magney, founder and president of technology consultancy VSI Labs, explains it this way: "I think time should be used as the fourth dimension, which is what some companies are doing. Honestly, 4D is more of a marketing hype, since the time element is derived from Doppler. So, if that's the case, then traditional 2D can be called 3D."

Phil Magney 

In other words, the time factor has always been key to radar functionality. When asked the same question, Torsten Lehmann, executive vice president and general manager of radio frequency processing at NXP, pointed out that the fourth element of 4D imaging sensors is "lateral resolution." He said: "4D imaging sensors can not only measure distance and speed, but also height, direction and angle of arrival, while identifying objects with higher resolution. 4D imaging radar can identify not only horizontal planes, but also vertical planes. For example, the car can decide whether to pass 'under' or 'over' an object."

Torsten Lehmann

Lehmann continued: “Imagine a car driving on the highway at 80 kilometers per hour, and a motorcycle – a small object with low reflectivity – is approaching from behind at 200 kilometers per hour. Unlike cameras and LiDAR, these improved radars can identify the motorcycle, which is initially far away, and recognize that the two objects are moving at different speeds.”

It can be said that so far, radar is the only sensor that can operate above 300 meters and identify high-speed objects, while cameras and LiDAR cannot handle such long distances and speeds.

With the advent of high-resolution imaging radar, many radar suppliers are eager to promote radar as the only high-speed sensor that can operate in adverse weather and lighting conditions.

What is the primary goal of 3.4D imaging for automobiles?

Radar is a key element of the sensor suite in ADAS and autonomous driving . The technology is already in commercial use, especially for various ADAS functions. Its use will increase in both the short and long term. In the short term, legislation and voluntary safety commitments will further drive adoption. In the long term, higher levels of autonomous driving will increase the radar content per vehicle, creating a multiplier effect on the market. It’s an exciting time for radar technology and many changes are happening.

However, it is a long tunnel from theory to application, and it takes time to see the light. The same is true for autonomous driving, which is usually measured by the "level of automation". In short, this is a scale from 0 to 5, L0 means absolutely no autonomy, and L5 means full unmanned driving function. For reference, most Tesla cars are currently rated at L3, which is considered a good score.

Evolution from L0 to L5

Autonomous driving means safe vehicles, and while moving toward higher levels of autonomous driving is a worthy goal from a purely technical perspective, the value ultimately comes down to saving lives. That’s why adopting and improving radar technology is an increasingly important matter for ADAS developers.

“It’s important to remember that many of the announcements to date have been proofs of concept, many of which are based on general-purpose processors, and that significant improvements will come from radar-specific processors optimized for (automotive) use cases,” said Matthias Feulner, ADAS marketing director at NXP Semiconductors. “At the most basic level, our goal is to avoid accidents and save lives, and ADAS technology and radar sensors in particular can help improve driving safety and avoid casualties.”

4. Musk’s choice

In October 2020, a hacker named "green" who specializes in Tesla software discovered hints of new features in software updates. He reminded consumers that a new radar option called "Phoenix" was added to Tesla's recent software update. Phoenix is ​​the name of the Arbe radar system. It can identify, evaluate and respond to challenges ranging from ordinary to special scenarios through 4D ultra-high-resolution real-time imaging to serve real-world driving needs.

Hackers revealed

Regardless of speed, altitude, distance, size or surrounding weather and lighting conditions, Phoenix distinguishes between real threats and false alarms to keep drivers, pedestrians and other vulnerable road users safe.

Tesla is preparing to update the Model 3's fascia, and the new sensors will be part of the update, one source said.

Musk recently said: "Image recognition today is still based on isolated pictures, which are actually closely related in time. So if you transition to 4D, you add the time dimension to the three-dimensional space, which is essentially video. This architectural change has been in the works for a while and has not really been promoted to any vehicle in mass production, but this is what you really need for full self-driving."

Perhaps this is the best reason why Musk hates using LiDAR sensors in autonomous driving systems and "wouldn't use them even if they were free."

5. One of the hottest trends

In general, 4D imaging radar is one of the hottest trends. In the future, as radar technology moves to smaller nodes, highly integrated packaged single solutions will appear; antenna array sizes will be greatly expanded, enabling better azimuth and elevation resolution, making point clouds denser. Deep learning-based algorithms will also develop in parallel, enabling radar to perform 3D target detection, classification, and tracking. In this case, radar will begin to use some LiDAR technology to improve resolution while maintaining independent advantages in weather and light levels.

It’s an exciting time for radar. Automotive radar is forecast to become a $12 billion market by 2030 with modest price reductions. In the future, 4D imaging radars that provide dense 4D point clouds will enable higher-precision object detection, classification, and tracking. In the first decade, ADAS (L1 and L2) will be the main market driver, while in the second, autonomous vehicles will be the main market driver. In the long term (2030-2040), autonomous mobility (L3, L4, and L5) will drive the market, with increasing radar content.