How to improve autonomous driving through sensor fusion technology

As ADAS technology expands to time-sensitive critical applications such as emergency braking, forward collision warning and avoidance, and blind spot detection, it can provide reliable decisions in real time by combining data from multiple sensors, resulting in a safer autonomous driving experience.

Using AI-assisted cameras to identify road signs and keep the vehicle within lane markings, it enables smarter and safer driving. But what happens when fog affects both the camera’s view and the driver’s view?

"Cameras may be powerful in object recognition, but they are not as effective in bad weather or at night," said Miro Adzan, general manager of advanced driver assistance systems (ADAS) at Texas Instruments. "Radar, on the other hand, can continue to work in rain, snow or light fog. Driver assistance systems need to integrate multiple different sensors so that the vehicle can take full advantage of the advantages of these different technologies when driving."

With the advantages of different types of sensors, not only can you switch between different conditions or applications, but even in clear weather, the camera will be able to capture the details of objects more accurately, while the radar will measure the distance of objects more accurately.

As the use of these systems expands into critical, time-sensitive applications such as emergency braking, forward collision warning and avoidance, and blind spot detection, design engineers need to integrate these disparate sources of information into a single picture from which to make reliable decisions in real time.

“For automated parking, you need to combine data from cameras, radar and sometimes ultrasonic data to get an accurate picture of what’s going on around the vehicle,” said Curt Moore, general manager of Jacinto™ processors at Texas Instruments. “Each of these sensors alone is not accurate enough, but when used together, you get a much more accurate picture of the space around you. This allows you to park safely in tighter spaces.”

The proliferation of automotive sensors

Advanced safety systems are no longer just for high-end cars. Nearly 93% of cars produced in the United States are equipped with at least one ADAS feature. By September, 99% of new cars in the United States will come standard with automatic emergency braking.

This shift is driven by the decreasing cost and size of sensors, such as the TI mmWave radar sensor, which integrates the entire radar system into a chip the size of a coin.

“Ten years ago, radar was used primarily in military applications because of size, cost and complexity,” Miro said. “Now, radar is about to become standard equipment in cars.”

While the proliferation of affordable sensors has opened up new applications, it has also created new challenges for ADAS engineers, who need to design systems that aggregate and efficiently process all the data streams while meeting stringent price/performance and power constraints.

Communications Challenges

In single-sensor ADAS systems, data from object detection can be pre-processed near the sensor so that the information is immediately available. However, sensor fusion technology requires that raw, high-resolution data be immediately transmitted to a central unit for processing to form a single, accurate model of the environment to help the vehicle avoid collisions.

“With all the data coming from these sensor nodes, you need to make sure it’s all synchronized so the vehicle can sense what’s happening around you and make critical decisions,” said Heather Babcock, general manager of FPD-Link™ products at Texas Instruments. “To transmit synchronized data in real time, high-bandwidth, uncompressed transmission is necessary because compressed data introduces latency.”

Originally created to carry digital video streams from graphics processors to digital displays, the FPD-Link communications protocol is now designed to transmit large amounts of uncompressed data over several meters using simple, easy-to-route cables.

“You have a standard protocol on one end, along with the FPD-Link serializer, which has very secure and reliable proprietary encoding capabilities to convert the data stream,” Heather said. “This is matched up with a paired deserializer on the other end, which reconstructs the data into its original format and transmits the data over a variety of other interface protocols that are supported in the TI portfolio.”

Enable more effective decision making

Once this data reaches a central processor, it is often combined into a unified model of the car’s surroundings using computationally intensive signal processing and deep learning algorithms, increasing the power input and heat output required.

Because the physical constraints of a car place strict limits on the size and weight of the battery and cooling infrastructure, ADAS engineers need processors designed specifically to perform these tasks as efficiently as possible.

TI's Jacinto processors combine dedicated digital signal processing (DSP) and matrix multiplication cores to operate at the industry's lowest available power, even at temperatures as high as 125 degrees Celsius.

“There are huge advantages to integrating the DSP and processor into a system-on-chip,” Curt said. “When used separately, each requires its own memory and power supply, which increases system cost. Another advantage is that by integrating such operations into one chip, latency is reduced.”

In addition to energy-efficient processors, TI’s automotive-grade power management ICs have functional safety features for sensor fusion, front cameras and domain controllers, improving the vehicle’s overall power efficiency and functionality.

Beyond individual devices, TI’s entire ADAS product ecosystem is created for seamless compatibility, allowing automakers to choose from a holistic portfolio that scales to suit the needs and price points of their vehicles.

“We take into account the challenges of vehicles in all ADAS blueprint design stages,” Miro said. “This makes system design much easier for our customers.”