Xilinx explains the pain points and solutions for ADAS hardware development

The complexity of advanced driver assistance systems (ADAS) is growing exponentially as more sensors and algorithms are combined to provide greater functionality and safety. Many advanced computing chips, including FPGAs, are being used to solve these problems.

I spoke with Willard Tu, senior director of automotive at Xilinx, about the challenges and best practices in developing ADAS and autonomous driving systems. In particular, we discussed Xilinx's automotive products and its Adaptive Compute Acceleration Platform (ACAP).

Willard Tu, Senior Director of Automotive Business, Xilinx

Xilinx's automotive products address the unique needs of advanced automotive systems and applications? Can you briefly describe the company's history in this space?

Xilinx has been working with automotive customers and partners for more than 20 years, starting with our devices for infotainment and driver information systems. In the early days, Xilinx FPGAs were primarily used for robust, low-latency connectivity and data (audio/video) processing. In 2011, we launched our first System-on-Chip (SoC) device.

Xilinx SoCs combine automotive microcontroller architectures (e.g., enhanced CPUs and peripherals) with scalable programmable logic. This makes Xilinx devices the centerpiece of automotive ECU design and enables developers to create unprecedented optimized designs by effectively partitioning between embedded software and custom hardware acceleration.

"FPGA architectures create unique value in ADAS and autonomous driving applications, where developers rely on innovative sensor processing approaches to differentiate their product performance and features. These Xilinx advantages are extended with the introduction of our ACAP products, which add specialized elements for on-chip data movement and neural network processing as we bring AI processing to automotive applications."

Today, we see higher demand for FPGA performance, capacity, and I/O capabilities to enable high-speed data aggregation, pre-processing, and distribution (DAPD) and compute acceleration to support L2 to L4 ADAS and AD applications. In this space, customers are demanding device safety and functional safety. Fortunately, Xilinx has a long history in other markets we serve, such as A&D and ISM, and our automotive market can leverage this heritage.

In addition, customers value the adaptability and scalability provided by the Xilinx portfolio. As ADAS/AD algorithms and methods continue to innovate, adaptability is critical, while scalability is a key factor in creating cost-effective platforms that need to meet an ever-changing set of feature packages.

From front-facing cameras and surround-view systems, to 4D imaging radar and LiDAR sensors, to in-cabin monitoring solutions for hands-free, gesture-based control and alertness monitoring, it is critical that Xilinx solutions deliver extremely reliable real-time performance at low power.

Whether reacting to hazards on the road or ensuring comfort features run smoothly as expected, automakers need to meet stringent requirements for computing latency, performance, energy efficiency, and adaptability of distributed edge sensors and centralized domain controllers to optimally handle an increasing number of complex and diverse applications. XA devices meet these needs.

What unique problems do automakers, tier 1s, and robotaxi developers face in autonomous vehicle development that Xilinx Versal/ACAP and traditional FPGA technology can solve? How can FPGA technology solve these challenges better than current GPU and CPU offerings?

Latency is a critical factor in enabling autonomous vehicles, especially for safety. Overcoming latency issues is one of the greater challenges facing automakers, tier 1s, and robotaxi developers. The ability to create optimized parallel processing pipelines in FPGAs that operate independently and simultaneously with their own dedicated logic and memory resources, achieving high performance and very low latency, addresses this challenge.

此外，与GPU和CPU相比，在FPGA中并行操作的能力意味着时钟速率可以降低。这将提高支持日益复杂的汽车系统的性能效率，与GPU和CPU相比，它的功耗显著降低。这意味着更高的可靠性和性能，同时保持在汽车热范围内。

More important than latency is safety. To address safety-critical applications, Xilinx provides certified "safety islands" that can process data from multiple sensors and cameras to respond in real time to other vehicles, pedestrians, animals, and unexpected road hazards. Sensor data can be analyzed and fused for effective redundancy, and FPGAs provide a unique ability to create application/function-specific diagnostic circuits and cross-monitors, enabling automotive developers to meet ASIL requirements. No other technology on the market can do this.

To meet the needs of artificial intelligence processing computing performance, Xilinx has expanded its 16 nm Zynq UltraScale+ MPSoC product portfolio, and its automotive qualified devices provide nearly 3,000 DSP resources (multiplication and accumulation units) and scale the original logic and memory resources accordingly. In addition, Xilinx's new Versal ACAP also provides greater flexibility, with scalar, adaptable and AI engine performance that is 20 times higher than today's fastest FPGA and 100 times higher than today's fastest CPU (see figure).

Xilinx's versatile Adaptive Compute Acceleration Platform (ACAP) combines the flexibility and connectivity of FPGAs with core capabilities such as AI engines and Arm Cortex processors.

What are the biggest challenges for customer solutions when developing and implementing ADAS and AD applications?

Power consumption, latency, and functional safety are all challenges for automotive customers in ADAS/AD. Xilinx continues to innovate and deliver products in these areas.

One of our autonomous vehicle partners, Pony.ai, found that switching to an XA-FPGA device solved performance issues for a sensor fusion system that previously had output errors of nearly half the length of the vehicle, which proved to be extremely dangerous in the real world. The challenge for our customers developing ADAS and AD solutions is not having enough compute power and low latency to manage the multiple complex calculations that are critical to ensuring their vehicles are safe, or ideally, safer than human operators. Our devices provide reliable performance and are able to perform larger and more complex calculations, getting closer to removing the latency barrier for the most critical applications.

In another example, when designing MBUX, Daimler engineers faced a unique challenge. It had to design the entire MBUX interior auxiliary computing subsystem within the roof of the vehicle, a very thermally constrained environment. Daimler engineers benchmarked many powerful computing platforms but chose Xilinx because it had the highest performance per watt and lowest latency, plus automotive-grade certification.

Are Xilinx products implemented in end-user production applications or are they primarily used for prototyping at this point?

Our devices are widely used in development and production. Over time, Xilinx has supported more than 170 million vehicles worldwide, of which 70 million are used in the production of ADAS systems. Xilinx works with more than 200 automotive companies, including major global Tier1, OEMs and start-ups.

The concept that FPGA technology is only used for development and prototyping has been proven wrong many times, and Xilinx products are now in the third generation of front cameras and ADAS central modules. Since the 90nm process node, Xilinx has created extremely cost-competitive solutions in the automotive field.

Our newest devices in our 16nm portfolio, the recently announced Zynq UltraScale+ MPSoC 7EV and 11EG, continue to support ADAS and AD applications, with these particular products specifically developed to handle the high-volume demands of centralized domain controllers.

What specific applications do the new Zynq UltraScale+ MPSoC 7EV and 11EG each support, and what advantages do they offer the automotive industry over other solutions? Which hardware specifications/features are most critical to accelerating the deployment of autonomous vehicles?

Both XAZU7EV and XAZU11EG were developed in response to customer demand for DAPD and to accelerate ADAS and AD calculations from L2+ to L4. As the number of camera, radar, and lidar sensors increases, and the data from each sensor becomes increasingly dense (e.g., camera resolutions from 1 megapixel to 8 megapixels and beyond), the need for dedicated independent processing pipelines is also increasing.

At the same time, ASPP/GPU vendors are trying to hand off more CPUs when they encounter problems and "thread the needle" in defining the appropriate number and type of interfaces, guaranteeing non-interference (the basis for functional safety) and managing unprecedented software architecture complexity, which is a huge challenge. Xilinx introduced the XAZU7EV and XAZU11EG devices to provide scalability in I/O, support full independence, but at the same time provide a more powerful sensor processing pipeline. This makes these devices very effective in the DAPD role.

The two latest larger devices, the XAZU11EG, offer more than 650,000 programmable logic cells and nearly 3,000 DSP resources, 2.5 times the previous largest device. The DSPs can be arranged in a "systolic array" of multiply and accumulate units that form the core of Xilinx's deep learning processing unit (DPU) for neural network calculations.

The XAZU7EV has a video codec unit for H.264/H.265, and the XAZU11EG has 32 12.5Gb/s transceivers and four PCIe Gen3 x16. With both new devices, we enable automotive customers to use DAPD and compute within tight power envelopes to bring AD vehicles to mass production.

What are the safety requirements for implementing ADAS and AD applications? What are the performance characteristics of the new devices? What are the most noteworthy performance specifications of the XA7EV and 11EG?

The entire Zynq UltraScale+ MPSoC portfolio is qualified according to the AEC-Q100 test specification. These devices integrate a Xilinx FPGA with a feature-rich 64-bit quad-core Arm Cortex-A53 and dual-core Arm Cortex-R5 based processing system that is certified to ASIL-C level in the low power domain.