Development and application of programmable platforms for driver assistance systems

Safety is the most important concern for car consumers. Figure 1 shows the results of a study conducted by Visteon, which shows the requirements of customers for cars, with vehicle safety at the core. The focus on car safety is not only for the driver and passengers, but also for others on the road. Safety equipment has moved from the physical field to the electronic field, from advances in tire and brake technology, to side impact protection and airbags, to today's assisted driving systems. The latest cars use a large number of electronic technologies and sensors to continuously monitor and evaluate the surrounding environment, display relevant information to the driver, and in some cases, even take over control of the vehicle. These electronic systems play an important role in improving car safety, comfort and driving efficiency.

Assisted driving systems can provide basic safety features, such as adding infrared (IR) cameras to improve observation capabilities. More advanced designs can also use a wide range of sensors to warn of potential dangers, so that the vehicle can be aware of surrounding traffic conditions, lanes and directions of travel, and possible collision targets. The ultimate goal is that the vehicle can automatically respond to this information, providing information to the driver and the ability to control the vehicle in special situations to ensure the safety of passengers. For example, some of the latest trucks are equipped with video cameras to monitor the road ahead. If the vehicle changes its path without using indicators, for example, because the driver is too tired, the system will give an audible warning through the speakers in the vehicle.

Assisted driving can also provide a higher level of comfort by eliminating tedious driving actions. For example, traditional cruise control allows the driver to set a fixed driving speed and manually control it when needed. Today's cars offer automatic cruise control (ACC) functions, which can automatically control the throttle and brakes to adapt to the speed of the vehicle in front, thereby maintaining a safe distance from it. If the vehicle in front speeds away or changes its driving path, ACC automatically returns to the preset speed of traditional cruise control.

Assisted driving systems also hold promise for improving traffic efficiency by using so-called “electronic traction devices.” For example, the lead truck in a convoy is driven manually by the driver, but the following trucks are driven automatically. In addition to relieving the driver of many of the burdens, the distance between trucks can be greatly shortened because the electronics respond more quickly. This not only saves complete road space, but also saves fuel due to the impact of the rearward airflow of the vehicle in front.

Another emerging safety technology is called "passive occupant recognition systems." The U.S. government requires that all new cars starting in 2006 must be able to deploy airbags based on the size of the occupants. Such systems allow protective airbags to be "intelligently" deployed or retracted. This system based on the weight of the occupant will help automakers meet the requirements of the recently announced Federal Vehicle Standard Safety Regulation FMVSS-208. The regulation requires that airbags must be able to deploy more effectively based on the weight of different occupants. Starting in 2004, each automaker must equip 35% of its vehicles sold in the United States with advanced airbag systems, and this number will increase to nearly 100% by 2006. The simpler systems are implemented using weight sensor technology installed under the occupant's seat cushion. Advanced occupant recognition algorithms and fast signal processing allow the car's airbag controller to deploy or retract the occupant airbag according to different situations, which can greatly improve occupant safety and reduce repair costs. More advanced systems use cameras installed in the car to detect and identify occupants, while algorithms take into account the occupant's adjustment and distance from the airbag to determine the time, speed and extent of airbag deployment in the event of an accident.

Application of Xilinx FPGA in Assisted Driving System

Figure 2 shows a conceptual block diagram of the Xilinx field programmable gate array (FPGA) applied to the ACC assisted driving system.

The system is divided into ultra-high-speed input processing and relatively low-speed sensor input and output control information, each under the control of a corresponding processor (for example, a Xilinx MicroBlaze 32 embedded soft-core processor or an IBM PowerPC embedded in a Virtex-II Pro FPGA). The high-speed section is dedicated to real-time processing of video camera information mounted in front of the vehicle. Due to the nature of the application (collision avoidance, emergency handling, and warning), real-time processing is absolutely critical. Two or more cameras are usually required to obtain stereo images, so that the depth of the image (which directly relates to the actual distance of the object in front) can be calculated in the FPGA. Combining radar and laser measurements, as well as motion detection information from gyroscopes and wheel sensors, the vehicle's surroundings and driving path can be calculated quite accurately. Using a fully flexible FPGA instead of off-the-shelf video components, equipment manufacturers can easily develop unique and optimized edge detection, image depth, and enhancement algorithms that differentiate the performance of competing manufacturers' systems. Capturing and processing this information in real time requires the use of computationally intensive digital signal processing (DSP) algorithms. However, software processing cannot meet the performance requirements; although traditional DSP processors are also an option, they usually require multiple devices to complete such high-speed tasks. Even ASSP video processors cannot match the extremely high-speed DSP performance of Xilinx FPGAs (also known as XtremeDSP processing). After video processing, the decision tree mechanism can be divided into a hardware portion for emergency algorithms (such as emergency collision avoidance procedures) and a processor software portion for audible warnings such as driving path deviations. Partitioning speed-critical processing into FPGA hardware also allows testing of real-time speeds, which is not possible with software.

[page]

XtremeDSP real-time image processing

So why can Xilinx FPGA provide faster video processing performance than traditional DSP? The most fundamental reason is that the FPGA structure can realize parallel processing of data. The latest Virtex-Pro series devices from Xilinx also integrate embedded high-performance multiplier module arrays to further improve image processing capabilities. In contrast, DSP processors execute instructions and data sequentially and process them in a serial manner. Therefore, FPGAs can be configured as multiplication and accumulation (MAC) unit arrays that can perform multiple operations in parallel (within a single clock cycle), rather than requiring multiple clock cycles to complete execution in one or a small number of MAC units as in traditional DSPs.

Xilinx FPGAs have the added benefit of being able to utilize the exact array of MACs to match computational requirements. These features are ideal for completing image calculations. This allows multiple clusters of pixels in an image, such as macroblocks of a discrete cosine transform (DCT), to be calculated in parallel, rather than having to scan the entire image sequentially. The increased performance of FPGAs also brings additional benefits, such as the amount of memory required to buffer pixel values ​​can be smaller because they can now be processed in real time.

In addition to real-time performance, the reprogrammability of Xilinx FPGAs provides excellent system flexibility and supports algorithm upgrades (even after deployment). This is very important because current assisted driving systems are still in the early stages of development. As edge and object detection algorithms continue to improve, hardware upgrades can be completed in minutes without redesigning the circuit board.

Bridging Automotive Networks with Programmable Peripherals

As real small networks evolve in cars, equipment manufacturers must determine which of the many network protocols will be the most successful or which standards can bring them the greatest benefits. Different network technologies are used to meet different needs in the car, from the multimedia range in the cockpit (Multimedia Oriented Systems Transport, MOST) to the vehicle control network (such as FlexRay). A pre-verified control area network (CAN) interface core is selected as an example in Figure 2.

One such emerging network protocol that can be applied in the car is Bluetooth. Bluetooth wireless technology is a low-cost, low-power, short-range radio frequency technology for mobile devices and WAN/LAN access points. This standard, which originated from the computing and telecommunications industries, describes how devices such as mobile phones, computers and PDAs can be easily connected to each other using a short-range wireless connection.

For example, a driver can use a Bluetooth cordless headset to communicate with a mobile phone in his pocket. This avoids driver distraction and improves safety. The automotive industry has set up a special interest group (SIG) to define the Bluetooth automotive standard. Members of the SIG include the Automotive Multimedia Interface Cooperation (AMIC), BMW, Daimler-Chrysler, Ford, General Motors, Toyota Motor and Volkswagen AG. An example of Bluetooth in the car is Johnson Controls' hands-free mobile phone system "BlueConnect", which allows drivers to stay connected through a Bluetooth-enabled mobile phone while keeping their hands on the steering wheel.

However, there are still issues with the long-term support of Bluetooth devices, and the impact of ambient noise in the car on the operation of Bluetooth devices also needs to be carefully considered. The lifespan of cars and other vehicles is much longer than that of consumer products or mobile phones, so chip manufacturers must solve the problem of mismatched support and service life cycles. However, at the Convergence 2002 exhibition held recently in Detroit, Chrysler Group exhibited a car with Bluetooth technology.

One of the biggest benefits of using FPGAs over ASSPs is that it allows engineers to design interfaces and peripherals that precisely match system requirements. This is particularly useful in the early stages of development when trying to connect to different automotive networks. When trying to get a product to market quickly, chipset or ASIC redesign is both expensive and time-consuming. In the early days of standard implementation, if the network protocol specifications change, in order to support the latest version, when using FPGA designs, you only need to simply modify the software and then re-download the FPGA hardware configuration. This can even be done over a wide area network using Xilinx IRL (Internet Reconfigurable Logic), so hardware modifications can be completed through remote maintenance without costly dispatch fees or additional manpower.

Xilinx IQ Solutions for Automotive Applications

To meet the needs of automotive electronics designers, Xilinx has introduced a new family of devices that support the extended industrial temperature range. Called the "IQ" range, these new devices include Xilinx's existing industrial (I) FPGAs and CPLDs that now qualify for the extended temperature grade (Q) (Table 1). The first devices to qualify for the new IQ temperature range are the Spartan-XL 3.3V FPGAs, which range in density from 5K gates to 3K gates, and the XC9500XL 3.3V CPLDs, which come in 36 and 72 macrocells. In the coming months, the IQ temperature range will expand to include FPGAs with densities up to 300,000 gates, and CPLDs with densities up to 512 macrocells, as shown in Table 2.

in conclusion

The development and application of assisted driving systems require high-performance image processing, but do not want to sacrifice the flexibility required in the early stages of target detection and automotive network technology research and development. Using Xilinx FPGA as the core of such systems provides the industry with the best DSP performance and unparalleled support for network connection standards, while providing system designers with a completely flexible design platform. Through such systems that can work in real time, it becomes possible to provide drivers with emergency driving warnings or auxiliary vehicle control functions, thereby greatly improving the safety of vehicle driving and riding.