How does Hyperlux image sensor meet the key parameters required by AEB system?

On the surface, the plan to mandate AEB systems is a good idea. But if NHTSA's proposal is to have an impact, it must get ahead of the industry it hopes to shape. To understand whether the standard will have an impact on the auto industry in a timely manner, and whether NHTSA's plan goes far enough, it is necessary to understand not only the proposal itself, but also AEB technology in general. By exploring the underlying technology, as well as the pace of progress by automakers and market trends, it is possible to accurately judge the potential value of NHTSA's plan.

Figure 1: Schematic diagram of automotive perception technology

NHTSA's plan

*Data transferred from previous report as automakers are not required to submit updated data after reaching targets

Figure 2: Most automakers see significant growth in deployment between 2020 and 2022

Comparing IIHI’s data to NHTSA’s original 2020 report highlights some interesting trends. First, 14 of 20 automakers equipped at least 95% of their vehicles with AEB in 2022, up from 10 two years ago. This rapid progress is further highlighted when we look at the six exceptions described below. In 2022, Kia had an AEB rate of 94%, and Acura (Honda) had an AEB rate of 93%, but Acura had been above that rate until parts shortages in 2021; the other four manufacturers’ rates increased from 38% in 2020 to an average of 72%.

In 2023, all manufacturers indicated that AEB deployment will increase again, with GM indicating that 98% of its 2023 model year vehicles will have the technology. This rapid year-over-year increase, along with the timing of the NHTSA proposal, raises some interesting questions. The main concern for many is whether this proposal will have a real impact if all manufacturers already have AEB in their vehicles before the standard is implemented. But understanding the reasons for this proposal can add some weight to its value.

First, it's important to take a closer look at the specifics of the proposal. For a vehicle-to-vehicle collision, if the driver fails to react in time, the AEB system would need to actively brake at 50 mph. If the driver brakes but does not reach the maximum braking force required, the AEB system would need to completely avoid a collision with another vehicle at speeds of 62 mph. The proposed vehicle-to-vehicle AEB scenarios include a vehicle approaching a stationary vehicle ahead, a slow-moving vehicle ahead, and a decelerating vehicle ahead.

For vehicle-to-pedestrian incidents, all vehicles would be required to take action at speeds up to 40 mph. Proposed vehicle-to-pedestrian AEB scenarios include a vehicle approaching a pedestrian crossing the road (from the left or right), a pedestrian stationary in the vehicle's path, and a pedestrian walking in the vehicle's path. The technology would also need to work at night, when NHTSA says more than 70 percent of pedestrian fatalities in the U.S. occur.

If the proposal is approved by the end of August 2023, it will become mandatory from September 2026 (i.e. 2027 model year). If it is approved later, mandatory deployment will be delayed by one year (i.e. 2028 model year).

Given statistics like nighttime fatalities that highlight the importance of AEB, and the fact that it saves 360 lives and prevents at least 24,000 injuries in the U.S. each year, it’s clear why NHTSA is targeting the technology. But is enforcement necessary given the level of sensors most vehicles already have? And do the proposed regulations really go far enough to meet the need?

AEB regulations: US vs. Europe

Since 2019, AEB has been mandatory for a five-star Euro NCAP rating, the accepted European method for rating vehicle safety. While this could be seen as progress, the European Commission has gone a step further by mandating that all new cars sold from 2022 must be fitted with vehicle-to-vehicle AEB, with vehicle-to-pedestrian systems becoming mandatory from 2024.

In addition, there are significant differences in scope between the NHTSA proposal and the Euro NCAP regulations. While both standards cover similar vehicle-to-vehicle and vehicle-to-pedestrian AEB scenarios (including adult/child pedestrians and nighttime conditions), Euro NCAP also covers several vehicle-to-bicycle and vehicle-to-motorcycle AEB scenarios, taking into account a wider range of vulnerable road users (VRUs).

Furthermore, unlike the NHTSA proposal, Euro NCAP also covers AEB scenarios including vehicles turning (left or right) and reversing towards pedestrians.

A typical AEB system combines image sensors, LiDAR, and RADAR to sense objects that could cause a potential collision. The vehicle analyzes the image in real time and reminds the driver to brake if a collision is imminent. If the driver fails to brake in enough time, the vehicle will take the initiative to brake.

For image sensors, the AEB system needs to be converted into parameter requirements such as field of view (FOV), resolution, frame rate and low-light performance. These image sensor characteristics will directly affect the detectability of objects and the delay of object detection.

As shown in Figure 2, the percentage of vehicles sold with AEB features has increased significantly, but this is not the only major change. The underlying technology in vehicles has advanced significantly in recent years. Processing systems are more powerful, communication systems are faster and have higher data rates, and sensors are becoming more accurate, especially in low-light conditions. These factors together have made AEB systems smarter and more reliable, enabling them to take life-saving actions in a wider range of situations.

With more than 15 years of ADAS imaging experience, ON Semiconductor has been at the heart of this technological revolution. As the world's leading producer of image sensors for ADAS systems (including AEB), ON Semiconductor delivers 70% of all image sensors in vehicles on the road today.

Let’s consider the highest speed (and therefore longest braking distance) of the vehicle-to-pedestrian scenario: a vehicle is approaching a pedestrian who is walking in the vehicle’s path. The vehicle can be traveling at up to 40 miles per hour, while the pedestrian’s walking speed is 3.1 miles per hour.

For this vehicle-to-pedestrian scenario, if we consider typical urban road conditions, a 120° field of view for the ADAS ahead, a stopping (braking) distance for a 40 mph vehicle, a 0.5 to 1 second response (reaction) time, and a minimum of 8 pixels per pedestrian for correct identification, we can deduce that a minimum (horizontal) resolution of approximately 2480 pixels is required (using high school math geometry).

The Hyperlux AR0823AT sensor (3840 x 2160 pixels) meets NHTSA’s most stringent horizontal resolution requirements for vehicle-to-pedestrian scenarios by more than 50 percent, effectively providing up to 12 pixels per pedestrian to enable powerful AEB recognition algorithm operation.

One missing element of the 2016 recommendation is the new NHTSA recommendation, which requires accurate detection of pedestrians in low-light conditions and at night. This is already a priority for ON Semiconductor and is a hallmark of the Hyperlux sensor family, which features a 2.1 µm pixel size and industry-leading 150 dB high dynamic range (HDR) for accurate imaging in scenes ranging from 0.05 lux to 2 million lux.

Let’s compare the performance of the Hyperlux sensor family in low-light conditions with the NHTSA proposal, again considering the most stringent vehicle-to-pedestrian scenario (e.g. a pedestrian walking in the path of an oncoming vehicle), but this time at night. The proposal mentions an ambient light level of 0.2 lux (moonlight conditions) and vehicle low/high beam lighting, but does not specify the illumination level of the vehicle lighting (unlike Euro NCAP).

A 2019 AAA study of U.S. and European headlight performance showed that the average low-beam illumination level at pedestrians is about 9 lux. Assuming a worst-case pedestrian reflectivity of 10% (dark clothing) and using a typical automotive lens (F-number 1.4), the illumination level at the sensor is 0.11 lux.