Challenges in Advancing Automotive Imaging

As vehicles move from being fully controlled by the driver, to providing driver assistance, to eventually taking over the task of driving, they need to be able to perceive their surroundings. While there are several different sensor modalities that vehicles can use, image sensors are one of the most versatile and popular due to their unique ability to capture shape, texture, and color at a relatively low cost.

Inadequate lighting and high temperatures can degrade sensor performance, and road conditions become challenging. Therefore, image sensors must deliver outstanding performance in all conditions for autonomous driving.

There are many challenges in deploying image sensors in an automotive environment. Lighting conditions can create extreme contrast and glare from wet roads, while weather conditions including rain, fog, and snow can impede visibility. Traffic lights, road signs, and vehicle headlights and taillights often use LED lighting. One of the great things about LED lighting is that it is very efficient; however, it is often pulsed. While this is invisible to the human eye, an image sensor will present it as a flickering stream of images.

One of the main roles of automotive vision is to detect objects in the vehicle's path. The further away an object is seen, the longer the decision and reaction time available to the vehicle. This is why high resolution and high image quality are required to discern distant objects.

Cost is critical as vehicles deploy more image sensors throughout the system—not just for forward vision but also to provide a 360-degree view and monitor the passenger compartment. Some cars have more than a dozen image cameras.

From assistance to automation

The Society of Automotive Engineers (SAE) has defined a six-level model that maps the progression from a vehicle with no intelligence to a fully automated vehicle under all driving conditions.

Currently, many vehicles are capable of operating at Level 2, which includes the most basic controls, such as correcting lane drift on the highway. The transition to Level 3 is significant because Level 3 provides even more automated control of vehicle movement. Image sensors need to offer 8 megapixel (MP) resolution to support this – a four-fold increase over what is typically used today. In some cases, this is sufficient for some autonomous operations, such as on the highway. Moving into Level 4 and 5 operations, image sensor resolution will need to be even higher to support autonomous operation in all situations.

Likewise, surround-sensing and blind-spot cameras are bumped up in resolution to 3 MP or even 8 MP, depending on their use case, and include both LED flicker mitigation and high dynamic range (HDR) operation.

Additionally, non-Bayer filters have increasingly replaced Bayer color filter arrays (CFAs) to improve low-light operation while still providing good color performance.

Pixel size

Increasing the resolution of a sensor results in a significant increase in cost if the pixel size remains constant (currently from 4.2μm to 3μm). However, reducing the pixel size to 2.1μm will result in a significant reduction in the cost of an 8 MP sensor, meaning that a 3 MP sensor with 4.2μm pixels will cost much less than an 8 MP sensor with 8.2μm or 1μm pixels.

One might think that, as a result, there would be some trade-offs in key performance parameters such as low-light performance, signal-to-noise ratio (SNR), or HDR. However, this is not the case. The low-light performance metrics (SNR2 and SNR1) of ON Semiconductor sensors with 1.3 μm, 3 μm, and 75.3 μm pixels are essentially similar. The new ON Semiconductor 2.1 μm pixel image sensor has better SNR and HDR performance than the 3 μm pixel image sensor.

Additionally, the Onsemi 2.1 μm 8 MP sensor solution improves detection distance at similar or lower cost compared to other suppliers’ 3.0 μm 3 MP or 5 MP sensors.

In the challenging example of rock detection at night illuminated only by headlights, the 3 μm 3 MP and 5 MP sensors achieved detection distances of 125 m and 150 m, respectively. In comparison, the ON Semiconductor sensor achieved 170 m (Figure 4). This additional distance equates to more system response time, helping to improve safety.

Image quality and higher automotive temperatures

将滤色片从拜耳更改为RYYCy或RCCB，并结合Clarity+等高质量HDR色彩管线，可显著提高传感器性能和图像质量。非拜耳滤色片图案允许更多的光子进入每个像素，从而提高低光性能。这使得传感器能够在具有挑战性的条件下更好地“看清”，同时产生经过处理成高质量图像的色彩准确的原始捕获。

SNR is an important parameter for all image sensors as it relates to the system's ability to detect objects in the image generated by the sensor. At high temperatures, the SNR of a typical 3 μm discrete diode sensor drops to around 20 dB. At this level, noise is clearly visible and object detection is more difficult. Similar ON Semiconductor sensors can provide SNR levels in excess of 30 dB. At this level, noise is significantly reduced and object detection is much easier, providing a more pleasing image for viewing applications.

温度始终是图像传感器面临的一个挑战，并且会显著降低图像质量和性能。在汽车应用中尤其如此，在这些应用中，传感器在80°C或更高的结温下运行超过80%的使用寿命 - 由于放置在阳光直射下，并设计在狭小的封闭空间中，其他电子设备在工作期间产生热量。

Even at a junction temperature of 125°C, the 2.1μm pixel size onsemi image sensors can achieve SNR performance of over 25 dB in medium and high light conditions, ensuring accurate object detection in all operating conditions.

Hyundai 2.1 μm automotive HDR LFM image sensor

ON Semiconductor’s latest automotive image sensor offers 3840 x 2160 (8.3 MP) resolution and the latest generation 2.1μm super-exposure pixels. The sensor uses true LED Flicker Mitigation (LFM) pixel technology to produce HDR images up to 155 dB and over 110 dB in flicker-free operation. HDR frame rates are up to 60 fps, while reducing the frame rate to 45 fps increases HDR from 110 dB to over 145 dB.

In terms of low-light performance, the 2.1 μm sensor performs on par with or better than the best 3 μm pixel sensors. The difference in HDR image quality between the 2.1 μm sensor and a competing 3 μm sensor is shown, highlighting better dynamic range and capturing better details and the true colors of traffic lights. The transition SNR exceeds 100 dB at junction temperature (Tj) to 30°C, and even at extreme temperatures (Tj=125°C), the SNR exceeds 25 dB. In all conditions, the sensor produces clear images with high color fidelity, thanks in part to the range of Bayer and non-Bayer CFAs technologies - RGGB, RCCB, RCCG and RYYCy.

Summarize

Advanced autonomous vehicles are increasingly reliant on high-performance imagers to enable them to perceive their surroundings. While it is possible to increase the performance of image sensors, it is challenging to do so without increasing costs.

ON Semiconductor imaging device designs show that reducing pixel size makes the price of 8 MP sensors similar to current 2 MP 4.2 μm sensors and 4-5 MP 3 μm sensors without compromising low light performance SNR and HDR. In addition, adopting a non-Bayer CFA further enhances the all-important low light performance.

Temperature is always a challenge, with sensors mounted in limited spaces, with heat-generating components inside and exposed to sunlight. ON Semiconductor sensors can provide excellent performance at temperatures up to 125°C, ensuring high-quality images are captured under all operating conditions.

Next-generation image sensors are critical to the transition to safe and capable vehicles with greater autonomy.