Technical article: ADAS focuses on functional safety of image sensing

By Giri Venkat, Automotive Imaging Solutions Architect and Technical Marketing, ON Semiconductor

Advanced driver assistance systems (ADAS) are becoming increasingly sophisticated, to the point where the prospect of fully autonomous driving no longer seems so far away. At the heart of ADAS are image sensors, and their functional safety is gaining importance and attention as their role becomes increasingly important to the overall effectiveness of both passive and active ADAS.

The introduction of the ISO 26262 vehicle safety standard and the concept of Automotive Safety Integrity Level (ASIL) emphasizes the requirements for functional safety.

It is very important for automotive designers to understand the ASIL as it applies to and pertains to image sensors in ADAS. It is also critical to understand fault detection techniques, the nature of potential faults and their impact on data reliability, the correction and accumulation of faults to the overall functional safety of sophisticated ADAS.

The latest ADAS systems include multiple image sensors that are able to detect and identify various types of hazards, including pedestrians, static objects, and vehicles. The sensors are able to identify and communicate whether any avoidance action is required based on the hazard trajectory. One thing all of these machine vision-based systems have in common is that a full or partial failure of the sensors could have potentially serious or even life-threatening consequences.

ASIL Overview

Functional safety or ASIL is a standards-based safety regime/criteria specifically for vehicles. It originated from ISO 61508, a comprehensive industrial safety standard, but is currently being developed as a standalone standard (ISO 26262) to keep pace with specific automotive needs and rapidly evolving automotive technology.

ISO 26262 covers electrical and electronic systems installed in "series production passenger cars," and while many standards simply mandate a series of tests and conditions that products must meet to be certified, ISO 26262 is a risk-based standard. This means it defines a process for assessing the risk of hazardous situations and identifying and implementing safety measures to control system failures. The approach also hopes to detect and control random failures, or at least mitigate their effects as much as possible.

Image sensor fault types

A typical image sensor is a relatively complex subsystem that contains an array of photosensitive elements that convert incident light into analog electrical signals. These signals are then digitized so that they can be stored, processed, and transmitted to the ADAS system for further analysis, such as obstacle recognition.

Figure 1: Typical image sensor block diagram

In the world of functional safety, an "unsafe failure" is a condition that could cause the ADAS system to make an incorrect decision. This could range from a complete sensor failure to a single pixel, row or column failure of the sensor, or a problem in the analog/digital circuitry. Typical failures could include "stuck" pixels, electrical noise effects, missing rows, missing columns, shifted pixels, transmission errors or color casts.

Scene: Scene

Reference Image: Reference image

"Fault" Image: "Fault" Image

Figure 2: Example of a fault in an analog circuit

As object detection algorithms become more sophisticated, often detecting pixels as small as 10×10, smaller faults can “mislead” ADAS systems. In fact, fault detection is a considerable challenge, partly due to the different nature and types of faults in the analog and digital domains, and also due to the high pixel count of modern sensors.

Fault detection

There are almost as many ways to detect failures as there are types of failures. For example, a stuck pixel, row, or column can be detected relatively easily by the host processor, but becomes time-consuming and resource-intensive as the number of sensor pixels increases. By using transmitters and receivers with built-in error checking, transmission errors can be detected - and even corrected. This approach does not place a load on the system processor, but it increases the cost of the overall system.

There are many disadvantages to using an ADAS system processor to detect faults. First, there are a large number of subtle fault types that cannot be detected at the system level, including color deviations or noise-based issues. Another disadvantage is the cost of system-level fault detection; given the number of potential fault types and the large number of pixels, rows, and columns that need to be checked, very high-performance CPUs, GPUs, and memories are required. Even if the cost is acceptable, the added complexity will affect the effectiveness, responsiveness, and energy efficiency of the entire ADAS system.

This leads to the most concerning aspect of system-level fault detection - the time it takes to run all the algorithms. While the processor is crunching numbers, the vehicle could quickly approach an obstacle, or a pedestrian, with potentially catastrophic consequences.

Sensor-based fault detection

Modern image sensors include many integrated test features, such as the ability to perform a cyclic redundancy check (CRC) to detect faults in transmission, or the ability to perform additional counts when counting frames, rows, or pixels to ensure that the correct frame was sent.

When functional safety systems are built into sensors, the role of the system is reduced to simply checking status indication bits or registers, which immediately indicate faults while consuming almost no system resources. This also means that high-performance CPUs, GPUs, and memory are no longer required, reducing system cost and complexity.

A key metric for low-latency ADAS systems is the Fault Detection Time Interval (FTDI), which is a measure of how long it takes the ADAS system to detect a fault, and the Fault Tolerant Time Interval (FTTI), which is the time it takes the system to respond, restoring the system to a safe state.

Fault: Fault

Fault Detected: A fault has been detected.

Potential Hazardous Event: Potential Hazardous Event

State

Normal Operation: Normal operation

Sensor Frame Time: Sensor frame time

System Detection Time: System detection time

Safe State

Time:

Normal Operation, Late Detection (System-based fault detection): Normal operation, late detection (system-based fault detection)

Enhanced Operation, Early Detection (Sensor-based fault detection): Enhanced operation, early detection (sensor-based fault detection)

Figure 3: System detected FDTI (top) vs. sensor detected FDTI (bottom)

As shown in Figure 3, when fault detection is built into the sensor, the detection time is greatly reduced, giving the system more time to recover to a safe state before a dangerous event can occur.

A modern image sensor: the AR0233AT

One of the latest image sensors is the AR0233AT from ON Semiconductor. This 2.6-megapixel 1/2.5-inch device is designed for many automotive applications, including ADAS systems.

The backside-illuminated (BSI) technology uses 3.0-micron pixels and delivers excellent low-light performance exceeding 95 decibels (dB) in a single exposure. On-chip high dynamic range (HDR) improves low-light performance to over 140 dB, or 120 dB with LED flicker suppression.

Safety is at the heart of this sensor, and the device is qualified to ISO 26262 ASIL-B and AEC-Q100 Grade 2. A complete safety package with over 8,000 injected faults is available to designers.

Summarize

As ADAS systems become more prevalent and increasingly used in vehicle design to influence decision-making and avoid hazards, we are becoming more reliant on them. As a result, they now fall under the purview of ISO 26262, a functional safety standard for passenger cars that requires a “cradle-to-grave” approach to identifying hazards and mitigating their effects as much as possible.

Given the complexity of image sensors, the increasing number of pixels that need to be checked, and the large number of possible failure scenarios, offloading the inspection work to the ADAS system is unlikely to be successful and will definitely increase costs.

As a result, the latest image sensors for ADAS systems incorporate self-detection technology, which not only speeds up the fault detection process, but also means that all the ADAS system's processing power can be used to make the right decisions to avoid hazards, making our roads safer.