Improving the accuracy of industrial automation systems using eHDR technology

Industrial automation is becoming increasingly popular in order to improve efficiency and performance. In application scenarios such as autonomous mobile robots (AMRs), warehouse robots, drones, agriculture, factory inspections, and security/surveillance, machine vision-based artificial intelligence (AI) and advanced technologies are implemented to perform key functions. To improve existing object detection and recognition capabilities, it is necessary to solve the problem of capturing images of moving objects and fine details at longer distances under adverse lighting conditions. The newly launched AR0822 sensor from ON Semiconductor has an embedded high dynamic range (eHDR) function that solves the complex design challenges faced by engineers when developing autonomous machine vision systems.

The Importance of High Dynamic Range in Industrial Automation

Many industrial applications work in scenes with harsh lighting and both bright and dark areas. For example, outdoor AMRs must operate accurately in scenes with both sunlight and dark areas; whether the monitoring system alarm is triggered depends on whether it can distinguish the movement of unauthorized personnel from dark areas to bright areas. While factory engineers control lighting to achieve ideal brightness levels where details need to be captured, some warehouses may not be able to control lighting or may need to be rebuilt at a high cost. In addition, industrial robots should be able to extend to multiple locations to operate, which may result in uncertain lighting conditions. For example, drones transporting packages may encounter such situations at night or in high-intensity sunlight. These are a few examples of how these autonomous systems must operate in high dynamic range scenarios.

Multiple exposures = higher dynamic range

The dynamic range of a sensor refers to the ability of an image sensor to capture image details in both dark and bright areas, and is measured in decibels (dB). Generally speaking, a dynamic range of 120 dB captures details in all but extreme scenes (automotive applications require higher dynamic ranges, especially for passenger safety). In scenes such as these that contain both dark and bright areas, using a single long exposure may result in oversaturation of the bright areas, while a single short exposure may not capture details in the dark areas.

Therefore, to capture details in all areas, the sensor can use short, medium, and long exposure times to capture multiple images through multiple exposures. These exposures are then combined into a single image to achieve a high dynamic range, a process called linearization.

Figure 3: The scene in Figures 1 and 2 now has a high dynamic range

Now, with a range of details from light to dark, AI recognition systems can improve recognition and identification performance.

High-resolution challenges

As recognition distances increase and image details increase, higher resolutions are required. At 5 to 10 meters away from the object, 1080p is insufficient to provide the details required for object detection and classification. As a result, the demand for 4K and even higher resolution sensors continues to increase. For images of these sizes, bandwidth becomes a design challenge because the size of each image increases significantly. In addition, if 120 dB HDR is to be achieved, the sensor needs three exposures and a 3x effective frame rate because the sensor captures three different images for each frame. For example, for three-exposure HDR, if the effective frame rate of the sensor is 30 frames per second (fps), the sensor array, circuitry, and outputs need to operate effectively at 90 fps. A three-exposure HDR 4K image at 30 fps requires 9000 Mbps if linearization is performed in an ISP independent of the sensor, which is a major challenge for processing performance and interface speed. For camera systems, combining high resolution with off-sensor HDR is a difficult challenge.

Figure 4. A typical high dynamic range architecture sends multiple images across the interface between the image sensor and the ISP, which can strain or exceed available bandwidth at high resolutions.

To alleviate both of these issues, the ON Semiconductor AR0822 intelligently embeds high dynamic range by integrating real-time linearization into the sensor, making it easier for traditional processing performance and interface speeds to meet demand even at higher resolutions.

Figure 5. AR0822 embedded high dynamic range (eHDR) image sensor

Motion and LED lighting artifacts

Artifacts appear in HDR images because the three exposures are taken at different points in time. The position (angular velocity) of a fast-moving object will be slightly different in each exposure.

Figure 6. A spinning fan showing motion artifacts caused by multiple-exposure HDR.

Additionally, scenes with LED lighting may exhibit other artifacts. To save power, LEDs are turned on and off at a rate that the camera can see, rather than the human eye. The LED may be on for the first exposure and off for the second and third, resulting in different light levels across multiple exposures and images.

Figure 7. LED light box with two columns lit and intensity differences caused by LED flicker artifacts

The AR0822 eHDR includes “smart linearization” to address these artifacts caused by combining multiple exposure frames. This is possible by sensing the difference in signal levels captured by different exposures within a frame, thereby attempting to reduce artifacts typically caused by motion or LED flicker.

Figure 8 and Figure 9. Thanks to the smart linearization of the AR0822, the fan has no motion artifacts and both columns of the LED light box are fully lit.

eHDR enhances object detection and recognition by delivering high-quality performance in challenging lighting conditions combined with the object resolution required by AI systems in security/surveillance, AMR, and industrial mobile robotics applications.

Our white paper provides details on the AR0822 and how using eHDR can increase dynamic range and overcome other challenges when implementing higher dynamic range image sensors in embedded autonomous machine vision applications.