Two eyes are better than one: Continental launches new stereo camera

Recently, Continental announced that it will launch a new stereo camera as one of the components of its ContiGuard® safety system for automotive forward braking systems.

The forward braking system - ContiGuard® safety function is mainly used to reduce the risk of collision between pedestrians and vehicles and the consequences of collision at intersections. Currently, in Germany, for example, intersection collisions account for approximately 46.6% of all car accidents resulting in serious personal injury.

Dr. Andreas Brand, head of passive safety and advanced driver adaptive system ADAS (Advanced Driver Adaptive System) business at Continental Chassis and Safety Systems, said that the company's positioning of accident prevention and prevention systems is not limited to distinguishing obstacles, but should also play an auxiliary role in various dangerous environments, which means that it is particularly necessary to monitor the surroundings of the vehicle at all times through new ways. The emergence of stereo cameras is similar to the vehicle monitoring system having dual "eyes". By comparing the differences in photos from each camera, the type of obstacle can be determined from a multi-dimensional perspective, such as goods left on the ground by a truck, pedestrians, animals, etc., and the size and distance of the obstacle can be obtained.

If relying solely on a single camera, it would be difficult to fully and reliably complete the above tasks with a single-dimensional image. Drivers would also need to be trained on how to identify motor vehicles and motorcycles, and the types of identification would be limited to the training content.

Andreas Brand also pointed out that the stereo camera can also be linked to the already mature driver assistance systems, such as Lane Departure Warning, Traffic Sign Recognition and Intelligent Headlamp Control, etc. The company believes that this integration shown by the stereo camera is a new trend of medium- and long-term development, and will be promoted in the future for all models from compact cars to luxury cars.

Figure 1 shows the working principle of Continental's newly developed stereo camera. The stereo camera can reliably identify pedestrians crossing the center of the road and help the vehicle to implement emergency braking.

Figure 1: Stereo cameras can reliably identify pedestrians crossing the center of the road

One system covers two camera devices and performs complete image analysis

When talking about Continental's newly developed stereo camera equipment, let's first introduce the concept of CMOS camera.

Common cameras use charge-coupled devices (CCDs) or complementary metal oxide semiconductors (CMOSs) to make sensors. In fact, neither CMOS nor CCD is a device that is really used to process image sensing - this task is completed by a "image semiconductor" that combines P (positive) and N (negative) types. CCD is a method of storing electrons from image semiconductors in an organized manner, while CMOS is actually a technology that places transistors on silicon blocks. Both have nothing to do with image sensing, and the reason they are named is mainly to distinguish them.

However, due to differences in storage and processing technology, the two sensors differ in performance such as sensitivity when used in cameras. Usually, the human eye can see targets with an illumination of less than 1Lux (on a full moon night), and CCD sensors can usually see slightly better than the human eye at 0.1-3Lux. The sensitivity of CMOS sensors is generally in the range of 6 to 15Lux. The fixed image noise of CMOS sensors is an order of magnitude higher than that of CCD sensors, so CCD sensors are more sensitive and are more widely used in the field of cameras. However, it is easier for CMOS sensors to make up for the shortcomings of sensitivity by increasing their size. At the same time, their speed advantage is obvious, 10 to 100 times faster than CCD sensors, which is especially suitable for high-frame camera applications.

The stereo camera system developed by Continental this time consists of two high-resolution CMOS monocular cameras (Mono-camera), which are installed behind the windshield, and the two camera devices are about 20 cm apart. Usually a monocular camera can only estimate the distance, while the stereo camera system can measure the target distance and the height above the road surface, so the left and right optical channels can compare the difference in the perception of the target.

In other words, the electronic analysis effect of the stereo camera can give a stereoscopic perception (Spatial Vision), that is, the parallax shift between two pictures. At a medium distance of 20-30 meters, the accuracy error of the stereo camera in detecting the target is about 20-30 centimeters.

Even in unfavorable conditions or harsh observation environments, even if other shooting technologies have reached the recognition bottleneck, stereo cameras can still maintain high precision. For example: multiple targets are close to each other and difficult to detect; only part of the target is observed; the contrast between the target and the background is low. The optical path comparison capability of stereo cameras mainly comes from redundant information, that is, comparing the same parts in two pictures to improve the reliability of data. In addition, under conditions that are not conducive to observation, such as dusk, the two optical paths can support each other and play a stronger role.

Figure 2: Continental's new stereo camera system uses a CMOS monocular camera

6-D analysis enhances safety performance

In addition to the 3D position information of the detected object, the stereo camera also provides vital supplementary information for active safety systems. The stereo camera can determine the movement direction of each pixel of the identified object along three axes (horizontal, vertical, and longitudinal).

The 6-D recognition function makes the location and direction of the target clearly identifiable. Compared with target recognition based on ordinary features, the recognition function of the stereo camera has a high degree of decision reliability; if the driver fails to respond to the target brake in time, the stereo camera can even help the vehicle implement emergency braking with a maximum overload of 1 g. The system is accurate enough to support the stereo camera to carefully calculate the potential collision point position, so as to make full use of the remaining time to take protective measures. The entire camera system can function in all speed ranges.

On the other hand, within the stereo camera field of view, the system can determine whether there is a way to successfully implement evasive action; other options include issuing a collision warning, and if it is determined that timely avoidance is not possible, automatic braking can be determined as early as possible. This function can make full use of precious hundreds of milliseconds in emergency braking, which may cause continuous scratches on the vehicle or even serious injuries to people without this short moment. The maximum working distance of the stereo camera system is 60 meters, which provides a good technical basis for the forward-looking implementation of the braking system.

“In the future, stereo cameras could even be used to detect small objects, such as children, crossing the road, whether walking or riding a bicycle,” said Wilfried Mehr, head of Advanced Driver Assistance Systems at Continental. “The system we have developed has comprehensive object recognition capabilities, which is unprecedented in product technology.”