Automotive safety technology: Nissan turns to camera-centric development

"Previously, we have been thinking about and developing with millimeter-wave radar and ultrasonic sensors (sonar) as the axis, but now the development of safety technology is gradually shifting to cameras as the axis." Shunichi Toyozo, who is in charge of technology development at Nissan Motor (Nissan Executive Officer, Head of EV Technology Development Division, Electronic Technology Development Headquarters), said this at the "Advanced Technology Briefing & Test Drive 2011" held in early October 2011.

Nissan has been working on developing technologies to improve vehicle safety based on millimeter-wave radar and ultrasonic sensors. However, the direction of development has changed, and the core of safety technology has shifted to cameras due to the recent development of camera recognition technology. According to reports, millimeter-wave radar and ultrasonic sensors will mainly play a supporting role.

Adding moving object detection function

Nissan's representative safety technology using cameras includes a "surround view display" that displays a bird's-eye view of the vehicle's surroundings. The so-called surround view display combines images from four cameras, one each in front, behind, left and right of the vehicle, into one image display. Nissan equipped the "Elgrand" launched in October 2007 with this system.

This time, Nissan has added a function to detect moving objects to the surround view display. The company has also developed an algorithm to identify surrounding "moving objects" based on camera image data and equipped it on the electronic control unit (ECU). It will first be used in the Elgrand, which will undergo a partial improvement in November 2011.

One camera achieves three functions

As a next-generation technology using cameras, Nissan has released a demonstration that multiple functions can be realized by installing a camera on the front and rear of the vehicle (Figure 1). Nissan said, "We will vigorously promote technology development in the direction of suppressing costs while increasing added value by making the best use of the hardware already installed."

Figure 1: Multiple functions are realized by using one camera each in the front and rear
Nissan has developed a technology that improves safety by using only the front and rear cameras. In addition to identifying cars and pedestrians in front of the vehicle, it can also realize multiple functions such as recognition of road signs and detection of moving objects behind the vehicle.

A front camera installed at the front of the car can realize three functions: notifying the driver when the vehicle ahead is approaching; detecting the distance between the road markings and the position of the car, and issuing a warning sound if there is a sign of lane deviation; and reminding the driver to pay attention when there are pedestrians ahead when driving at low speed. Vehicles and pedestrians are distinguished by the shape and speed of moving objects.

The prototype car has a front camera installed in front of the interior mirror. The camera module is integrated with the ECU that performs image recognition processing. The camera's image element uses a 640×480 (VGA) pixel CMOS sensor. The horizontal viewing angle is only about 40 degrees, and the maximum forward distance that can be recognized is about 100 meters.

A rear camera can realize three functions: road sign recognition; detection of vehicles and motorcycles approaching from the blind spot behind the vehicle; and recognition of moving objects (people and vehicles, etc.) behind the vehicle when exiting the parking lot. The camera element uses a CMOS sensor with the same pixels as the front camera. However, the horizontal viewing angle is increased to 180 degrees. Therefore, the camera's detection range is only about 20 to 30 meters.

In the case that one camera can realize multiple functions, it is estimated that the three functions of the rear camera will be promoted as standard in the future. At present, North America is planning to install a rear view camera that can confirm the situation behind from the driver's seat around 2014-2015. The reason is that many North American users park their cars in the parking lot with the front of the car facing forward, and there are many cases of collisions with pedestrians behind the car when reversing out of the parking lot.

Nissan Motor has identified the trend that rearview cameras will become mandatory in the future, and plans to "gradually adopt" three functions based on rearview cameras (Feng Zeng) starting with new models launched in 2012.

Millimeter waves penetrate the gap between vehicles and roads

Nissan also demonstrated a new function that increases the safety technology value of conventional millimeter-wave radar using a prototype vehicle: a driving support system that uses millimeter-wave radar to identify forward conditions that are blocked by vehicles ahead and avoid collisions (Figure 2).

Figure 2: Using millimeter waves to understand the driving conditions of the second car ahead
Nissan has unveiled a driving support system that uses millimeter-wave radar to identify the forward conditions that are not visible due to the vehicle ahead, thereby avoiding collisions. The goal is to reduce collision accidents by understanding the driving conditions of the second car ahead.

When the vehicle ahead is high on a highway, if the second vehicle ahead is traveling very slowly or decelerating urgently, in addition to the danger of rear-ending the vehicle ahead, accidents often occur where the vehicle ahead suddenly changes lanes and collides with the second vehicle ahead.

The system developed this time uses millimeter-wave radar to monitor the distance and relative speed to the second vehicle in front, and alerts the driver through display, sound and tightening of the seat belt when there is a risk of collision.

Millimeter-wave radar can use the same product as before, so there is no need to change the hardware. The second car in front can be detected by using the millimeter waves passing between the floor of the front car and the road.

In fact, the previous system was already able to grasp the driving status of the second car ahead through millimeter waves. However, at that time, the focus was on high-precision measurement of the relative speed between the vehicle ahead and the vehicle ahead, so "noise removal was performed on the reflected waves from the second car ahead" (Nissan Motor).