Research on in-vehicle DMS: Vehicle installation has entered a period of rapid growth, with a year-on-year increase of 360% in the first quarter of 2020

DMS (Driver Monitor System) is the abbreviation of driver monitoring system, which refers to the information technology system that monitors the driver's fatigue state and dangerous driving behavior around the clock during driving. After discovering that the driver is fatigued, yawning, squinting, or other wrong driving states, the DMS system will promptly analyze such behaviors and provide voice and light prompts to warn the driver and correct the wrong driving behavior. DMS is generally divided into active DMS and passive DMS.

 
Passive DMS determines the driver's state based on the steering wheel and driving trajectory characteristics. Active DMS systems are generally based on cameras and near-infrared technology to detect the driver's state from eyelid closure, blinking, gaze direction, yawning and head movement. This report only studies active DMS.
 
In 2006, the Lexus LS 460 was first equipped with active DMS. The camera was installed on the top of the steering column cover with six built-in near-infrared LEDs. However, automakers have not been very interested in active DMS, believing that on the one hand it increases the cost of the whole vehicle, and on the other hand consumers do not necessarily pay for it. However, a series of safety accidents in recent years have greatly increased the importance of DMS in automatic assisted driving systems , especially L2/L3 functions. Starting in 2018, with the mass production of L2 systems and the upcoming mass production of L3 systems, active DMS systems have begun to increase in volume.
 
Euro-NCAP has released a 2025 roadmap, requiring that all new cars must be equipped with DMS from July 2022. China has legislated to force commercial vehicles to be equipped with driver monitoring systems, and the requirements for passenger cars are also being promoted. Related chip, software, and algorithm manufacturers are actively promoting the development of DMS technology, and the active DMS market will usher in an explosion.

OEMs’ adoption of DMS systems

Source: Zoss Automotive Research

According to data from Zoss Automotive Research, the number of new passenger cars with active DMS systems installed in China in 2019 was 10,170, a year-on-year increase of 174%. The number of installations in Q1 2020 was 5,137, a year-on-year increase of 360%. The growth momentum comes from the adoption of active DMS systems in models in the 150,000-200,000 yuan price range, with brands such as WEY, Xiaopeng Motors, and Geely Auto all installing them.
 
Most  Tier 1s  have launched complete DMS solutions, including Valeo, Bosch, Continental, Denso, Hyundai Mobis, Visteon, and Veoneer. Among Chinese companies, DMS products from companies such as Hikvision, SenseTime, Baidu, and Dahua have also been deployed on models of various brands.

Comparison of some DMS system suppliers' products

Source: Zoss Automotive Research

The rise of DMS algorithm vendors

The core function of DMS is to monitor the driver's fatigue and distraction. However, based on more sensors, vision + infrared cameras, and even millimeter wave radars, more functions can be realized, such as: face recognition, age and gender estimation, emotion estimation, seat belt detection, posture position, forgetfulness detection, cabin abnormality detection, child detection, etc. Through the recognition of face, gender and expression, identity authentication and richer human-vehicle interaction can be achieved. At present, the application of DMS is only in the early warning stage, but once combined with ADAS/AD system, it can also realize personalized body control and other functions.
 
Taking Valeo's driver monitoring system as an example, it has realized an artificial intelligence-based system that uses convolutional neural network algorithms and has emotion recognition functions, which can realize personalized settings for drivers through face recognition.
 
However, these rich functions cannot be completed by Tier 1 alone in the short term, so most DMS systems are realized by cooperation with Tier 1+ algorithm companies.
 
ArcSoft has a long-term accumulation in computer vision technology and has developed DMS, ADAS, BSD and other products specifically for the vision-related functional requirements of the automotive industry. In 2019, ArcSoft's intelligent driving algorithm business developed rapidly, achieving operating income of 16.0566 million yuan, a significant increase from 2.2021 million yuan in the previous year. In the first quarter of 2020, the intelligent driving algorithm business achieved operating income of 17.0076 million yuan, which has exceeded the full-year revenue of 2019.
 
EyeSight, Smart Eye, FotoNation, Seeing Machine, etc. are similar software algorithm companies, and they have accumulated technology for several years. Even so, there are still many difficult problems in the application of DMS technology:

• The biggest technical difficulty of vision-based DMS is its performance in strong or weak light conditions. If the light is too strong, the image will appear completely white. If the light is too weak, the image will appear completely black. In this case, no matter how powerful the algorithm is, it will be useless.

• How to quantify and define fatigue and sleepiness is also a bottleneck faced by DMS. How to measure fatigue? Fatigue is related to body temperature, skin resistance, eye movement, breathing rate, heart rate and brain activity. The most effective method is to measure pulse and heart rate variation HRV, but the technology in this area is not mature enough.

• The signal-to-noise ratio and contrast vary greatly, image occlusion and jitter, and light differences in different weather and time periods lead to different image brightness.

• Challenges of face detection: in-plane rotation; out-of-plane rotation; presence of makeup, beard and glasses; expressions (happy, crying, etc.); face occlusion; real-time processing requirements.

• DMS delay issues caused by insufficient computing power and communication.

• There are too many false alarms, which cause interference to users.

• There are too few sample databases to train the algorithm.

There are many scenarios that visual cameras cannot solve, so companies in the industry are trying to add more sensors and introduce more powerful chips to solve them.
 

Hardware improvements and cockpit monitoring

 In May 2019, OmniVision Technologies and Fullan Microelectronics announced a joint solution that can capture and process high-quality in-car color and infrared (IR) images with a single camera, both day and night. The solution combines OmniVision's OV2778 RGB-IR image sensor (2 million pixels) with Fullan's FH8310 image signal processor (ISP). The OV2778 image sensor has high sensitivity in all lighting conditions. The solution can provide high-quality video for automotive in-cabin monitoring systems (IMS), while integrating solutions such as facial recognition, object and unattended child detection, remote monitoring, and online car-hailing and RoboTaxi recording.

Image source: OmniVision Technologies

In June 2019, Israeli startup Vayyar launched a radar chip that can realize the location and classification of passengers in the car, the size of the occupants, vital signs and posture analysis. The in-cabin solutions include seat belt reminders, optimized airbag deployment, gesture control, driver drowsiness alerts and baby detection alerts. In
 
September 2019, ON Semiconductor launched a cabin monitoring system that will include driver monitoring and occupant monitoring functions. The solution includes three 2.3 million pixel RGB-IR image sensors from ON Semiconductor. The multi-camera cabin monitoring system uses Ambarella's CV2AQ system chip and integrates Eyeris's AI software, which can perform complex body and facial analysis, passenger activity detection and target detection tasks.

Image source: ON Semiconductor

In February 2020, ADI announced a collaboration with Jungo to develop a camera solution based on ToF and 2D infrared technology to enable in-vehicle driver and cabin monitoring. ADI's ToF technology is combined with Jungo's CoDriver software to monitor the drowsiness and distraction of people in the car by observing the head, body position, and eye gaze.
 
Valeo has also launched cabin monitoring technologies, such as rear-seat monitoring solutions based on visual technology and liveness detection solutions based on millimeter-wave radar.
 
EyeSight's Cabin Sense in-cabin monitoring solution includes detecting the number of passengers, whether passengers are wearing seat belts, monitoring passenger postures, and whether there are objects such as children or bags left in the car.
 
In short, DMS will maintain rapid growth in recent years with the large-scale application of L2/L3, and will evolve into a cabin monitoring system in the future, which will play an important role even in the L4 era.