DMS monitors driver status by visually measuring pulse

The EU will force all new cars to be equipped with DMS starting in July 2022. The EU's requirements are divided into two items, one is driver drowsiness and attention warning; the other is advanced driver distraction warning. However, the embarrassing thing is that the EU has not yet come up with a unified DMS measurement standard, only a short sentence, driver drowsiness and attention warning' means a system that assesses the driver's alertness through vehicle systems analysis and warns the driver if needed.

 
How to quantify and define fatigue and sleepiness is a difficult problem, and determining the true value is also a difficult problem. The most effective method is pulse and heart rate variability (HRV), but most visual DMS systems are currently unable to measure pulse and heart rate variability (HRV). The US NHTSA began studying fatigue driving as early as 1999, and test drafts were released in 2009 and 2012, but both included lane departure data.

This method is a late-developing method, lacking the ability to predict in advance, and is not very meaningful. It is also difficult to measure. How can a driver pretend to be sleepy? The 2012 NHTSA document also mentioned that pulse measurement is the best, most authoritative, and unanimously recognized method in the medical community. At present, most visual DMS use eye information such as line of sight position, number of blinks, eye closure time, eyelid closure degree, etc. to judge whether a person is tired or sleepy. Wearing glasses, sunglasses, and myopia glasses will seriously affect the robustness of the algorithm. This eye-based information varies from person to person. People's eyes are too different, so it is difficult to accurately reflect a person's fatigue or sleepiness state, and it is also difficult to come up with a standard that everyone agrees on. They can only act independently, and each company has its own set of standards. The testing standard is even more difficult. How to define the ground truth value? Pulse is the best ground truth. If it is pulse measurement, it is very simple. You can measure the driver's pulse in contact. This is a very easy to quantify truth value. By comparing it with it, you can see the pros and cons of the system.
 
Euro NCAP and Australian NCAP have also added DMS as a bonus item for assisted driving, but there are currently no testing standards and data can only be provided unilaterally by manufacturers.
 

In recent years, with the rise of PPG technology, visual direct pulse measurement technology has begun to appear and gradually mature. The world's first commercial visual pulse measurement technology has appeared and is applying for FDA certification. To avoid suspicion of advertising, I will not mention the name of the company for the time being. I can only reveal that it is a start-up company in Hsinchu, Taiwan. It can not only be used for DMS, but also the driver's health information can be monitored in real time and sent back to the headquarters to prevent accidents caused by sudden heart attacks, which is especially necessary in the bus field. The prospects in the medical field are extremely broad. At present, pulse measurement requires skin contact, which is not suitable for children, burn patients, fever patients, mental patients or elderly people who do not want to go to the hospital at home. Nurses can provide contactless care. It also has great application potential in the field of face payment. Using facial recognition and pulse measurement can determine whether it is a live body and increase payment security. It is no longer possible to fool people with pictures or videos. It also has a future in the security field.
 

It can also further judge people's emotions. People's emotional fluctuations may not be seen on their faces, but changes in pulse and heart rate make it very difficult to lie. It can even be used as a simple lie detector. It can also play a huge role in the current epidemic. The accuracy of forehead thermometers is not high, and mobile phones cannot measure body temperature. Pulse is more reliable than body temperature. The immune system increases activity and breathing difficulties will inevitably accelerate the heartbeat. In the future, mobile phone cameras can also measure pulses, while the current so-called health codes are completely based on self-consciousness and have extremely low reliability.

Photoplethysmography (PPG) is a method that uses photoelectric technology to detect changes in blood volume caused by heartbeats under human skin tissue. Changes in blood volume reflect the shape of the pulse wave. Some mobile phones can use cameras to detect pulses using this technology, but with current technology, mobile phones still require fingers to be placed close to the camera. PPG technology only requires some photoelectric equipment: a light source to illuminate human tissue (skin) and a light detector to detect light intensity. When light of a certain wavelength hits the surface of the skin, part of the light is absorbed by the human tissue, and the photoelectric receiver receives part of the light after transmission and reflection. The facial skin is the most sensitive, and the capillaries on the face are the most dense (the face is always exposed to the air, and needs enough capillaries to heat and keep warm).
 

According to the Lambert-Beer law, when monochromatic light enters human tissue, the light intensity absorbed by non-pulsating tissues such as skin, muscle, and soft tissue does not change over time; while the blood volume in the artery fluctuates periodically with the diastole and contraction of the heart, so the light intensity absorbed by it changes synchronously with the heart beat. At this time, the light intensity detected by the light detector is also a fluctuating signal. Finally, the photoelectric conversion device is used to convert this fluctuating signal into an electrical signal, in which the fluctuating component generated by the change in blood volume is the PPG signal.

As shown in the figure, there are two modes of PPG. In this process, most of the light is absorbed by bones, tissues and blood. At the same time, blood absorbs more light than other surrounding tissues, so the light intensity detected by the light detector will be weakened. The key factors affecting the intensity of light absorbed by blood are: blood volume, movement of blood vessel walls, and direction of red blood cells. After years of research and analysis, it is known that skin is the easiest tissue to use when obtaining PPG signals. The light beam is incident on the surface of the skin, and part of it is absorbed and scattered by the skin tissue. The scattering and absorption are mainly determined by the pigments (hemoglobin, melanin, etc.) in the skin tissue; and the other part of the light beam that is not absorbed will eventually pass through the skin surface again and enter the air. The intensity of the light that is not absorbed can be detected by the light detection device on the skin surface to form a PPG signal.
 

The absorption of light spectrum by tissues in different parts of the human body is different in the human circulatory system: because the blood volume in the blood vessels fluctuates synchronously with the beating of the heart, the absorption of light by the blood fluctuates. However, the absorption of light spectrum by tissues such as skin, muscles and bones is basically constant. In a cardiac cycle, when the heart contracts, blood is ejected into the arteries, the amount of light absorption increases, and the detected light intensity decreases accordingly; while when the heart relaxes, the opposite is true, and the detected light intensity increases. In this way, the light intensity received by the optical receiver changes pulsatingly, and the pulsating changes presented are the waveform of the volume pulse wave.
 

The PPG signal waveform carries a lot of physiological information, so the physiological information of the human body can be extracted from the PPG as the basis for clinical diagnosis and treatment. In traditional Chinese medicine, "pulse diagnosis" as a key means of diagnosing diseases is the earliest application of pulse waves. In 1938, Hertzlman first proposed the photoplethysmography method (PhotoPlethysmography), and derived a peak time from the rising edge of the PPG pulse wave, and normalized it to the heart rate. Professor Luo Zhichang of Beijing University of Technology and others have been committed to the research of pulse wave engineering, and developed a blood flow parameter monitoring system based on the photoplethysmography method based on the relationship between pulse pressure and volume pulse blood flow.

Not only pulse, but also HRV, or heart rate variability, refers to the tiny variations in the human heart beat cycle, usually the tiny fluctuations in the R wave interval between successive heartbeats. Studies have shown that the modulation of the cardiac autonomic nervous system by factors such as the brain's higher neural activity and the spontaneous rhythmic activity of the central nervous system leads to the generation of HRV. The HRV signal contains a lot of information about the neural and fluid regulation of the cardiovascular system. By detecting the data of the patient's heart rate variability, a lot of information about the balance state of the autonomic nervous system can be obtained. The proposal and research of HRV is a breakthrough and achievement of great significance in the history of modern medicine, especially for the early diagnosis, treatment and prognosis evaluation of some cardiovascular diseases and other related diseases. The clinical value of HRV analysis and its potential is remarkable.