Smarter cars can learn and interact with drivers

    Can cars recognize human emotions? In the past, this was impossible. But now, France is developing such technology, using cameras installed on car dashboards to recognize human expressions.

    The Swiss Federal Institute of Technology in Lausanne and PSA Peugeot Citroën are currently testing the technology in a test vehicle.

    The camera installed behind the steering wheel detects human facial expressions and compares them with the anger expressions preset by the system. Once the system determines that the driver's expression is "angry" or "anxious", the system will prompt the driver to take a short break; in extreme cases, the advanced driving assistance system will temporarily take over driving.

    However, the technology is not yet perfect. The main problem is the recognition accuracy. Sometimes the driver's mood is normal but the system still judges him as "angry" because of the occasional facial expression.

    In order to improve recognition, the research team plans to use two high-definition cameras to detect the driver's facial expressions and accelerate processing through a high-performance image processor (GPU). When the light inside the car is dim at night, infrared illumination will be used for further recognition. At the same time, the researchers will also equip the system with a "self-learning" function. As the system and the driver get used to each other, the system will be able to learn the driver's multiple expressions representing the same emotion, thereby making the relevant system operate more accurately.

    The next step for the researchers is to enable the system to have the function of "lip reading", which can also cooperate with the active safety system and voice control system in the car, so that the car can better "understand" the driver's intentions. However, to achieve this goal, the application of advanced noise separation technology is also required.

    Volvo Cars is also developing a similar system that can detect whether the driver is driving tired or distracted. It also installs sensors in the instrument panel to monitor the driver's blinking frequency, head position and head tilt.

    Volvo calls this type of technology Driver State Estimation, and it believes it is an essential technology in future self-driving cars. In some complex sections where autonomous driving is not possible, the car must "know" whether the driver is capable of taking over the control of the car.

    When situations such as following too closely to the vehicle in front or drifting out of the lane occur, the car must be able to "wake up" a drowsy driver or even make corrective actions on its own.

    Since the car system can detect the driver's attention state, the safety system can work according to this information. For example, if the system detects that the driver is in a highly focused state, in some cases, the system will not actively intervene; otherwise, the system will intervene.

    Volvo's driver status detection system shines an infrared light on the instrument panel to the driver. The human eye cannot see infrared light waves, so this system will not interfere with the driver's vision.

    The driver status detection sensor can also adjust the direction of interior lighting or exterior headlights by detecting the driver's eye movement - the headlights or interior lighting will shine towards the position where the driver's line of sight is focused. For example, if the driver wants to see the appearance of the passengers in the car clearly, or wants to see the pedestrians about to cross the road on the side of the road, the interior and exterior lighting system based on the eye sensor can understand the driver's intention.

    Volvo Cars is currently verifying this type of technology in test vehicles.