A brief discussion on Bosch's cockpit monitoring technology

An in-cabin monitoring system shows how it detects where a driver is looking, as indicated by the yellow line. If the driver looks too long at non-“safe areas” such as the side mirrors, rearview mirrors and outside the windshield, the system may issue a warning.

While Bosch is working on advanced systems such as fatigue driving detection, it is also committed to providing drivers with more value-added functions, such as business information of nearby coffee shops.

John Nowinski, North American project leader for Bosch's cockpit sensing system, raised an extreme scenario that the cockpit system may face when trying to avoid drunk driving and impaired driving. In some cases, such as when the driver who just got in the car is being attacked and violated from outside the car, is it a better solution to hand the car over to a human driver?

John Nowinski discussed various aspects of the company's research into cockpit sensing and how it can be used for safety, utility and comfort purposes. The product area head of cockpit sensing solutions discussed the difficulties of impaired driver detection. On the eve of the Consumer Electronics Show (CES 2024) in Las Vegas, Nowinski said at a Bosch press conference that this is the type of problem that the National Highway Traffic Safety Administration (NHTSA) hopes to solve in future impaired driving detection systems. The cockpit technology introduced by Bosch at the conference can not only avoid dangerous driving behaviors such as driving under the influence, but also bring more value-added functions to drivers.

Such scenarios not only make it more difficult for automotive engineers to solve the problem of impaired driving detection, but also make it difficult for regulators to set standards for the technology, he said. NHTSA issued an Advance Notice of Proposed Rulemaking (ANPA) in December 2023 as the first step in implementing the final fatigue driving detection standard, after which the technology will gradually become standard for all newly sold light vehicles. "One of the biggest difficulties in setting standards is to stipulate how to detect impaired states, because once regulations are made, it will limit the exploration of other potential solutions," said Nowinski. He said that the ultimate goal of the technology is not to determine the driver's blood alcohol content (BAC), but to determine "whether the driver has the ability to drive." To this end, Bosch combines infrared cameras and RGB cameras to accurately measure the degree of opening and closing of the driver's eyes (such as drooping eyelids may mean that the driver is in an unconscious or sleepy state), changes in mouth shape (such as frequent yawning may indicate that the driver is sleepy), and even the decline in scanning speed (that is, the speed of changing the direction of the eyes, which may indicate that the driver is drunk driving) to determine whether the driver has the ability to drive.

Nowinski said that even the testing and training for developing such systems are difficult because the subjects need to reach a certain level of intoxication in a medical sense. In this regard, Bosch's Internet of Things (IoT) team is working with Swiss medical institutions to carry out this research. He also said that he personally believes that passive breath detection technology will not be used in the final solution. But competitors such as Magna are developing this technology-part of the reason why Magna acquired Veoneer autonomous driving technology in 2023 is to obtain this technology. Competition in the field of cockpit monitoring is very fierce, and Bosch's competitors include not only Magna, but also Continental, Infineon, Gentex and other companies.

Lifesaving car radar

At the conference, Bosch's cockpit technology demonstration car also demonstrated a 60 GHz cabin monitoring radar that can monitor a range of targets, from whether the driver is holding items that may cause distraction (such as mobile phones, food) to changes in the breathing patterns of the driver, passengers and even babies.

The breathing pattern detection function can identify drivers in danger, so that the vehicle can be stopped safely and emergency services such as 911 or emergency contacts can be contacted. Verena Ihring, director of strategy and business development for Bosch Cockpit Sensing Cross-Domain Computing Solutions, demonstrated the radar's detection function for infants in rear-seat child safety seats facing backwards. The radar signal can pass through all objects except solid metal (including swaddling blankets). But even if the safety seat is made of metal, the radar system can identify it by forming a complex radar wave reflection path in the car.

The radar and cameras (1.5 megapixel camera for driver monitoring and 5 megapixel camera for multi-passenger monitoring) can also be used to sense the weight of passengers and deploy airbags at different speeds based on their weight and sitting position, ensuring the safety of smaller passengers and passengers in non-standard sitting positions (such as front seat passengers with their feet on the dashboard). Some of the cost of this technology can be offset by removing the relatively simple seat bottom weight sensors that are currently used only to turn airbags on and off.

Ihring also introduced the installation positions of these cameras in the cabin, saying that the camera monitoring the driver should be installed at a lower position, and the camera monitoring multiple passengers should be installed at a higher position. She also introduced that the monitoring system can detect people outside the car and whether they have entered the car in an abnormal way, and thus determine whether the driver has encountered the theft or attack mentioned at the beginning of this article.

Connect with the outside world

Bosch believes that cockpit detection technology combined with cloud computing can provide users with more value-added functions. Nowinski gave an example: after the system detects that the driver may be drowsy, it can use Amazon cloud computing to ask the driver if he needs a cup of coffee when he gets home, and send the coffee-making instructions to the networked coffee machine at home. Nowinski also gave another example: after the driver saw a coffee shop, the system detected his gaze on the coffee shop, and the driver immediately asked the system: "Alexa, is that store crowded?" At this time, the system will infer the current customer flow of the store based on the number of mobile phones currently in the store or the customer data received in the past hour. Stefan Buerkle, President of Bosch Cross-Domain Computing North America, emphasized that any improved function must be based on safety. "The industry's mission is to create connected experiences for end customers, but it must ensure that these experiences do not distract drivers or cause traffic accidents."

Other innovative technologies

Bosch announced a series of automotive and non-automotive products and innovative technologies at the official CES press conference. What can be confirmed is that in the face of climate change, Bosch has not shied away from the issue of sustainable development and recognizes its necessity. In terms of automobiles, Tanya Ruckert, member of the Bosch Management Board, mentioned the following three topics:

Automated Parking Charging: This system uses Bosch's automated parking system to drive the vehicle to the location of a robotic charging device, automatically connect to the power source, unplug the power source when charging is complete, and return to its parking space.

Bosch is continuing to develop silicon carbide chip technology, which is more efficient for use in electric vehicles. It is currently building a $1.5 billion factory in Roseville, California.