Research and design of fatigue driving warning system based on ZigBee

Abstract: Today's fatigue monitoring system has a relatively simple monitoring method and low reliability. If multiple monitoring methods are integrated, it will be limited by the increasingly difficult wiring in the limited space of the car. In response to this series of problems, this paper proposes a fatigue driving warning system based on ZigBee wireless sensor network technology and sensor information fusion technology. The system is responsible for the establishment and management of the ZigBee network by CC2430 and coordinator, and the high-performance processor completes the fusion of fatigue information. Tests have shown that the system is suitable for vehicle operation and can better monitor fatigue behavior, with a reliability of up to 95%.

1 Introduction

With the rapid development of the automobile industry, traffic accidents related to it are also increasing rapidly. Among these accidents, traffic accidents caused by fatigue driving account for about 16% of the total, and on highways, they exceed 20% [1]. Therefore, the development of fatigue driving prevention and alarm devices has become the focus and difficulty of today's fatigue driving research. With the emergence and development of new technologies, this paper proposes a fatigue driving system based on the combination of ZigBee wireless sensor network technology and information fusion decision technology. The network established by ZigBee technology has the following characteristics: the system performance is poor due to the single monitoring method [2], and the expansion of the system is limited by various aspects such as vehicle body space wiring and cost. Therefore, by utilizing the wireless, safe, reliable, and low-power characteristics of ZigBee technology, combining single-chip microcomputer control technology with it and introducing it into the design of vehicle sensors, it can not only save the installation of communication cables and reduce the installation workload, but also realize data transmission and network interconnection safely and reliably, thereby developing more applicable vehicle sensors. In addition, the low power consumption and low cost characteristics of ZigBee technology are very suitable for driving.

2 Fatigue Detection System Architecture and Principles

2.1 ZigBee network construction and communication

Based on the short communication distance between nodes in the car, there is no need for routers to expand network coverage. This model adopts the Zigbee star network structure, which only requires a coordinator and various sensor devices to form a network, thereby reducing the complexity of the entire system. The coordinator in the middle is responsible for initiating and maintaining the network, and handing the collected information to a high-performance processor to complete the information fusion decision, and then the processor transmits the fatigue judgment result to the alarm.

(1) Lane departure detection, eye frequency detection, eye closure time detection, and yawning. The sensors first collect the original image information, then transmit the information to their respective DSP chips for processing to obtain fatigue information, and finally transmit the collected signal processing results to the ZigBee SoC module. TI's CC2430 is used here. A single CC2430 chip integrates the ZigBee radio frequency (RF) front end, memory, and microcontroller. In the receiving and transmitting modes, the current consumption is less than 27mA or 25mA respectively. CC2430 is more suitable for the requirements of vehicle-mounted systems that require very long battery life. The information from these four places is finally sent to the coordinator by CC2430. The specific implementation framework flow chart is shown in Figure 1.

(2) The Coordinator is responsible for networking and managing each terminal sensor. The basic process of networking is as follows: first, perform energy scanning and activate the scanning channel. If a suitable channel is found, create a unique 16-bit network PAN ID. In the ZigBee network system, the network short address of the Coordinator is fixed to 0. Then, it starts broadcasting network information to the surroundings, and accepts and processes requests to join the network within its network coverage, and then adds the information of new nodes. The networking flow chart is as follows, see Figure 2.

From the flowchart, we can see that the coordinator does not process and save the information sent by the sensor node. It directly hands over the fatigue information to the high-performance processor for processing, so that the coordinator can better manage the network. The processor is responsible for realizing the fusion judgment of multiple fatigue characteristics. This model unifies the design of the coordinator and the signal aggregation node (gateway). The coordinator/gateway is responsible for the communication with each terminal device and the external network. If the driver's fatigue driving is serious and it is easy to cause a traffic accident, the coordinator will send the driver's information to the gateway, and then convert it into the information format of the external network, and finally communicate with the highway safety network through GSM/GPRS and send it to the remote monitoring device.

[page]

2.2 Sensor Information Acquisition Technology

The driver's facial image is acquired through an infrared camera and LED, where the LED used for lighting can emit light with spectra of 850nm and 950nm. When different infrared spectra are used, the pupil of the eye will appear in different colors. When illuminated by 850nm infrared light, the pupil appears red, which is commonly known as the red-eye effect; while when illuminated by 950nm infrared light, the pupil appears black. Except for the pupil, the other parts of the face in the two images are the same. By comparing the two images, it is easy to locate the eyes, and then a series of image processing is performed to obtain facial parameters and realize eye tracking. In addition, the use of infrared LEDs can reduce interference from surrounding light to ensure image quality while reducing interference to the driver's vision, because its light is almost invisible. The comparison of the eyes is shown in Figure 3:

In order to obtain the two images (a) and (b) of Figure 3 at the same time, the infrared camera device shown in Figure 4 can be used. When the incident light hits the middle beam splitter (which can separate the incident laser line into two beams with a reflection/transmission ratio of 1), the incident light can be divided into two beams in parallel, and then enter the camera through 850nm and 950nm filters respectively. In this way, the two images obtained at the same time are the same except for the pupil color. In order to complete the processing of a large amount of image data in a limited time, the DSP image processing chip uses TMS320DM642, whose image processing speed can reach more than 25 frames per second, that is, it can complete the operation and processing of one frame of image within 40ms. In addition, the CCD camera is a PAL system with 25 frames per second. These devices are sufficient to complete real-time image processing and complete the extraction and calculation of the driver's fatigue characteristics such as faster blinking frequency, slow eye closure, and yawning.

Lane deviation detection is based on the behavior of the vehicle to indirectly reflect the signs of driver fatigue. We point the CCD camera towards the direction of the car's travel and monitor the direction of the vehicle's travel and the turn signal at the same time. If the car changes direction and the turn signal is not turned on, it is considered that the driver may have entered fatigue driving . The detection of vehicle behavior is not based on human performance activities, which can complement the shortcomings of facial detection and human differences. At the same time, it can also give certain reminders when the driver is distracted and makes mistakes due to other factors (such as mobile phones, music, and children) rather than fatigue.

2.3 Sensor Fusion

We use 120ms as a small cycle, because this system can collect one frame of image within 40ms, and the last 80ms is used for image processing, so there are 500 frames of image in one minute. These images are used to analyze the driver's fatigue. According to the principle of ergonomics, when the human body is fatigued, the blinking frequency is significantly faster than normal in a certain period of time. This is the driver's reaction to try to stay awake when fatigued. When entering a deeper level of fatigue, the characteristic of eye closure time will be longer. When awake, the process of opening and closing the eyes only takes a few frames to a dozen frames (within 0.25 seconds), while when tired, it takes 20 frames or one or two seconds; when yawning, the vertical radius of the mouth is significantly increased. We first collect the driver's blinking frequency, eye closure time and mouth information when normal, and then compare it with the situation when fatigue occurs to determine the degree of fatigue. We use fuzzy logic to make fusion decisions on the collected information. For example: when the Coordinator only receives abnormalities in blinking frequency and blinking time, it will be fused in the following way:

(1) Establishment of membership functions of input and output variables: For two input variables, the time of eye closure and blinking frequency, and one output variable (driver's fatigue status), three different degrees of fuzzy sets are defined respectively. For each variable, an appropriate membership function is selected. The explanation is as follows: blinking frequency = {fast, medium, slow}; blinking time = {short, medium, long}; fatigue status = {no fatigue, slight fatigue, fatigue}. In this paper, the inductive reasoning method is used to determine the membership degree, and the triangular membership function is used.

(2) Fuzzification and establishment of fuzzy reasoning rules: Fuzzification is to convert the precise measurement value into the domain corresponding to the input variable through normalization, and then convert it into a suitable fuzzy language variable, that is, membership, for fuzzy reasoning through the defined membership function. In this paper, the input variables are blinking time and blinking frequency. After processing the collected images, we get the eye opening and closing status, and convert the blinking frequency into fuzzy language such as fast blinking frequency, medium blinking frequency, slow blinking frequency, long blinking time, medium blinking time, short blinking, etc. Because we use three degrees of fuzzy sets for blinking frequency and blinking time. Therefore, at most 32=9 control rules can be obtained from each other, and the reasoning is as shown in Table 1:

(3) Fuzzy clarification: Fuzzy clarification is to convert the fuzzy variables after fuzzy logic reasoning into actual operation quantities. The centroid method is used in this paper, and its calculation formula is as follows. R: fuzzy controller output; k: number of rules; xi: membership of the i-th rule; Fi: centroid value of the i-th rule membership function. The higher the fatigue state value, the more tired. In this paper, the importance of eye closure time is higher than the blinking frequency, because the longer the blinking duration means that the eyes are closed for a longer time during the blinking process. Regardless of whether the driver is tired or not, the longer the eyes are closed, the higher the danger.

Comparison of experimental results: We used blinking time, blinking frequency and the membership value after fusion of the two features to judge the 50 videos of drivers in a fatigued state. The experimental results show that the accuracy rate is significantly improved after fusion. Similarly, when fatigue features such as yawning and lane deviation appear, we use the same method to fuse, and the accuracy rate of the system reaches 95%.

3 Conclusion

The full text gives a comprehensive introduction to the fatigue monitoring and warning system based on ZigBee technology , and gives the key technologies in the system design. The whole system has unique advantages: (1) It adopts the technology based on ZigBee wireless sensor network, avoiding the high cost and serious interference of vehicle wiring; (2) It adopts multi-sensor information fusion technology, which can better analyze and decide the driver's fatigue behavior. Therefore, the automobile fatigue driving monitoring system based on ZigBee technology has certain reference significance and good application prospects for reducing traffic accidents caused by fatigue. The innovation of this paper: It proposes and implements a fatigue monitoring system combining ZigBee network and multi-sensor fusion technology, which is suitable for vehicle operation and has good scalability, effectively improving the accuracy of fatigue judgment and enhancing the robustness of the system.