Lane Departure Warning (LDW) System Technology Introduction

People are paying more attention to automobile safety, and high-tech applications in automobiles are emerging one after another. Lane Departure Warning (LDW) system is one example. Experts point out that about 50% of automobile accidents are caused by deviation from the normal driving lane. The main reasons are that the driver is distracted, inattentive or fatigued. The LDW system will remind the driver when the vehicle is driving at high speed that the car is deviating from the normal driving lane, allowing the driver to correct the driving route in time. This is another safety device installed in the car after seat belts and airbags.

System Overview

Figure 1 is a schematic diagram of the LDW system. A camera mounted facing forward monitors the road the vehicle is traveling on, with a range of 50-70 feet. Typically, one or more CMOS image sensors provide multiple images of the road, and these sensors are connected to multiple video ports of the processor. Here, video-based signal processing replaces the analog signal chain, which not only improves the accuracy of the system, but also saves the system's material cost.

Figure 1 Schematic diagram of ldw system and external connections

After the data enters the system, it is converted into a processable format in real time. Inside the processor, it is first preprocessed to filter out the noise mixed in during image capture. Then the position of the vehicle relative to the lane markings is detected. The input information stream of the road image is converted into a series of lines that outline the road surface. The lane markings can be found by looking for edges in the data field. These edges actually form the boundaries that the vehicle should maintain as it moves forward. The processor must keep track of these markings at all times to determine whether the driving route is normal. Once it is found that the vehicle has accidentally deviated from the lane, the processor makes a judgment and outputs a signal to drive the alarm circuit, allowing the driver to immediately correct the driving route. The alarm can be in the form of a buzzer or horn, or it can be a verbal prompt, and there is also a vibrating seat to remind the driver.

The LDW system also needs to take into account the normal use of the car's braking and steering devices. These devices will affect the work of the LDW and complicate the system. Therefore, the LDW system does not work when driving slowly or braking or turning normally.

Workflow

Figure 2 describes the workflow inside the processor in more detail. In general, the LDW system must have two major functions, namely edge detection and tracking. The former is used to determine the lane markings; the latter allows the vehicle to drive along the normal lane. Before digital processing, smoothing filtering must be performed. Since image sensors are not ideal, meteorological conditions, ambient temperature, vehicle movement, and electromagnetic interference will introduce noise during the image acquisition process. Noise makes the image blurred. More precisely, pixels that originally had the same grayscale value in the original image have different grayscale values ​​in the noisy image.

In addition to noise, there is also the influence of quantization error, which can cause the edge boundary to fall on multiple pixels, also making the boundary less clear. Noise and quantization error are uncontrollable, so the input video stream must be filtered and smoothed. Without this step, it is very difficult to find clear road signs. Smoothing filtering of digital video streams should also take into account that the video stream is a real image sequence that changes at a specified rate, and the image filter should work fast enough to keep up with the continuous reception of the input image. Therefore, it is crucial that the image filter kernel optimizes execution with the least possible number of processor cycles. An effective filtering method is to use a basic two-dimensional convolution operation.

Figure 2 LDW system internal image processing algorithm flow

[page]

接下来要进行的是边缘检测。所谓边缘就是指图像局部亮度变化最为显著的部分。对于数字图像，图像灰度值变化可用梯度来表示，sobel算子是常用的边缘检测算法。发现的边缘则可确定车行道的标志。这一过程又要涉及hough变换。它是图像处理中识别几何形状的基本方法之一。最基本的hough变换是从黑白图像中检测直线。它将图像平面上的像素点映射到参考平面（即一个缓冲区）上的点，通过统计特性来确定直线的参数。

The hough transform calculates a sinusoidal curve for each pixel in the input image, so the calculation workload is very large and some techniques are needed to speed up the calculation. First, some calculation results can be calculated in advance, so they can be used as reference values ​​through a lookup table. Second, some reasonable assumptions are made about the position and nature of the lane markings in the input image. That is, if only those points that are potential lane markings are calculated, a large number of unnecessary operations can be avoided, thereby simplifying the calculation and improving the quality.

The output of the hough transform is a set of straight lines, some of which may be lane markings. Since the lane markings of many highway systems are standardized, a set of rules can exclude some straight lines from candidate lane markings. Finally, this set of possible lane markings is used to determine the position of the vehicle.

Lane tracking

Lane information comes from a variety of possible sources within a car, which, combined with measured parameters such as speed, acceleration, etc., help track the lane. Based on the measurement results, the lane system makes an intelligent decision whether an unintentional lane deviation has occurred. In more advanced systems, other parameters such as time, road conditions, and driver alertness can also be modeled.

Predicting lane geometry is a difficult problem, which is often solved by using a Kalman filter to predict the curvature of the road. The Kalman filter can predict the future state and then use it to calculate the parameters of the next post to reduce the computational load in the Hough transform.

Once the processing framework is established, system designers can add their own IPs at each decision step in the processing thread. The simplest system should take into account the attributes of the vehicle during the decision process. For example, the lane change alarm should be suppressed when turning or applying the brakes. In other words, the system should be able to determine whether the lane change is intentional or unintentional. More complex systems can also involve GPS coordinate data, driving route, time, climate and other parameters.

Application

Iteris is one of the first companies to develop the LDW system. Its AutoVue LDW system is a self-installable kit that includes wiring harnesses, speakers, switches, connectors, cables and other parts. It is said that it can be installed in three hours under normal circumstances. It is suitable for most of the Class 8 trucks with 1930 data buses used in North America. The three major European manufacturers, Evobus GMHH, Mercedes-Benz Omnibus and Man Busse, have also recognized this product.

Infini TI has developed a new LDW system that uses a small camera mounted on the front windshield, a speed sensor , an indicator and an audible alarm. The company plans to equip it on the popular FX off-road SUV. The company is also preparing to equip the next generation of luxury M45 performance sedans with the LDW system.

Mobileye's LDW system uses a single-lens image processor to determine the lane markings of the road and measure the position of the vehicle relative to the markings. It is characterized by the ability to detect various types of markings, solid, dotted, box-shaped and cat-eye. In the absence of markings, the system uses the road boundary to determine the position. The system fits three road model parameters: lateral distance, slope and curvature to improve the reliability of the alarm. The system can also work in rainy days and at night.