Application and development of Gosuncn license plate recognition system in the field of intelligent transportation

As an important part of safe cities and smart cities, intelligent transportation systems (ITS) are emerging and developing rapidly in major cities. The main object managed by ITS is vehicles. Under the current vehicle control system, the car license plate number is the only identity identification of the vehicle (Note: Although RFID and other technologies can also identify vehicles, they have not yet been extended to the entire vehicle management system, and the vehicles need to be modified, so they are not universal at present). Automatic license plate recognition technology can realize automatic registration and verification of the vehicle identity without making any changes to the car, so the license plate recognition system is one of the important components of the modern intelligent transportation system.

The main application of license plate recognition system in intelligent transportation is in the intelligent monitoring and recording of highway vehicles and automatic recording of violations (including running red lights, no left turn, crossing the line, driving in the wrong direction, speeding, etc.). It is used to record the license plate of the vehicle, and then index the vehicle file and the owner file for traffic management and law enforcement, and can be derived to the application of checking fake license plate vehicles. In addition, license plate recognition technology can also be applied to various occasions such as highway toll collection, parking management, weighing system, traffic induction, highway inspection, vehicle dispatching, vehicle detection, etc.

License plate recognition systems are widely used and also face various scenarios and special situations. How can license plate recognition technology better serve these applications? Are there any new problems? Let’s start with how license plate recognition systems work.

How license plate recognition works

License plate recognition is a pattern recognition technology that uses dynamic video or static images of vehicles to automatically identify license plate numbers and license plate colors. The core of the technology includes license plate positioning algorithms, license plate character segmentation algorithms, and optical character recognition algorithms. A complete license plate recognition system should include vehicle detection, image acquisition, license plate recognition, and other parts. When the vehicle detection part detects the arrival of a vehicle, it triggers the image acquisition unit to collect the current video image. The license plate recognition unit processes the image, locates the license plate position, and then segments the characters in the license plate for recognition, and then forms the license plate number output.

The vehicle detection part usually uses ground sensing coils or radars. Some license plate recognition systems also have the function of judging whether there is a car through video images, which is called video vehicle detection.

Since the road is open to traffic 24 hours a day, the license plate recognition system needs to work around the clock and all-weather. In order to ensure the accuracy of recognition at night, it will be equipped with LED strobe lights or flash lights to provide fill light.

The structure of a typical license plate recognition system is shown in the figure below. The front-end device is connected to the back-end platform through a transmission network.

Figure 1: The structure of the license plate recognition system (coil triggering method)

The license plate recognition system usually completes the recognition output through the following steps:

Vehicle detection: buried coil detection, infrared detection, radar detection technology, video detection and other methods can be used to sense the passing of vehicles and trigger image acquisition and capture.

Image acquisition: Real-time and uninterrupted recording and acquisition of passing vehicles is carried out through a high-definition camera capture host.

Preprocessing: noise filtering, automatic white balance, automatic exposure, gamma correction, edge enhancement, contrast adjustment, etc.

License plate location: Perform row and column scanning on the grayscale image after image preprocessing to determine the license plate area.

Character segmentation: After locating the license plate area in the image, the character area is accurately located through grayscale, binarization and other processing, and then the characters are segmented according to the character size characteristics.

Character recognition: Scaling and feature extraction of segmented characters, and matching and identification with standard character expressions in character database templates.

Result output: Output the results of license plate recognition in text format.

Figure 2 License plate recognition process

Functions and applications of license plate recognition system

The main functions of license plate recognition systems in the current market include:

1) Automatic recognition of vehicle license plates, including complete license plate information, color, characters, Chinese characters, and numbers;

2) Automatic detection of vehicle speed;

3) Identification and alarm of illegal black-plate vehicles;

4) Linkage control between vehicle identification information and vehicle information of the vehicle management office;

5) Vehicle driving direction judgment and monitoring.

The main application areas of license plate recognition systems in the current market include:

1) Intelligent traffic management at traffic intersections;

2) Automatic collection of traffic information;

3) The police and other law enforcement agencies set up temporary inspection stations to inspect vehicles passing through and give priority to identifying vehicles to be inspected;

4) Automatic toll collection systems at checkpoints such as roads, bridges, and tunnels;

5) Automobile entrance and exit management of modern residential areas, parking lots, and important government agencies;

6) Photo capture and recognition at road security checkpoints, and traffic flow monitoring.

Implementation of License Plate Recognition System

There are usually three types of implementation schemes for license plate recognition systems. The first is a snapshot camera + industrial computer, which is the earliest application scheme. The camera (coil, radar trigger or video detection) captures the vehicle image, and the software in the industrial computer recognizes the license plate. An industrial computer can manage multiple cameras (multiple lanes) at the same time; the second is a snapshot camera + embedded analysis host, which changes the less reliable industrial computer to an embedded host, while the snapshot camera does not need to be changed, so this method has gradually replaced the first scheme; the third is an embedded integrated snapshot camera, which integrates capture, control, recognition, recording, compression, and transmission, greatly simplifying the work of managing terminal equipment and back-end platforms. At the same time, it has been improved in terms of reliability, security, installation and maintenance convenience, and environmental adaptability. It is becoming the most promising implementation scheme.

Table 1 Comparison of implementation schemes for license plate recognition systems

Project capture machine + industrial computer capture machine + embedded host integrated capture machine

System reliability Windows system, low reliability, easy to be infected by virus Embedded Linux system, high reliability Embedded Linux system, high reliability

Environmental adaptability is generally required to be between -10℃ and +55℃. Generally adaptable to harsh environments, requiring between -30℃ and +70℃.

Data security is encrypted in the industrial computer, security is generally encrypted in the embedded host, security is better, it is encrypted in the capture machine, security is good

Convenience of use Complex structure, large volume, troublesome installation and maintenance Compact structure, easy installation Integrated and easy installation

Power consumption is nearly 100 watts, about 35W<15W

Here we will talk about the high-definition snapshot camera products that integrate vehicle video detection and license plate recognition that are currently available on the market. Compared with the traditional coil trigger method, the use of video detection can avoid damaging the road surface, does not require additional external detection equipment, does not require correction of the trigger position, saves costs, and can more flexibly detect vehicle behavior (such as illegal U-turns, crossing the line, turning left, etc.), and is more suitable for mobile and portable applications. However, the license plate recognition system needs to have a very high processing speed and use excellent algorithms to achieve image acquisition and processing without basically losing frames in order to perform video vehicle detection. If the processing speed is slow, it will cause frame loss, making it impossible for the system to detect vehicles traveling at a faster speed. At the same time, it is also difficult to ensure that the recognition process starts at a position that is conducive to recognition, affecting the system recognition rate. Therefore, combining video vehicle detection with automatic license plate recognition has certain technical difficulties, which is why there are not many products available at present.

In addition to the front-end capture machine and analysis host, a license plate recognition system should also have a corresponding back-end management system, which affects whether the license plate recognition system is easy to use. The functions of the back-end management system usually include:

1. Reliable storage of recognition results and vehicle image data. When the multifunctional system operation causes network errors, it can protect the image data from being lost and facilitate manual investigation afterwards;

2. Effective automatic comparison and query technology. The identified license plate number must be automatically compared with thousands of license plate numbers in the database and an alarm will be prompted. If the license plate number is not read correctly, fuzzy query technology must be used to obtain a relatively "best" comparison result;

3. For networked operation, it provides functions such as real-time communication, network security, remote maintenance, dynamic data exchange, automatic database update, hardware parameter setting, and system fault diagnosis.

Evaluation of License Plate Recognition System

my country's Ministry of Public Security has clear technical specifications and management regulations for road traffic and vehicle driving monitoring, mainly including "General Technical Conditions for Automatic Red Light Running Recording Systems" (GA/T 496-2009), "General Technical Conditions for Intelligent Monitoring and Recording Systems for Highway Vehicles" (GA/T 497-2009), "Technical Specifications for Image Evidence Collection of Road Traffic Safety Violations" (GA/T 832-2009), "Technical Specifications for Automatic Recognition of Motor Vehicle License Plate Images" (GA/T 833-2009), etc.

There are two main indicators for technically evaluating a license plate recognition system, namely recognition accuracy and recognition speed.

Recognition accuracy

The most important indicators of whether a license plate recognition system is practical are the recognition rate and recognition accuracy. According to the definitions in GA/T 497-2004 and GA/T 497-2009:

Recognition rate = number of vehicles whose license plates are automatically recognized / total number of vehicles with valid license plate information.

Recognition accuracy = number of vehicles with correct license plate information / total number of vehicles with valid license plate information.

(Note: Valid license plate information means that the vehicle license plate is complete, clear, installed in accordance with regulations, and is not blocked or damaged.)

In the newly issued GA/T 497-2009, the license plate recognition rate indicator evaluation has been cancelled, and only the license plate recognition accuracy rate is retained. It is stipulated that the vehicle license plate recognition accuracy rate during the day should be no less than 90%; the vehicle license plate recognition accuracy rate at night should be no less than 80%.

Since the conclusion that the license plate information is valid requires manual judgment, the vehicle license plate image and recognition results need to be stored for easy retrieval and review. Then, by counting the number of actual vehicle images and correct manual recognition results, we can obtain the recognition rate and recognition accuracy, as well as the credibility and false recognition rate.

In order to test the recognition rate of a license plate recognition system, the system needs to be installed in an actual application environment, run for more than 24 hours a day, and collect license plates of at least 1,000 vehicles in natural traffic for recognition.

Recognition speed

The recognition speed determines whether a license plate recognition system can meet the requirements of real-time practical applications. If a system with a high recognition rate takes several seconds or even minutes to recognize the result, then this system will be meaningless because it cannot meet the real-time requirements of practical applications. For example, one of the functions of license plate recognition applications in highway toll collection is to reduce travel time. Speed ​​is a strong guarantee for reducing travel time and avoiding lane congestion in this type of application.

According to the requirements of GA/T 833-2009, the recognition time is ≤ (A/B) × (K × 100) (ms).

In the above formula, A represents the image resolution used for recognition; B is a fixed constant whose value is 768×576=442368; and K is the number of license plates in the image.

That is, when the license plate image is 768×576 pixels, when there is one license plate in the image, the recognition time is ≤100ms; when there are two license plates in the image, the recognition time is ≤200ms; when there are three license plates in the image, the recognition time is ≤300ms; when there are four license plates in the image, the recognition time is ≤400ms.

Problems of license plate recognition system and application of new technology

As can be seen from the above, the higher the recognition rate and accuracy of the license plate recognition system, the better. However, we must clearly realize that it is impossible to achieve a 100% recognition rate. On the one hand, the recognition effect will be seriously affected by the license plate being damaged, blurred, blocked, or bad weather (snow, hail, fog, etc.). On the other hand, the segmentation and recognition of some Chinese and English characters are difficult, such as the character "川" which is easy to be segmented incorrectly, and the easily confused characters such as "0-Q", "2-Z", "4-A", "5-S", "7-T", "8-B", and "OD". Because the statistics of the recognition rate are based on the total number of vehicles with valid license plate information, if the license plates in various environments and situations are taken into account, the recognition rate of the license plate recognition system in actual application will be greatly reduced, and it still relies on manual judgment and recognition when it cannot be recognized.

In view of the disadvantage of low character recognition rate in traditional license plate recognition algorithms, a recognition method based on convolutional neural network has emerged. By learning samples of license plate character images and optimizing the weight parameters of each layer of the neural network, the character recognition rate of license plates can be greatly improved. The simulation results show that the correct recognition rate of characters in license plates can reach 99% by using the convolutional neural network recognition method. The recognition rate and anti-interference performance are significantly better than traditional recognition methods such as structural feature method and template matching method (the latter two are only 94% and 95% respectively).

Taking advantage of the neural network, an improved recognition mechanism based on convolutional neural network is used to recognize the characters in the license plate. This recognition method optimizes the weight parameters of each layer in the network system by learning the license plate character images under ideal preprocessing conditions, greatly improving the recognition rate of characters in the license plate. In practical applications, if the network structure is further optimized, the license plate characters under poor preprocessing conditions can be recognized for the shortcomings of unclear license plate positioning and character segmentation errors in the early preprocessing.

Author: Safe City and Intelligent Transportation Division of Gosuncn Technology Group Co., Ltd.