Application of real-time video fog penetration technology in video surveillance

PM2.5, a professional term in the meteorological field, has recently become a popular topic of social concern. Liquid droplets and small solid particles in the air not only harm human health and cause frequent traffic accidents due to smog, but also significantly reduce the quality of outdoor surveillance videos. In smog weather, the image color is dim, the contrast becomes low, and the details of some important targets are submerged in the fog and difficult to detect, which greatly affects the practicality of the video surveillance system.

Therefore, removing fog from videos and improving image quality has become an increasingly important key technology to enhance the application value of outdoor video surveillance systems.

1. Impact of fog on video surveillance

It is generally believed that the atmospheric medium is mainly composed of air molecules, water vapor and aerosol. Aerosol is a dispersion system composed of small particles suspended in gas. Some particles have high hygroscopicity and act as water vapor condensation centers. Their size is related to the relative humidity of the environment, the supply of water vapor and the degree of condensation caused by collision. Due to the different sizes, types and aggregation degrees of various particles in the atmosphere, various weather conditions such as clearness, haze, fog, cloud and rain are generated.

Under clear weather conditions, the light reflected from the surface of an object will basically not be affected by various components in the atmosphere and will not be scattered, absorbed, reflected, etc. It can directly reach the imaging device to obtain a clear and fog-free image.

In foggy weather, the light reflected from the surface of the object will be affected by the suspended particles in the air when reaching the imaging device. Aerosol particles are the main factor in the formation of haze and the root cause of image quality degradation. The impact is mainly in the following aspects:

(1) Aerosol particles scatter light. The scattering loss causes the intensity of the “transmitted light” to decay, resulting in a decrease in the contrast of the image.

(2) Due to the non-uniformity of aerosol particles, the spherical wave is distorted into an aspherical wave, causing the image to become blurred and the edges and details to be reduced.

(3) Aerosol particles are relatively large in size, and their own imaging cannot be ignored, which can be roughly understood as "noise".

(4) The scattered part of the imaging light by aerosol particles will be superimposed on the original forward scattered part due to the effect of multiple scattering, resulting in a certain degree of blur.

2. Comparative analysis of real-time video fog penetration technology and other fog penetration technologies

Currently known fog-penetrating algorithms can be roughly divided into two categories: one is a non-model image enhancement method, which achieves the purpose of clarity by enhancing the contrast of the image to meet the requirements of subjective vision; the other is a model-based image restoration method, which examines the causes of image degradation, models the degradation process, and uses inverse processing to ultimately solve the image restoration problem.

At present, typical methods for fog penetration processing through enhancement include: histogram equalization, filter transformation method and fuzzy logic-based method. Histogram equalization method, in which the global method has a small amount of calculation but does not enhance the details enough; the local equalization method has a better effect, but may introduce block effects, large amount of calculation, noise amplification and the problem of difficult to control the algorithm effect. The filter transformation fog penetration algorithm can obtain relatively good processing results through local processing, but they have huge amount of calculation, high resource consumption, and are not suitable for equipment with high real-time requirements. The fog penetration effect of the fuzzy logic-based method is not ideal.

The enhancement-based method can improve the image contrast to a certain extent and improve the recognizability by enhancing the region of interest. However, this method fails to compensate for the cause of the image degradation process, so it can only improve the visual effect but cannot achieve a good fog penetration effect.
At present, the image restoration methods mainly include the following categories: filtering method, maximum entropy method and image degradation function estimation method. Filtering methods such as the Kalman filter method have a large amount of calculation overall. The maximum entropy method can obtain a higher resolution, but it is nonlinear, has a large amount of calculation, and is difficult to solve numerically.

Most image degradation function estimation methods are designed based on certain physical models (such as atmospheric scattering models and polarization characteristics of fog penetration models), and require the acquisition of multiple images at different time points as reference images in order to determine multiple parameters in the physical model and ultimately solve for the result image in a fog-free state. This limits the application of such methods in real-time monitoring.

Security products are now used in various complex scenes and inclement weather. All-weather real-time monitoring places stringent requirements on product portability, power consumption, processing effects, and processing adaptability. Good video fog penetration technology should integrate the technical advantages of image enhancement and image restoration based on the atmospheric transmission model, so as to obtain a more ideal image effect and be cited in actual engineering.

In image processing, the following model is generally used to express the foggy image we see:

I(X)=J(X)t(x)+A(1-t(x))

I represents the intensity of the image seen, J is the intensity of the scene light, A is the atmospheric light component, and t is used to describe the part of the light that is not scattered when passing through the medium. The goal of fog penetration is to recover J, A, and t from I, where J corresponds to the resulting image after fog penetration. Among them, J(X)t(x) is called the direct attenuation term, which represents the part of the scene light after attenuation in the medium, and A(1-t(x) is the atmospheric light component caused by forward scattering.

After fully analyzing the advantages and disadvantages of fog penetration theory and conducting in-depth research and exploration, Hikvision has developed a real-time video fog penetration technology based on the special requirements of video image fog penetration in the security monitoring field. This technology is based on the principle of atmospheric optics, distinguishes the depth of field and fog density in different areas of the image, and performs filtering processing to obtain accurate and natural fog-penetrating images.

This real-time video fog penetration technology can automatically adjust according to changes in fog conditions to adapt to various scenarios, avoiding the situation where the near-field fog penetration is too dark and the distant field is blurred; at the same time, it takes into account the efficiency and complexity of implementation, ensuring the real-time and engineering feasibility of the entire fog penetration.

At the same time, this real-time video fog penetration technology can not only effectively remove the impact of fog, but also avoid color errors caused by over-processing of certain scenes, as well as the unreality caused by excessive fog removal. As shown in Figure 2-1, a is an image containing mist, b is the effect of the traditional fog penetration method, and c is the effect after processing with real-time fog penetration technology. It can be seen that since the fog itself is not thick, the traditional fog penetration method b lacks adaptive ability, and the red frame appears black and color distorted, while the image effect in the red frame in c is natural and realistic.

The advantages of real-time video fog penetration technology compared with other fog penetration methods are mainly reflected in the following aspects:

1) Strong fog penetration ability. Real-time video fog penetration technology can accurately remove fog to a certain degree according to different depths of field. Traditional image enhancement methods may have a problem that the fog penetration effect of the near view is better, but there is still a lot of hazy fog left in the distant view; the fog penetration effect based on the image restoration method is related to the accuracy of the selected components. If the components are selected accurately, the fog penetration processing effect is better. If the components are not selected accurately, very bad results may occur. Therefore, the fog penetration algorithm performance of this method is not stable enough.

2) Good transparency. The image processed by real-time video defogging technology is very transparent and has good contrast. However, other methods have residual fog due to inaccurate depth of field estimation, making the image after defogging not very transparent.

3) High degree of detail preservation. The real-time video fog penetration technology includes special processing to preserve details, so the image after fog penetration can continue to retain or even partially enhance the details originally hidden behind the fog, which is difficult to achieve with other methods.

4) High color saturation and strong restoration ability. Real-time video fog penetration technology does not change the color tone of the image but only increases its color saturation. Other fog penetration methods may cause color distortion.

5) It will not cause the image to be dark. Since the real-time video fog penetration technology includes brightness enhancement performance, the resulting image will not be dark. Other methods may cause the problem of reduced contrast.

6) Wide range of applications. Real-time video defogging technology can also be used to process fog-free images, improve the contrast and saturation of the original image, and at the same time improve the transparency of the image, thereby improving the visual quality of the image.

3. Application prospects of real-time video fog penetration technology in the field of security monitoring

Real-time video fog penetration technology can improve the quality of video surveillance from multiple angles. First of all, it is a fog penetration technology that can be used for fog penetration treatment in various weather conditions caused by aerosols; it is also an enhancement algorithm that can significantly improve the contrast of the image, making the image transparent and clear; it can significantly enhance the image details so that the originally hidden image details can be fully displayed; it can improve the image saturation, making the image color bright and lively, and the image after fog penetration treatment maintains accurate color tone and natural appearance, thus obtaining good image quality and visual experience.

Real-time video fog penetration technology has strong engineering capabilities. This fog penetration technology can be applied to various resolutions including megapixel-level high-definition images; it can ensure real-time and accurate fog penetration processing at various resolutions; it can evaluate the current fog concentration in real time according to the target scene and adaptively adjust the fog penetration intensity, regardless of the change of fog conditions in the scene; because this fog penetration technology can obtain accurate depth of field information, it can accurately remove non-uniform fog in the same scene without residual fog penetration defects. Judging from the current test results, this technology has a good application prospect in outdoor video surveillance systems.

The real-time video defogging function has been initially implemented on some cameras. The actual defogging processing effects are shown in Figures 2-2 and 2-3. They are the two sides of an image, the left side is the image after defogging, and the right side is the original image without defogging. It can be clearly seen that the image on the left has been significantly improved in details, transparency, color and other aspects.

Real-time fog penetration technology can be combined with video compression and intelligent analysis technology to bring greater value. Since the current mainstream video compression algorithms are lossy compression, they will damage the details with low contrast in the image. Foggy videos generally have low contrast and few details, so they are often blurred and cannot be restored after encoding and compression. The use of real-time fog penetration technology can effectively enhance the contrast and details of the image, ensure that valuable information will not be lost by encoding and compression, and significantly improve the effectiveness of the information. It is similar for intelligent analysis. The error rate of the analysis results of images processed by real-time fog penetration technology, especially the missed reporting rate, can be significantly reduced, thereby greatly improving the practicality of the intelligent analysis system.

With the development of industry and its impact on the climate, haze has become an increasingly common weather phenomenon, which has a great impact on the image quality of outdoor monitoring systems. At present, in actual projects, radar, infrared and other means are mainly used to perceive and monitor targets in haze climates. Due to technical and cost limitations, radar, infrared and other means have not yet achieved ideal solutions in the field of video surveillance. The emergence of real-time video fog penetration technology can bring multiple values ​​to the field of video surveillance.

From the perspective of application scenarios, real-time video fog penetration technology can be used in a variety of outdoor occasions, such as accident-prone areas on highways and checkpoints on highways; auxiliary driving facility monitoring areas for buses; key places and areas of concern to public security organs; key monitoring areas of power plants and power transmission equipment; primary and secondary school campuses, urban commercial centers and city squares, etc.

From the perspective of the applied industries, it includes the transportation industry, public security industry, education, aviation, digital products, remote sensing image processing, food safety monitoring, and even special military applications. From the perspective of applied products and solutions, the real-time video defogging technology can be applied to the front-end cameras and speed balls to defog and improve the image quality in the field of security monitoring; it can be applied to DVR to improve the image quality; it can be applied to large-screen displays to improve their color saturation and image quality; it can also be used in embedded client software to improve the quality of preview images, etc.