Information perception system integrating infrared, low light and visible light

The information perception system based on the sensor network is an important detection equipment. The current information monitoring and early warning system usually uses one detector from CCD, micro -optical vision and thermal imager to work alone or 2-3 detectors to switch between them. There are various detection blind spots and their respective advantages cannot be brought into play. Micro-light television and CCD are easily affected by weather and have poor ability to penetrate smoke. Thermal imagers have the advantages of strong ability to penetrate smoke and dust and can work all day, but their resolution is low and the images they produce do not conform to the visual habits of the human eye. Xiangrong Group Co., Ltd. and Nanjing University of Science and Technology have developed an information perception system that integrates infrared, micro-light and visible light through multi- spectral image fusion technology for these three complementary detection devices. This article will introduce its main functions.

1 IC CD and infrared image fusion

Using the multispectral image fusion system, the images of the same scene were collected and grayscale fused in a low illumination environment, as shown in Figure 1. Among them, a) is the ICCD image, b) is the infrared image, and c) is the grayscale fused image. It can be seen that the infrared image can accurately detect thermal targets, while the ICCD image can provide richer detail information, but it is difficult to detect targets that cannot be distinguished from the scenery. After image fusion processing, in the visible light and 3-5 μm bands, the fused image not only accurately reflects the target information, but also better reflects the details.

Figure 1 Grayscale fusion image of ICCD and infrared

The images of the same scene were collected and color fused in a low-light environment, as shown in Figure 2. a) is the ICCD image, b) is the infrared image, and c) is the color fused image. It can be seen that in the visible light and 3-5 μm bands, the color fused image not only retains the grayscale fusion features, but also better conforms to the visual characteristics of the human eye in the process of target detection, tracking, and recognition, which helps to improve the probability of target detection and recognition.

Figure 2 ICCD and infrared color fusion image

2 CCD and infrared image fusion

In addition to ICCD and infrared fusion functions, the multispectral image fusion system also has another notable feature: CCD and infrared fusion. Under the same field of view and high illumination environment, the image acquisition and fusion of the Sun Yat-sen Mausoleum scenic area were carried out. Figure 3 shows the situation of vehicles and pedestrians on the road in the forest. In the fused image, the spectral characteristics and complementarity of the two sensors are highlighted in the visible light and 3-5 μm bands.

Figure 3 Fusion of CCD and infrared images of the same field of view

Figure 4 shows the observation of distant buildings. The buildings that were not observed in the CCD detection system are clearly visible in the fused image because of the observation effect of the 3-5 μm band thermal imager.

Figure 4 Infrared and visible light fusion images of the same field of view

In general tracking systems, a large field of view search and a small field of view tracking method are usually used, and the target is observed by switching in the display system. Using image fusion technology, a search and tracking fusion system with different large and small fields of view can be developed. Figure 5 shows the infrared and visible light fusion images of different fields of view. For the same target, in two different spectral ranges, the thermal radiation radiation characteristics and spectral reflection radiation characteristics of the target can be more objectively evaluated.

Figure 5 Infrared and visible light fusion images of different fields of view

Whether for military or civilian use, if CCD and thermal imagers are installed in the information platform, image fusion technology can be used to fuse information in the visible light band, 3 to 5 μm or 8 to 14 μm, thereby fully improving the perception performance of the target to be discovered.

3 CCD and radar image fusion

CCD and radar image fusion, foreign countries have reported that the fusion of synthetic aperture radar and video images has a large number of applications in the military and civilian fields. Using the currently developed information perception system that integrates infrared, low-light and visible light, we can explore the fusion of radar images such as collision avoidance imaging radar and damage assessment radar with infrared, low-light and visible light images.

Figure 6 is a damage assessment radar image of a certain airport obtained by my country at an altitude of 4000 meters. Except for the airport runway, there are some problems in the target recognition of the image. If it is used for damage assessment in military terms, it is still unable to identify the actual situation of the runway.

Figure 6 Radar image of an airport at 4000m

Figure 7 is a satellite image (CCD) of the airport downloaded from the website, which is the result of multiple stitching. The image can be enlarged on the website to obtain a clear visual effect.

Figure 7 Satellite image CCD obtained from the website at the same viewing distance

The two images were fused, and the effect is shown in Figure 8. Comparing Figure 8 with Figure 6, targets that cannot be identified in the radar image can be annotated by comparing them with the satellite image, which increases the understanding of the radar image. If used as a damage assessment radar, it can use the information on the current website to pre-establish a large database for the area of ​​interest. In wartime, it can be fused with the radar image to more accurately identify the degree of damage caused by the war to the area.

Figure 8 CCD fusion of radar and satellite images at the same viewing distance

Regarding the detection application of real-time fusion of radar and video images, if the platform is equipped with CCD, thermal imager and radar system, the fusion of CCD, thermal imager and radar images can be used during the day to form a full-spectrum fusion observation of suspicious targets in the visible band, 3-5μm or 8-14μm and radar band. At night, the fusion of ICCD, thermal imager and radar images can also be used to conduct real-time fusion observation of suspicious targets in the full spectrum band, which can improve the detection and identification probability of the target. If vehicles traveling on the highway are equipped with synthetic aperture radar and thermal imager, the use of image fusion technology can greatly reduce vehicle rear-end collision accidents.