Design of real-time image target search and tracking system based on DSP

　　1. Introduction

　　The TV image tracker is a simple and intelligent image tracking device. It can extract and separate moving targets in the field of view field by field in a relatively complex background according to the standard TV format, extract the target brightness and structural characteristics, and measure the azimuth and pitch angle errors of the target center to the turntable visual axis. The tracking angle error information is transformed and sent to the turntable servo system, which drives the turntable to ensure that the tracked target is always in the center of the imaging sensor 's field of view, realizing automatic tracking of the target.

　　Since the data in the field of view needs to be processed field by field (20ms), the amount of data to be processed is large and the algorithm complexity is high. Traditional processors generally cannot meet the speed requirements. This system uses the TMS320C5416 signal processor of TI Company in the United States as the core, realizing the task of real-time acquisition of image data in the field of view and completing the corresponding image processing algorithm calculation. The main frequency of TMS320C5416 can reach 160MHz, and the total storage space on the chip is 128M×16bit. It is a high-performance, low-power general-purpose digital signal processing chip. The system can process 50 fields/second, with adjustable image resolution standard TV image signals.

　　2. System hardware block diagram

Figure 1 System hardware overall structure diagram

　　As shown in Figure 1, the system adopts a dual DSP + CPLD architecture. The system has two inputs, one for digital video signal input and the other for PAL video signal input from the camera.

The main DSP is responsible for processing the tracking algorithm and communicating with the host computer, and the other slave DSP is responsible for generating simulated Gaussian noise in real time to detect the effect of the tracking algorithm under various noise conditions. The two DSPs communicate through a shared dual-port RAM or HPI. The image data in the field forward process is stored in the image SRAM. The main DSP reads the image data from the image dual-port RAM to the DSP in the field reverse process. When the field forward process begins, the main DSP starts to perform the image processing algorithm. In the next reverse process, the main DSP writes the processing results and related data into the graphic display dual-port RAM and starts to read the next field data from the SRAM. After the DSP processing is completed, the processing results are displayed on the monitor with the cooperation of the timing circuit and the video composite circuit, completing the real-time image processing task.

　　2.1 Image acquisition module

　　The main function of the image acquisition module is to obtain the grayscale data and synchronous clock in the input video signal, which is the benchmark for subsequent processing. The system is designed with synchronous separation and phase-locked technology and uses discrete components. The specific implementation is that the signal is divided into two paths after coming out of the CCD. One path is separated by the synchronous separator LM1881, and the composite synchronization HS and field synchronization VS are output as the control signal of the subsequent circuit. The other path is restored by the bit and DC, and then amplified to adjust the image signal to the reference voltage range of the A/D converter. The pixel clock signal can be obtained by phase-locking and multiplying the horizontal synchronization signal. The phase-locked loop chip uses 74HC4046. The input video signal is phase-locked by the phase-locked loop to output the system pixel clock to provide it to the A/D converter for use to obtain digital image data.

　　2.2 Timing Circuit Module

　　The timing module is mainly implemented by a CPLD (95288XL from Xilinx), which includes phase-locked counting, standard video line and field signal generation, DSP external memory interface chip select read and write signal generation, partial memory address generation, parallel-to-serial conversion of graphic signals, user-defined I/O, etc.

　　2.3 Communication interface module

　　This system uses an asynchronous serial port to receive debugging commands sent by the PC and return the calculation results to the PC.

　　The serial port provided by 5416 is a synchronous serial interface and does not support the universal asynchronous receiver/transmitter (UART) standard. This system uses MAXIM's MAX3100 chip to realize the conversion from synchronous serial port to asynchronous serial port. 5416 uses FSR and FSX as the synchronization signals for each transmission, and FSX as the selection signal of MAX3100. The synchronous receiving clock CLKR and the synchronous transmitting clock CLKX use the internal clock source in this system, and CLKX is used as the synchronous clock of MAX3100. The system uses the receiving interrupt of MAX3100 as the external interrupt signal of DSP to notify DSP that the data is ready and can start receiving.

　　2.4 Memory Access Module

　　The image buffer uses a single-port large-capacity SRAM, which can store the entire field image. The forward process receives the data collected from the acquisition module. The reverse process DSP reads the data to be processed from the SRAM into the on-chip data area for processing.

　　Graphic data dual-port RAM is used to store characters, graphics, etc. for display. A certain position in the field of view corresponds to a bit in a unit (byte) in the memory. The DSP writes the characters or graphics to be displayed in the form of dot matrix into the storage unit specified by the memory, and the memory reads the dot matrix signal of the characters or graphics to be displayed under the system timing control. The 8-bit graphic data is converted into a serial signal by the CPLD parallel-to-serial conversion circuit and superimposed on the analog video signal for display.

　　3. System software design

　　This system can be used as a general platform for digital image processing, handling a variety of image processing programs, and has flexible software design.

　　The main DSP is used to complete the search and tracking algorithm. There are many search and tracking algorithms. The classic correlation tracking algorithm is used as an example to illustrate the general process of image tracking processing software design. If necessary, different tracking algorithms can be added.

　　Correlation tracking is the work of finding the best matching sub-area in an image by using the image similarity measurement method. The optional algorithms include normalized cross correlation (NCC) and mean absolute difference accumulation (MAD). Due to the large amount of data processing in the correlation operation, in order to facilitate real-time implementation, this system uses the correlation matching algorithm of the minimum absolute difference accumulation. This method calculates the similarity measurement between two images f1 and f2 as follows:

　　Among them, f1 and f2 represent the sub-area images of the template and the search area image respectively. After calculating the cumulative sum C of the absolute differences between all sub-areas of the search area image and the template image, the sub-area position corresponding to the minimum C is determined, that is, the best matching point.

　　For a 32×32 template and a 64×64 search area image, the number of calculations for each frame of the image is only to determine the best matching position. In addition to the template correction and decision-making work, the amount of data calculation for each frame is very large. For example, when the instruction cycle is 6ns, the operation time is about 7ms. In order to reduce the amount of calculation, the method of coarse-first and fine-second image resolution can be adopted. When the image resolution is reduced by half, the amount of calculation is reduced by nearly 15 times. In the actual system, the execution time of template coarse sampling matching tracking is about 1ms, which can meet the real-time requirements of the system.

　　Practice has proved that in the process of tracking sequential images, if the image at the best matching position of the current image is simply used as a template to match the next frame of the image, the tracking result is easily affected by the sudden change of the image in a certain frame and deviates from the correct position. Therefore, it is necessary to consider formulating a suitable new template based on the matching degree (suitability) of the best matching position of the old template and the current image, which is equivalent to guiding the matching tracking process to achieve a better tracking effect. This system designs a template weighted correction scheme, namely

　　Among them: A is the original template image content; B is the sub-image content of the best matching position in this matching;

　　M is the modified template image content; W is the weighting coefficient (according to

Intra-frame and inter-frame correlation confidence selection)

　　During the tracking process, due to the complex background, partial occlusion may occur. This system has taken anti-occlusion measures to eliminate the impact of partial occlusion to a certain extent. The basis is that when occlusion occurs, the absolute difference accumulation sum of the best matching position will be much larger than when there is no occlusion. During the matching process, the occlusion evaluation of the matching results is as follows:

　　1. First, determine the occlusion area threshold, that is, the percentage threshold of the occlusion part to the template area. If it exceeds the threshold, it is considered to be occluded;

　　2. Then, the grayscale threshold of the occluded pixel is determined based on the average absolute difference of the pixel at the best matching position. If the grayscale value of the pixel in the template area corresponding to the real-time search area is greater than the grayscale threshold, the pixel is considered to be occluded.

　　3. Calculate the number S of pixels whose absolute difference between the current best matching position image and the template image exceeds the grayscale threshold of the occluded pixels, and then determine whether S exceeds the area threshold of the occlusion. If so, the target is considered to be occluded, the target occlusion flag is set, the initialization matching result stack routine is called, the template correction cycle counter is reset to stop correcting the template, and then the current matching result is replaced with the statistical average of the previous matching results; otherwise, it is considered to be not occluded and the target occlusion flag is cleared.

　　The specific implementation process of the system software is as follows: First, initialize the system. In the initialization subroutine, define a status word SysStatus to control the program flow. For example, when SysStatus=0, it is the search state, when SysStatus=1, it is the correlation tracking algorithm, and when SysStatus=2, it is the contrast tracking state. After the initialization is completed, the system enters a loop waiting state, queries the state of SysStatus in each field, and waits for the state to change. The initial capture of the tracking target template can be completed by the target search algorithm in the search state, or it can be completed manually. Taking manual capture as an example, the user selects the target area in the field of view of the upper PC, and then transmits the target position coordinates and the search area size information to the DSP through the RS232 serial port . The DSP assigns the value to the global variable SysStatus according to the received state. Then enter different tracking states according to the value indicated by the SysStatus parameter.

　　The specific relevant tracking algorithm process is shown in Figure 2 below:

Figure 2 Correlation matching flow chart

　　After a data processing is completed, the result is fed back to the host computer through the serial port, and the window is drawn at the target position through the graphics memory and displayed on the monitor. Then the next processing begins.

　　The real-time Gaussian noise is produced by DSP and superimposed on the collected input image to test the actual effect of different tracking algorithms under Gaussian noise of different intensities.

　　The algorithm generates Gaussian random noise from uniform noise through a mapping table, which has the advantages of high speed and high precision. The principle block diagram is shown in Figure 3: The left half is the uniform noise generation part, which uses the modulus remainder method and relies on the existing k random number seeds x(1),...x(k) that are uniformly distributed in (-x, x) and independent of each other to continuously generate new random numbers and output them. The right half uses the output value of the left part as the offset address, plus the base address of the right table to generate the mapping table address, and outputs the address result obtained after the table lookup to obtain the required Gaussian noise value, and then enters the next cycle.

Figure 3 Noise generation principle diagram

　　The DSP mainly completes the noise generation task. If the system does not need to simulate Gaussian noise, this DSP can also be used for other purposes according to actual needs, increasing the scalability of the system.

　　4. Conclusion

　　This system uses advanced devices with high integration and low power consumption, such as DSP and CPLD, in terms of hardware, which makes the system circuit have the advantages of small size, light weight, low power consumption and high reliability, and is an excellent image processing platform. The target search algorithm based on direction coding and various tracking algorithms such as correlation and contrast have been successfully implemented on this system. Experiments have proved that this system fully meets the real-time requirements and can search and track targets stably, accurately and quickly.

　　References:

　　【1】 Pan Anjun, Zhang Kai, Yan Zhaochun, et al. Application of TMS320 series chips in TV trackers. Systems Engineering and Electronic Technology, 1994.3:pp1～6

　　【2】TMS320VC5416 Digital Signal Processor Data Sheet. Literature number:SPRS095H. Texas Instruments Incorporated (TI), March 1999 – Revised December 2001

　　【3】 Zhang Xiongwei, Cao Tieyong (eds.). Principles and Development and Application of DSP Chips (Second Edition). Beijing: Publishing House of Electronics Industry, 2000.9