Design of real-time image processing architecture based on DSP+FPGA+ASIC

Aguilera

Design of real-time image processing architecture based on DSP+FPGA+ASIC [Copy link]

　   With the rapid development of infrared focal plane array technology, infrared imaging systems have achieved high frame rate, high resolution, high reliability and miniaturization. They have been increasingly used in target tracking and intelligent traffic monitoring, and have been extended to a wider range of military and civilian fields. Real-time infrared image processing systems generally include real-time image processing algorithms at different levels, such as non-uniformity correction, image enhancement, image segmentation, regional feature extraction, target detection and tracking. Due to the large amount of data in image processing and the high correlation of data processing, real-time infrared image processing systems must have strong computing power. At present, some infrared image processing systems use FPGA to implement reconfigurable computing systems [1]. The computing speed is fast, but the implementation of complex algorithms is difficult and the flexibility is poor. Most infrared image processing systems use DSP+FPGA hardware architecture [2]. DSP is responsible for implementing image processing algorithms and FPGA is responsible for implementing various interface circuits. However, processing algorithms such as image multi-level filtering are highly complex. Using only DSP for processing will take a lot of time. Generally, parallel multi-DSP systems are used to meet real-time requirements [3]. However, the system is relatively complex, and the volume and power consumption are difficult to control, and the implementation cost is high. Therefore, it is necessary to reasonably allocate processing tasks in the hardware structure. For example, FPGA can complete the preprocessing of the input image and reduce the burden of DSP. The ASIC chip independently developed by this laboratory can perform multi-level filtering on the input infrared image, further liberating DSP. In view of the real-time requirements of the infrared image processing system, this paper proposes an image processing architecture based on DSP+FPGA+ASIC. 2. Real-time infrared image processing system Since the front-end processing data volume of the real-time infrared small target detection algorithm is large, the real-time performance is strong, and the algorithm complexity is high, our system structure needs to ensure flexibility and be able to adapt to various complex algorithms while ensuring high-speed processing capabilities. Therefore, the real-time infrared image processing system adopts the DSP+FPGA+ASIC architecture to meet the design requirements. Among them, DSP has a high degree of programmability and can complete relatively complex tasks. The processing speed of FPGA is slightly lower than that of ASIC, but it has certain flexibility. It can receive the data of the front-end of the infrared small target detection algorithm in real time and complete some preprocessing of the input image. ASIC chips have fast processing speed, strong performance and good reliability, and can complete specific tasks such as image filtering. The FPGA uses the EP1S10 chip of the Stratix series of ALTERA. The chip provides up to 10,570 logic units (LEs) and a memory capacity of 920,448016 bits. It also provides high-speed DSP modules and multipliers. The dedicated multipliers are suitable for coprocessors, which can share the complex and time-consuming arithmetic operations of the DSP and improve the performance of the system. The DSP uses the TMS320C6414 chip of TI [4]. The chip has a main frequency of 600MHz and a peak data processing capacity of 4800MIPS (4.8 billion instructions per second). It consists of 8 independent functional units. It contains two groups of general registers, each group contains 32 32-bit registers. The chip uses a 2-level memory structure. The ASIC uses a multi-stage filter chip independently developed by our laboratory [5]. The chip uses the SMIC 0.35mm process and the internal operating frequency of the chip can reach 50MHz. There are three data channels, which are cascaded with different numbers of 1*3 basic filter templates. Each data channel adopts a pipeline structure. Fixed-point operation is adopted, and the calculation accuracy is 8-bit binary decimals. It can process data with a bit width of 8-16 bits, and the throughput is 5M-10M pixels/second. It supports image filtering in three frame formats: 128*128, 256*256, and 320*240. 3. Reasonable allocation of processing tasks The infrared detector outputs image data to the signal processing board, and first enters the FPGA for preprocessing. The preprocessing includes non-uniformity correction and image enhancement. ASIC performs multi-level filtering on the input image data. FPGA can also work with DSP in subsequent algorithms. The algorithm has high requirements for running speed, and the processing of the algorithm structure is not too complicated can be implemented in FPGA, making the FPGA a coprocessor of DSP. The infrared image data after preprocessing is output to DSP for subsequent algorithm processing. The processing of DSP includes infrared potential small target detection algorithm. The potential small target and image data after DSP processing are transferred to DPRAM for storage through FPGA. The real-time infrared image processing system mainly consists of three parts: FPGA module, DSP module and ASIC module. Under the premise of meeting the real-time and high-speed reliability requirements of infrared image data processing, the tasks processed by each processing module should be balanced as much as possible. From the perspective of resources, the algorithm is fast in hardware processing speed and occupies less memory space, but has poor flexibility; the software is just the opposite, with slow processing speed but good flexibility. Since the design must meet the real-time requirements, it is hoped that the hardware can undertake as many tasks as possible. As a programmable hardware resource, PGA has considerable design flexibility, but considering the difficulty of design, implementation and debugging, it is not suitable for global and complex operations. ASIC chips have fast processing speed, but poor flexibility, and are only suitable for processing some specific operations that occupy a large amount of processor resources. As a software resource, DSP is characterized by good flexibility and is suitable for complex algorithms, but slow speed. From the algorithmic perspective, a complete algorithm has layers in processing. The algorithm's operation on raw data is simple and regular, while the processing of subsequent data is often more complex. At the same time, the algorithm will be pre-divided into several independent functional modules before implementation. Therefore, low-level, simple operations can be completed by the logic unit module in the FPGA or the ASIC chip hardware, while high-level, complex operations are completed by the DSP software. Based on the above analysis, in this design, the FPGA mainly completes the interface control logic with various external devices, and also completes some algorithms with large data volume, simple operation structure and high speed requirements; the DSP completes the subsequent target detection algorithm with complex structure; and the ASIC chip performs multi-level filtering on the input image. This solution ensures that the system has sufficient processing speed and can meet the real-time requirements of the infrared image processing system. 3.1 FPGA module FPGA mainly completes the following tasks: image data receiving logic, image data preprocessing module, input buffer FIFO, DSP interface control logic, ASIC interface control logic, DPRAM interface control logic, algorithm processing module. (1) Image data receiving logic: according to the field synchronization, line synchronization and clock signal of the digital video signal output by the detector, send control signals to receive image data. (2) Image data preprocessing module: perform preprocessing such as non-uniformity correction and image enhancement on the received image data. (3) Input buffer FIFO: The input buffer memory FIFO is configured by the internal FPGA. The preprocessed image data can be continuously and uninterruptedly written into the FIFO for the DSP to read for subsequent image processing. (4) DSP interface control logic: The DSP interface timing control logic must be written in strict accordance with the timing of the EMIFA programmable synchronization interface of TMS320C6414 and meet the corresponding setup and hold time requirements. The DSP starts the EDMA channel to read the infrared image data and continuously transmit the image through the internal FIFO of the FPGA. (5) ASIC interface control logic: Write its control logic according to the timing of the ASIC chip and import the image data into the ASIC chip for multi-level image filtering. (6) DPRAM interface control logic: Write the DPRAM control logic and transfer the data processed by the DSP to the DPRAM through the FPGA for storage. (7) Algorithm processing module: According to the needs of the specific algorithm, cooperate with the DSP to complete the calculations in the algorithm with high speed requirements and not too complex structure. 3.2 DSP Module DSP mainly completes the detection and tracking algorithm of infrared small targets, and its functions are as follows: (1) Start the EDMA channel to continuously read the infrared image data output by FPGA. (2) Run the target detection and tracking algorithm on the input infrared image data, detect the moving target, and determine its position. (3) Establish a connection with FPGA and hand over the more regular operations in the algorithm to FPGA for processing. (4) DSP transmits the processed operation results to DPRAM through FPGA for storage. 3.3 ASIC Module The ASIC chip receives the pre-processed image data from FPGA for processing. The processed three-way data is synchronously written into the three internal synchronous FIFOs. DSP can choose to output the results after multi-stage filtering as needed for subsequent algorithm processing. 4. Experimental results and analysis The real-time infrared image processing board has been successfully applied to multiple infrared detection systems with good results. Figure 3 shows the original infrared image obtained by the AutoNavi IR108D detector during the experiment, Figure 4 shows the image marked after multi-stage filtering, and Figure 5 shows the tracking image output after the target detection algorithm is processed. It can be seen that the system can detect and track small moving targets in infrared images in real time. 5. Using high-performance DSP (TMS320C6414), programmable logic device FPGA (Stratix series EP1S10) and dedicated ASIC multi-stage filtering chip, a DSP + FPGA + ASIC image processing platform architecture is proposed, and an infrared image processing system with strong processing capability and reliable and stable interface is designed. In the system, algorithms such as non-uniformity correction and small target detection are implemented. Experimental tests show that the real-time infrared image processing system can process the output digital video signal with 320×240 effective pixels per field, 14 bits per pixel, and a field frequency of 50Hz in real time, complete the detection and tracking functions of small moving targets in the field of view, meet the main performance index requirements of the system, and be successfully applied in the infrared detection system.