Design of video processing system based on FPGA+DSP architecture

In real-time video image processing, the low-level preprocessing algorithm processes a large amount of data and has high requirements for processing speed, but the algorithm is relatively simple and suitable for hardware implementation using FPGA, which can take into account both speed and flexibility. The high-level processing algorithm has a complex structure and is suitable for implementation on DSP chips with high computing speed, flexible addressing mode, and strong communication mechanism.

The biggest feature of DSP + FPGA architecture is its flexible structure, strong versatility, and suitability for modular design, which can improve algorithm efficiency. At the same time, its development cycle is short, the system is easy to maintain and upgrade, and it is suitable for real-time video image processing.

The system adopts a modular design approach and divides the entire system into three parts: video acquisition unit, video processing unit and video transmission unit.

The whole system uses FPGA as the core control unit and completes the median filtering of the video signal; DSP is used as the core processing unit of the whole system to perform JPEG compression on the collected video image information; a USB bus based on the PDIUSBD12 chip is designed in the video transmission unit to be responsible for the transmission of the video signal.

1 System Hardware Overall Architecture

A complete video processing system mainly consists of three parts: video acquisition unit, video processing unit and video transmission unit. When designing the system, it is necessary to ensure the seamless connection of each part.

The image acquisition unit is composed of FPGA and MB86S02 video acquisition chip, including the acquisition and preprocessing of video signals, converting the input video signals into digital image data that can be processed by the system, and storing them in a certain storage area according to a certain format.

The image processing unit is the core of this system, which compresses the image data to achieve the functions that the system needs to achieve.

The image transmission unit uses FPGA+USB to realize the transmission of video data. Through the USB bus based on the PDIUSBD12 chip, the compressed video image information is sent to the receiving end. The receiving end uses the application written on the PC to decompress and display the image.

The entire hardware system consists of two subsystems, FPGA and DSP. FPGA, as a video acquisition unit, pre-processes the acquired video signal and transmits it to DSP. DSP, as an image processing unit, is the core of this system and performs JPEG compression on the video image information pre-processed by FPGA. The performance of the DSP unit determines the performance of the entire system. After completing the image processing task, DSP returns the result to FPGA. FPGA writes the compressed image information into the data buffer of the interface control chip, which is responsible for information transmission. The overall block diagram of the system is shown in Figure 1.

Figure 1 System overall structure diagram

As shown in Figure 1, the MB86S02 video image sensor collects video image information under the control of the FPGA. After receiving the acquisition command from the PC, the MB86S02 starts to collect video signals. As the core control unit of the system, the FPGA is not only responsible for the acquisition of video images, but also for the preprocessing of video image information and the data interaction between the various unit modules of the system. In view of the large amount of video image data and in order to ensure the real-time requirements of the system, the system uses a large-capacity off-chip SDRAMR to cache the collected video image information. The SDRAM controller is implemented by the FPGA. After the video image information is cached by the SDRAM, it must first be filtered by the FPGA to eliminate the noise interference in the image information. In this system, the median filter is used to process the collected video information. The filtered data enters the DSP through the internal FIFO of the FPGA for the next step of compression processing. After the DSP is powered on, it first loads the boot program and waits for the FPGA to send a request. After receiving the request from the FPGA, the DSP establishes an EDMA channel to obtain video data from the FPGA. After storing a full frame, it starts to perform JPEG compression on the video image. After the compressed video image information passes through the FIFO cache, it is written into the data cache of the USB interface controller under the control of the FPGA and waits for the reading request from the PC. After receiving the reading request from the PC, the USB interface controller writes the data to port 1 of PDIUSBD12 so that the PC can read the data in the next step. 2 Overall design of system software

The software design of the system can also be divided into two parts according to the overall division of the hardware structure. The operation of the entire system is shown in Figure 2. The programs of FPGA and DSP run independently, and the real-time data interaction is completed through interrupt signals. The instruction from FPGA to DSP is to send an EDMA request through FPGA. DSP responds to the EDMA request, establishes an EDMA channel, and starts reading the pre-processed data from FIFO. When DSP transmits data to FPGA, it sends an interrupt signal to FPGA to let it read the compressed image data from FIFO.

Figure 2 System software flow chart

As shown in Figure 2, the entire system workflow can be briefly described as follows: After the system is powered on, the DSP first boots from the flash and runs the boot program, then enters the EDMA waiting state. After the FPGA is initialized, it waits for the external image acquisition command. After receiving the image acquisition command, it starts to acquire the image and pre-processes the acquired image. The pre-processed image is buffered by the FIFO. After storing a certain amount of data, the FPGA sends an EDMA request to the DSP through a half-full signal and waits for the DSP to respond. Once the DSP receives the EDMA request from the FPGA, it immediately establishes an EDMA channel and reads data from the FIFO to the L2 memory. After storing a frame of image, the DSP starts image compression. After waiting for an image to be compressed, the DSP will send an interrupt signal to the FPGA. After receiving the interrupt signal, the FPGA starts to read the compressed image data from the FIFO. After reading a frame of data, it determines whether the encoding signal is valid. If it is valid, the next frame of image is compressed according to the same rule. If it is invalid, the DSP is notified to end.

3 Conclusion

This design has been hardware verified, has met the predetermined design requirements, and has achieved real-time processing of large amounts of data.

The system is only 70×70mm in size, with power consumption less than 5W, median filter rate 20F/S on average, and JPEG compression rate over 25F/s on average. It not only meets the real-time requirements of the video processing system, but also has small size, low power consumption, and based on FPGA programmability, this system has good flexibility and scalability.