Design of a general image processing platform based on DSP and FPGA

1 Introduction

With the continuous development and improvement of digital technology, digital image processing technology has been widely used in industry, military, biomedicine, telecommunications and other fields. In practical applications, the ability to run complex and flexible image processing algorithms and large amounts of data transmission and processing capabilities has become a prerequisite for the stable operation of the image processing platform, and factors such as system real-time performance, volume, and power consumption are also crucial. Traditional digital image processing platforms are mostly implemented using general-purpose PCs, high-speed image acquisition cards, and VC++-based software platforms, but it is difficult to meet the current requirements for system volume, power consumption, and real-time performance. Therefore, the general-purpose image processing platform based on DSP and FPGA proposed here fully utilizes the advantages of FPGA's strong flexibility and DSP's fast computing speed and flexible addressing mode, better improves the integration of image processing systems, reduces system power consumption, and meets real-time requirements.

2 Working principle of high-speed image processing platform

To achieve real-time processing of high-speed images, the system uses a DSP and FPGA linear pipeline array structure, combining the advantages of FPGA in general interface design and simple signal processing with the fast digital signal processing capability of DSP, giving full play to the advantages of both. The system is mainly composed of DSP and FPGA subsystems. To ensure fast and stable communication of large image data streams, large-capacity data exchange is achieved between DSP and FPGA through external SDRAM. The DSP subsystem implements more complex image processing algorithms and provides image storage functions. The FPGA subsystem completes the preprocessing of CCD sensor image data and the general interface function of the microcontroller.

The schematic diagram of the system structure is shown in Figure 1. After the image data input by the CCD sensor is pre-processed by the FPGA, the data is transmitted to the DSP. The DSP performs real-time image processing on the input data, and sends the processed image through the EMIF interface and saves it to the external SDRAM. Similarly, the FPGA can also read the image data of the external SDRAM and display it in real time through the VGA interface. For a small amount of data streams, such as system parameters or the start and end information of image data transmission, it is implemented through the SPI interface. The DSP subsystem is internally expanded with an SD card interface and a USB host interface, which are mainly used for the storage and transmission of image data. The main interfaces of the FPGA subsystem include: I2C, SPI, UART, PS/2 and VGA interfaces, which are used for system upgrades and debugging to improve the versatility of the system.

3 System Hardware Structure Design

The system hardware design adopts modular design concept, and the whole system is divided into DSP subsystem and FPGA subsystem. The data exchange between the two is realized by dual-port RAM.

3.1 Device Selection

The system design uses TI's TMS320VC5509A DSP. This DSP has low power consumption, rich on-chip resources, and a main frequency of up to 200 MHz. It has 128 K×16 bit RAM and 32 K×16 bit ROM on the chip, 6 built-in DMA channels, 1 I2C interface, 3 McBSP interfaces, and 1 RTC module. Its external memory interface (EMIF) can be seamlessly connected to SDRAM, and it also has a USB interface. FPAG uses ALTERA's Cyclone-II series processors, which have powerful logic processing capabilities, thereby realizing the general interface design of microprocessors and simple information preprocessing functions.

3.2 Interface design between DSP and FPGA

To ensure the real-time performance of the system, the interface between DSP and FPGA needs to realize the function of smooth communication of large data streams. The internal result buffer of FPGA is simulated as an SDRAM interface, with CCD image signal input at one end and image data output at the other end and connected to the DSP data line. The EMIF interface of DSP is connected to a 4 M×16 bit SDRAM MT48LC4M16A2-75. By transmitting the processed image data back to the external SDRAM, the FPGA reads it in real time and displays it through the VGA interface, thus realizing the data communication function between DSP and FPGA. The dual-port RAM connection configured between the two is shown in Figure 2.

3.3 DSP Subsystem

The DSP subsystem mainly includes power management unit, EMIF interface, SD card interface, USB interface, JTAG debugging interface and boot loader circuit, etc. The power management unit mainly provides stable power for the system; the EMIF interface is mainly used for external expansion of memory; the SD card interface is used to store image data after power failure; the USB interface is used to connect other peripherals; the JTAG interface is used for circuit debugging, etc.

3.3.1 Power Management Unit

The DSP subsystem power supply can be divided into 1.6 V and 3.3 V. The DSP core needs 1.6 V power supply, and the peripherals and I/O ports use 3.3 V power supply. It is necessary to ensure that the core is powered on before the I/O, and the I/O is powered off before the core. The system uses the power supply device TPS767D301 to configure different voltage values. The device includes two voltage outputs, and the maximum output current of each channel can reach 1 A. The output voltage is stable. Figure 3 is the power management unit circuit.

3.3.2 External Memory Interface

The EMIF interface integrated inside the TMS320VC5509A supports not only asynchronous memory, but also synchronous burst static memory (SBSRAM) and synchronous dynamic memory (SDRAM). Here, the connection between EMIF and SDRAM is configured by programming registers. Setting MTYPE=011b of CE space control register 1 indicates that the connected memory is SDRAM. Figure 4 shows the connection circuit of the configured 4 M×16 bit SDRAM MT48LC4M16A2-75. Since the limit of a single CE space is 4 MB, two CE spaces are used, and the CEO pin is used as the chip select, and the CE1 pin is left floating. The external SDRAM is mainly used to store processed image data.

[page]

3.3.3 SD card interface

The built-in MMC controller of TMS320VC5509A supports reading and writing of MMC cards and SD cards, and supports MMC/SD protocol and SPI protocol. The operating frequency of the MMC controller can be set by program and multiplexed with McBSP interface pins. When using it, the external bus select register (EBSR) needs to be set. Figure 5 shows the signal connection diagram of the MMC controller and the SD card. The connection signals are: clock signal (CLK), control signal (CMD) and data signal (DAT0~DAT3).

3.3.4 Bootload Circuit

The function of Bootload is to load the user program from the slow memory outside the chip to the on-chip RAM after the system is powered on, and make it run at high speed. Here, EEPROM is selected as the external non-volatile program memory. The Bootload mode of TMS320VC5509A supports EMIF mode, SPI mode and McBSP mode, etc. There are two types of EEPROM bootstrap in SPI mode, one is based on 16-bit byte address, with a maximum addressing space of 64 K; the other is based on 24-bit byte address, with a maximum addressing space of 16 M. The first method is selected here, and the Bootload mode selection pin BOOTM[3:0] is brought out to facilitate system upgrades.

3.4 FPGA Subsystem Composition and Functions

In order to realize the universality and real-time performance of the image processing platform, the functions that the FPGA subsystem needs to realize include: open image data acquisition bus, DSP image processing real-time data bus, 100 MB Ethernet interface, UART interface, VGA real-time display module, I2C memory interface and PS/2 interface. Among them, the UART interface is convenient for system software development and debugging, the VGA interface is used for real-time display of image data, and the I2C interface is connected to an external EEPROM for power-off storage of system parameters. In order to realize the networking of multiple systems, the FPGA subsystem is also equipped with an Ethernet interface for multiple systems to transmit processing results back to the PC port. The PS/2 interface is a reserved port, and input devices such as keyboards will be added later as needed.

4 System Software Design

4.1 Software Development

The system software design uses TI's CCS3.1 (Code Composer Studio) as the development environment, and uses the DSP/BIOS real-time operating system that comes with CCS for design. Software editing, compilation, debugging, code performance testing, and project management are completed in CCS. By using a series of rich kernel services provided by DSP/BIOS, sophisticated and complex multi-task applications that meet real-time performance requirements can be quickly created. The DSP/BIOS kernel has a cross-platform standard API interface that can be called by user programs and is easy to port. In addition, in addition to supporting multi-threaded scheduling management, these services also support system real-time analysis and data management. The DSP/BIOS kernel has great size scalability. The minimum code size of the kernel image under multi-threaded configuration is only 1 K words, which occupies very little DSP resources.

4.2 Overall design of software system

Based on the hardware platform, the DSP/BIOS real-time operating system kernel in the CCS integrated development environment is used to develop a scalable software system. The system software part adopts modular and hierarchical design ideas. The software structure mainly includes: device driver layer, operating system layer, application program interface (API) layer and application layer. The device driver layer is responsible for the driver design of each module or peripheral related to the hardware; the operating system layer is responsible for the transplantation of the embedded real-time operating system; the application program interface layer completes the system control function, data reading and writing, etc., and realizes hardware independence; the application layer designs the control program related to the system application background. Figure 6 shows the system software operation flow. The software design is mainly divided into the receiving task after CCD image preprocessing, the fast digital image processing task, the logic control task and the image data return task. After the system is powered on, the program first executes the initialization of DSP and DSP/BIOS, then executes the function body and starts the DSP/BIOS operating system. The subsequent tasks are scheduled by the operating system. The image processing software platform constructed with the embedded real-time operating system DSP/BIOS can better meet the real-time requirements of the task, and the structure is stable and compact, and the portability is high.

5 Experimental Results

In order to verify the universality and real-time performance of the system, it is applied to the device detection of a certain type of placement machine, and the following three experiments are carried out: large-capacity data transmission experiment under DMA mode, threshold segmentation test experiment and template matching test experiment. Among them, the large-capacity data transmission experiment under DMA mode transfers the on-chip data to the off-chip SDRAM through DMA mode, and the image size is 600×480 bytes. The threshold segmentation and template matching experiments directly read the image data in the off-chip SDRAM, and perform threshold segmentation and 8×8 template matching experiments on the image respectively. The image processing algorithm uses the image processing library of TI company. IMG threshold() and IMG_mad_8×8(). Set the DSP running clock to 208 MHz, use timer 0 for timing, select channel 0 as the DMA channel, and the image data size is 600×480 bytes. The experimental results are shown in Table 1.

The above experimental data show that when using 600×480 area array CCD data acquisition and requiring each frame image processing time to be limited to within 30 ms, the system can well meet the current system needs.

6 Conclusion

The system can meet the needs of 600×480 area array CCD and ordinary linear array CCD sensors for system processing capabilities, and has strong throughput and real-time performance. Its design innovation lies in that it fully utilizes the powerful computing power and flexible addressing mode of DSP, combines the advantages of FPGA in general interface design and simple signal processing speed, and adopts a software architecture based on DSP/BIOS, making the system highly integrated, low power consumption, and with higher real-time performance and portability.