Design of motion vision processing system based on SOPC

1 Introduction

With the development of deep submicron technology, the capacity and density of FPGA are increasing. With its powerful parallel multiplication and addition (MAC) capability and flexible dynamic reconfigurability, it is widely used in many fields such as communication and imaging. However, in the implementation of complex algorithms, FPGA is not as convenient as embedded processors. Therefore, when designing systems with complex algorithms and control logic, it is often necessary to use it in combination with embedded processors. This is the SOPC (System on a Programmable chip) technology. SoPC is the product of the combination of SoC and FPGA. A single programmable and reconfigurable chip completes the main functions of the entire system. SoPC is flexible and can be designed using schematics, hardware description languages, and even C/C++ high-level languages. It also has the characteristics of being reconfigurable, scalable, and expandable, and is easy to upgrade. SoPC combines the advantages of both SoC and FPGA, and has the following characteristics:

●Rich IP core resources, including general IP cores and dedicated IP cores

●In-system programmable, simple and convenient design, compilation, download and debugging

●Built-in embedded soft-core processor, memory, peripheral interface controller

●A large number of programmable reconfigurable logic resources

●Powerful clock management circuit

●Support multiple I/O interface standards

In addition, due to the reduction of the transmission line distance between the processor and the memory, SoPC has obvious advantages over the board-level system in terms of speed, signal integrity, electromagnetic compatibility, etc.

2 Functional Block Diagram

Based on the development of the digital video vision acquisition and processing board-level system, this paper uses SoPC to implement the design of motion vision processing and control system. The hardware uses Altera's StratixII series FPGA, and the software development tools include QuartusII, NiosII5.1 IDE, DSP Builder, MegaCore IP Library5.1 and Matlab7.0. The working principle of the entire SoPC system is as follows: the video image data collected by the camera comes in through the image sensor interface; the digital signal processing block and the digital image and digital video processing IP core complete the processing of the video image, such as motion detection, segmentation, feature extraction, compression, etc.; the Nios II embedded processor mainly completes the control function of the entire system; the I2C bus is used to access the internal registers of the CMOS image sensor; the PC can access the SoPC through the USB interface, and if the distance is far, data can be transmitted through the Ethernet port; the video image and its processed results can be stored in CF cards, FLASH and other storage media through the external memory interface, and can also be directly displayed on the LCD screen through the LCD display interface. The functional block diagram of the motion vision SoPC is shown in Figure 1:

This module is mainly responsible for video image acquisition, camera parameter setting and pan-tilt stepper motor control. Among them, the internal control register of the camera can be set through the I2C bus. In FPGA, there are two ways to implement the I2C bus controller: one is to use software simulation in Nios II; the other is to use third-party IP cores provided by companies such as Sciworx, CAST, Digital Core Design, etc. These IP cores have working parameters and can be set as needed. Taking the I2C bus controller provided by CAST as an example, the maximum transmission rate is 100Kbps, and it can work in 4 modes, namely master transmitter mode, master receiver mode, slave transmitter mode and slave receiver mode. The dual-channel CMOS image sensor interface controls the timing, frame synchronization and line synchronization of video image acquisition; the pan-tilt control signal controls the rotation of the two stepper motors of the pan-tilt according to the results of video image processing. These two modules need to be developed by ourselves to form IP cores with independent intellectual property rights.

3 Video Image Processing Module

This module includes NiosII 32-bit embedded processor, digital signal processing block, digital image and digital video processing IP MegaCore, and some other logic circuits. It is the focus and core of the motion vision processing and control SoPC design, which is introduced in the following parts.

(1) Nios II embedded processor

Nios II embedded processor is a general-purpose RISC structure CPU, which is positioned in a wide range of embedded applications. In the Nios II IDE integrated development environment, follow the operation prompts to add and set relevant parameters, and a Nios II embedded processor can be generated within a few minutes. The hardware development process is as follows:

① Analyze the functions and performance to be achieved by the system

② Start SOPC Builder and select the specific FPGA model

③ Define modules such as CPU, peripheral devices, storage system, etc.

④ Assign base addresses and interrupt request numbers (IRQ) to each module

⑤ Generate Nios system modules, lock pins, and compile. The software development process is as follows:

① Start Nios II IDE in SOPC Builder

② Create a C/C++ software project and specify the target hardware

③ Use the project template to write the corresponding program

④ After compilation, it can be downloaded to the hardware for running.

Programs can be written in C/C++ or assembly language in Nios II IDE, and their file extensions are .c and .s respectively. A single Nios II/f CPU takes up about 1800 LEs. If some timers, peripheral devices, etc. are added, the occupied logic units will increase further.

(2) Digital signal processing block

Stratix II series FPGA has digital signal processing blocks (DSP Blocks) inside. The digital signal processing block can support multipliers with different data widths (9×9, 18×18, 36×36) and operation modes (multiplication, complex multiplication, multiply-add, and multiply-accumulate), and each DSP block provides 2.8 GMACS of DSP data throughput. The largest Stratix II device EP2S180 contains 96 digital signal processing blocks, which can provide 284 GMACS of throughput and support 384 18×18 multipliers. In addition, the digital signal processing block has added new rounding and saturation support to facilitate the import of DSP firmware code into FPGA. Some applications, such as voice processing, can use rounding and saturation because the storage buffer for storing data is of fixed width. Now that the digital signal processing block that supports rounding and saturation is used, it is very convenient to import the design based on the DSP processor into the FPGA for implementation.

To design a DSP system on Altera's programmable devices, it is necessary to have development tools that support both advanced algorithms and hardware description languages. MathWorks' MATLAB and Simulink system-level design tools have the ability to develop, simulate, and verify algorithms. Altera's DSP Builder combines these tools with Altera's development tools to provide a DSP development platform that shares system design, algorithm design, and hardware design.

(3) Video image processing

IP cores Third parties provide many customizable IP cores for communication, image encoding and decoding, and video processing. Reasonable use of these IP cores can greatly shorten development time while ensuring performance and reliability. The following is an introduction to color space conversion IP.

CSC (Color Space Convertorr) is an IP core in the MegaCore IP library file provided by Altera that is specifically used for image color space conversion. Compared with software conversion, it has obvious speed advantages and flexibility:

● Complete the conversion of one pixel per clock cycle

● In Stratix series FPGAs, the clock frequency is greater than 200MHz

● Support the interchange between RGB and YCbCr, YUV

● Users can customize the correlation coefficients of the conversion matrix

● Support signed and unsigned numbers

The data width of input and output is 2 to 32b

4 RAM data buffer

Stratix II series FPGAs contain up to 9Mb of on-chip RAM. These RAMs use a TriMatrix storage structure, including three sizes of embedded memory blocks: 512b M512 blocks, 4Kb M4K blocks, and 512Kb M-RAM blocks. Each can be configured to support various features, such as single-port RAM, dual-port RAM, FIFO, etc., providing solutions for large storage applications.

5 External Memory and Peripheral Interfaces

Stratix II series FPGAs are optimized for reliable data transmission of external memory and support the latest memory interface to access off-chip memory. Developers can quickly and easily integrate various large-capacity memory devices into complex system designs using Stratix II's advanced device features and customizable IP cores. Stratix II supports a variety of latest memory interfaces. The on-chip processors and peripherals of Stratix II series FPGAs are connected via the Avalon switching bus. The Avalon switching bus is a dedicated internal connection technology developed by Altera that uses minimal logic resources to support data bus multiplexing, address decoding, wait cycle generation, peripheral address alignment, interrupt priority assignment, etc. The IP cores that can be customized for the peripheral interface include controllers such as USB, I2C, Ethernet, and PCI. Most of these IP cores are provided by third parties and can be tried for free or purchased at a partial cost. The USB2.0 controller and Ethernet interface controller used in this system are both provided by Mentor.

6 Clock Management Circuit

The Stratix II series FPGA has up to 48 high-performance, low-skew global clocks, which can be used for high-performance functions or global control signals; up to 12 programmable phase-locked loops (PLLs), with complete clock management and frequency synthesis capabilities, including clock switching, PLL reconfiguration, spread spectrum clocks, frequency synthesis, programmable phase offset, programmable delay offset, external feedback, and programmable bandwidth. Stratix II has two types of general PLLs: enhanced PLL and fast PLL. The enhanced PLL is rich in functions and supports external feedback, spread spectrum clocks, programmable bandwidth, etc. The fast PLL is optimized for high-speed differential I/O interfaces and has a dynamic phase adjustment (DPA) function. These high-speed clock networks and rich PLLs combine to provide a strong guarantee for the system to work with minimal clock offset.

7 Other components of SoPC

FPGA configuration interface is used for configuration, compilation and online debugging of SoPC; LCD display interface can be connected to an external LCD display; alarm signal is a sound or photoelectric signal emitted when a moving target is detected and identified, which can be used for security; standard I/O port is reserved for future upgrade and expansion.

Innovation: Before the concept of SoPC was proposed, the integrated design of electronic systems was mainly based on board level. With the continuous increase of system clock frequency and the increasing complexity of circuit functions, this design method has become more and more difficult to implement, and electromagnetic interference and signal integrity problems have become increasingly prominent. Only by optimizing the layout and wiring of PCB can the transmission and processing requirements of high-speed signals be met. With the continuous development of the semiconductor industry, programmable chip systems will become the mainstream of future electronic product development and design with their high performance, reliability, low power consumption, cost and good portability. The motion vision SoPC better solves a series of problems of board-level circuits and can be widely used in many fields such as security monitoring, visual navigation, intelligent transportation, etc., and will definitely have a good market prospect.