Design and implementation of motion vision processing system based on SOPC

introduction

With the development of deep submicron technology, the capacity and density of FPGAs continue to increase. With its powerful parallel multiplication and addition (MAC) capabilities and flexible dynamic reconfigurability, FPGAs are widely used in many fields such as communications and images. However, in the implementation of complex algorithms, FPGAs are not as convenient as embedded processors. Therefore, when designing systems with complex algorithms and control logic, they often need to be used in combination with embedded processors. This is the SOPC (System on a Programmable chip) technology. SoPC is the product of the combination of SoC and FPGA. A single programmable and reconfigurable chip completes the main functions of the entire system. SoPC is flexible in design and can be designed using schematics, hardware description languages, and even C/C++ high-level languages. It also has the characteristics of being reconfigurable, scalable, and expandable, and is easy to upgrade. SoPC combines the advantages of both SoC and FPGA and has the following characteristics:

●Rich IP core resources, including general IP cores and dedicated IP cores

● In-system programmable, easy to design, compile, download and debug

●Built-in embedded soft-core processor, memory, peripheral interface controller

●A large number of programmable and reconfigurable logic resources

●Powerful clock management circuit

●Support multiple I/O interface standards

In addition, since the transmission line distance between the processor and the memory is reduced, SoPC has obvious advantages over board-level systems in terms of speed, signal integrity, electromagnetic compatibility, etc.

1 Functional Block Diagram

Based on the development of digital video vision acquisition and processing board-level system, this paper adopts SoPC to implement the design of motion vision processing and control system. The hardware adopts Altera's StratixII series FPGA, and the software development tools include QuartusII, NiosII5.1 IDE, DSP Builder, MegaCore IP Library5.1 and Matlab7.0. The working principle of the whole SoPC system is as follows: the video image data collected by the camera comes in through the image sensor interface; the digital signal processing block and the digital image and digital video processing IP core complete the processing of the video image, such as motion detection, segmentation, feature extraction, compression, etc.; the Nios II embedded processor mainly completes the control function of the whole system; the I2C bus is used to access the internal registers of the CMOS image sensor; the PC can access the SoPC through the USB interface, and if the distance is far, the data can be transmitted through the Ethernet port; the video image and its processed results can be stored in CF card, FLASH and other storage media through the external memory interface, or directly displayed on the LCD screen through the LCD display interface. The functional block diagram of the motion vision SoPC is shown in Figure 1:

2 Video Capture Module

This module is mainly responsible for video image acquisition, camera parameter setting and pan-tilt stepper motor control. Among them, the internal control register of the camera can be set through the I2C bus. In FPGA, there are two ways to implement the I2C bus controller: one is to use software simulation in Nios II; the other is to use third-party IP cores provided by companies such as Sciworx, CAST, Digital Core Design, etc. These IP cores have working parameters and can be set as needed. Taking the I2C bus controller provided by CAST as an example, the maximum transmission rate is 100Kbps, and it can work in 4 modes, namely master transmitter mode, master receiver mode, slave transmitter mode and slave receiver mode. The dual-channel CMOS image sensor interface controls the timing, frame synchronization and line synchronization of video image acquisition; the pan-tilt control signal controls the rotation of the two stepper motors of the pan-tilt according to the results of video image processing. These two modules need to be developed by ourselves to form IP cores with independent intellectual property rights.

3 Video Image Processing Module

This module includes NiosII 32-bit embedded processor, digital signal processing block, digital image and digital video processing IP MegaCore, and some other logic circuits. It is the focus and core of motion vision processing and control SoPC design, which is introduced in the following parts.

(1) Nios II embedded processor

The Nios II embedded processor is a general-purpose RISC structure CPU that is targeted at a wide range of embedded applications. In the Nios II IDE integrated development environment, you can add and set relevant parameters according to the operating prompts to generate a Nios II embedded processor within a few minutes. The hardware development process is as follows:

① Analyze the functions and performance that the system needs to accomplish

② Start SOPC Builder and select a specific FPGA model

③ Define modules such as CPU, peripheral devices, storage system, etc.

④ Assign base address and interrupt request number (IRQ) to each module

⑤ Generate Nios system module, lock pins, and compile software development process:

[page]

① Start Nios II IDE in SOPC Builder

② Create a C/C++ software project and specify the target hardware

③ Use the engineering template to write the corresponding program

④ After compilation, you can download it to the hardware and run it

Nios II IDE can use C/C++ or assembly language to write programs, and the file extensions are .c and .s respectively. A single Nios II/f CPU takes up about 1800 LEs. If some timers, peripheral devices, etc. are added, the occupied logic units will increase further.

(2) Digital signal processing block

Stratix II series FPGAs have digital signal processing blocks (DSP Blocks) inside. Digital signal processing blocks can support multipliers with different data widths (9×9, 18×18, 36×36) and operation modes (multiplication, complex multiplication, multiplication and addition, and multiplication and accumulation). Each DSP block provides 2.8 GMACS of DSP data throughput. The largest Stratix II device EP2S180 contains 96 digital signal processing blocks, which can provide 284 GMACS throughput and support 384 18×18 multipliers. In addition, the digital signal processing blocks have added new rounding and saturation support to facilitate the import of DSP firmware code into FPGA. For some applications such as voice processing, rounding and saturation can be used because the storage buffer for storing data is of fixed width. Now that digital signal processing blocks that support rounding and saturation are used, it is very convenient to import designs based on DSP processors into FPGAs for implementation.

To design a DSP system on Altera's programmable devices, development tools that support both advanced algorithms and hardware description languages ​​are required. MathWorks' MATLAB and Simulink system-level design tools have the capabilities of algorithm development, simulation, and verification. Altera's DSP Builder combines these tools with Altera's development tools to provide a DSP development platform that shares system design, algorithm design, and hardware design.

(3) Video image processing

IP cores provided by third parties include many customizable IP cores for communication, image encoding and decoding, and video processing. Proper use of these IP cores can greatly shorten development time while ensuring performance and reliability. The following is an introduction to color space conversion IP.

CSC (Color Space Convertorr) is an IP core in the MegaCore IP library file provided by Altera that is specifically used for image color space conversion. Compared with software conversion, it has obvious speed advantages and flexibility:

● Each clock cycle completes the conversion of one pixel

● In Stratix series FPGAs, the clock frequency is greater than 200MHz

● Supports interchange between RGB, YCbCr and YUV

● Users can customize the correlation coefficients of the transformation matrix

● Support signed and unsigned numbers

4 RAM data buffer

The data width of input and output is 2 to 32b 4 RAM data buffer Stratix II series FPGA contains up to 9Mb of on-chip RAM. These RAMs use TriMatrix storage structure, including three sizes of embedded memory blocks: 512b M512 block, 4Kb M4K block and 512Kb M-RAM block, each of which can be configured to support various features, such as single-port RAM, dual-port RAM, FIFO, etc., to provide solutions for large storage applications.

5 External Memory and Peripheral Interfaces

Stratix II series FPGAs are optimized for reliable data transmission of external memory and support the latest memory interface to access off-chip memory. Developers can quickly and easily integrate various large-capacity memory devices into complex system designs using Stratix II's advanced device features and customizable IP cores. Stratix II supports a variety of latest memory interfaces. The on-chip processor and peripherals of Stratix II series FPGAs are connected via the Avalon switching bus. The Avalon switching bus is a dedicated internal connection technology developed by Altera that uses minimal logic resources to support data bus multiplexing, address decoding, generation of waiting cycles, address alignment of peripherals, and interrupt priority assignment. The customizable IP cores for peripheral interfaces include controllers such as USB, I2C, Ethernet, and PCI. Most of these IP cores are provided by third parties and can be tried for free or purchased at a partial cost. The USB2.0 controller and Ethernet interface controller used in this system are both provided by Mentor.

6 Clock Management Circuit

Stratix II series FPGAs have up to 48 high-performance, low-skew global clocks, which can be used for high-performance functions or global control signals; up to 12 programmable phase-locked loops (PLLs), with complete clock management and frequency synthesis capabilities, including clock switching, PLL reconfiguration, spread spectrum clocks, frequency synthesis, programmable phase offset, programmable delay offset, external feedback, and programmable bandwidth. Stratix II has two general PLLs: enhanced PLL and fast PLL. Enhanced PLL is feature-rich and supports external feedback, spread spectrum clocks, programmable bandwidth, etc. Fast PLL is optimized for high-speed differential I/O interfaces and has dynamic phase adjustment (DPA) function. These high-speed clock networks and rich PLLs combine to provide a strong guarantee for the system to work with minimal clock offset.

7 Other components of SOPC

The FPGA configuration interface is used for SOPC configuration, compilation and online debugging; the LCD display interface can be connected to an external LCD display; the alarm signal is a sound or photoelectric signal emitted when a moving target is detected and identified, which can be used for security; the standard I/O port is reserved for future upgrades and expansions.

Innovation

Before the concept of SoPC was proposed, the integrated design of electronic systems was mainly based on the board level. With the continuous increase of system clock frequency and the increasing complexity of circuit functions, this design method has become increasingly difficult to implement, and electromagnetic interference and signal integrity issues have become increasingly prominent. Optimizing the layout and wiring of PCB alone can no longer meet the requirements of high-speed signal transmission and processing. With the continuous development of the semiconductor industry, programmable system-on-chip will become the mainstream of future electronic product development and design with its high performance, reliability, low power consumption, cost and good portability. The motion vision SoPC better solves a series of problems of board-level circuits and can be widely used in many fields such as security monitoring, visual navigation, intelligent transportation, etc., and will definitely have a good market prospect.