Video Processing System Development in a Fully Integrated Design Environment

Typically, the implementation of a video processing system needs to support a variety of video and audio standards and is responsible for converting signals from one standard to another. Multimedia applications require processing signals at video rates, which means that simulations must run in real time during development.

A typical video processing system uses a microprocessor to control a video pipeline that includes a video source and sink, a large memory for storing video data, and a video processing system (Figure 1).

When implementing and debugging various video algorithms, you need to verify their functionality through software and hardware simulation. The real-time nature of video streaming and the large amount of video data required for each frame make simulation of video processing applications particularly challenging.

Design Environment

The Video Basic Kit (VSK) supports rapid development and debugging of high-performance video processing systems for a wide range of video applications. The VSK uses the Xilinx® Virtex-4 XC4VSX35 device, which is optimized for DSP processing through a high ratio of multiply-accumulate blocks (also known as DSP48) in the architecture and has a rich feature set of video interfaces such as DVI, VGA, component (HD), composite, S-video and SDI.

Typically, developing video algorithms requires the use of hardware to verify video operations on real-time data streams, and a simulation environment is needed to develop and test video processing components. VSK provides both software simulation and real-time operations for each component of the video system, allowing you to develop video IP (including filters, video blocksets, accelerators, and video interface conversion) or final applications such as codecs, image enhancement, dynamic gamma correction, and motion estimation. Integration with tool kits and I/O diversity make it quick and easy to introduce video to the board and optimize its running algorithms.

The VSK is accompanied by reference designs, some written in HDL and others built using the Xilinx System Generator for DSP design environment. To remove the complexity of bringing in data through the various video interfaces and sending them to the Virtex-4 device, a library of video interface blocksets is included that allows all interface blocksets to be controlled by a MicroBlaze controller.

To highlight some of the capabilities of VSK, I will describe the MPEG-4 Part 2 decoder demonstration design.

MPEG-4 Part 2

The MPEG-4 decoder demonstration system consists of an FPGA hardware evaluation platform, Xilinx IP cores, and embedded software that together perform decompression operations on industry standard encoded video bitstreams.

For this design, the FPGA is programmed to perform decompression and drive the video display. A Compact Flash card is used to hold multiple compressed video streams and the FPGA configuration bitstream. An embedded processor within the FPGA reads the bitstream from the Compact Flash card, writes it to an external DDR memory, and then sends it to the MPEG-4 Part 2 decoder. The output of the decoder is then reformatted to the video standard to be displayed on an external monitor via a video I/O daughter card.

The system overview is shown in Figure 2. The MPEG-4 decoder core, DDR memory controller, color space converter, VGA interface, macroblock format converter, and MicroBlaze soft-core processor and related peripheral circuits are implemented in the XC4VSX35 FPGA. The ZBT memory, DDR memory, System ACE technology, Compact Flash connector, two-line LCD display, and a digital-to-analog converter are all located on the hardware platform.

Embedded Processors

Video systems generally require a control processor. This processor is usually used to communicate with the host system, set up video processing operations, calculate coefficients, and generally operates as a low-rate data processor. VSK has video standards for input and output sources, and combined with System Generator hardware co-simulation capabilities, allows you to quickly test and debug your system with real-time video streaming.

In this MPEG-4 demonstration design, the embedded MicroBlaze processor acts as the overall system-level controller, handling functions such as user interface, reading the compressed bitstream from the Compact Flash card, sending the bitstream to the MPEG-4 decoder core, and monitoring all system status flags.

The Xilinx System Generator for DSP greatly simplifies the design process of integrating a MicroBlaze processor into the framework. You can use Xilinx System Generator with the Embedded Development Kit (EDK) software tool to implement and simulate a system with a processor and FPGA video processor functionality operating on a live video stream. System Generator automatically generates software drivers to support reading and writing data to the System Generator design. Two methodologies currently support integration of a MicroBlaze controller:

The System Generator design is exported to the EDK system. When used in pcore (processor core) export mode, the memory map block and all other blocks are encapsulated into a pcore peripheral. Software drivers and documentation for the memory map interface are also generated and provided with the peripheral.

EDK projects are imported into System Generator designs for hardware co-simulation. When used in EDK import mode, an EDK file is imported into System Generator by running the EDK Import Wizard. When the Import Wizard is complete, the EDK system is pulled into the System Generator design like a black box. During the import process, the EDK system is extended with the Fast Simplex Link (FSL) interface, which is used to communicate with the memory map.

Hardware Co-Simulation

Viewing the final output video is an important quality measure for all video systems. VSK's input and output source video standards, combined with System Generator's hardware co-simulation capabilities, allow you to quickly test and debug your system with live video streaming.

System Generator provides a hardware co-simulation interface that compiles the System Generator diagram into an FPGA bitstream and associates the bitstream with a new runtime hardware co-simulation block. When you simulate the design in Simulink, the results of the compiled portion are computed in hardware rather than software.

System Generator provides high-speed hardware co-simulation interfaces that allow Simulink vector or matrix signals to be read or written to FPGA hardware in a single transaction. By using these interfaces, you can greatly reduce the number of PC/hardware transactions during simulation, further speeding up simulation beyond the speed that can be achieved with traditional hardware co-simulation. Leveraging the popularity and advancement of Ethernet technology, this interface provides a convenient and high-bandwidth co-simulation method for external FPGA devices.

VSK supports two Ethernet co-simulation modes:

The network-based Ethernet hardware co-simulation interface provides co-simulation access to the FPGA platform through the IPv4 network infrastructure. Since IPv4 networks are widely distributed, this interface provides a way to communicate directly with remote hardware connected to a wired or wireless network. This interface is particularly suitable for FPGA platforms in remote locations (such as across offices or countries), or when multiple designers must share a development board. The network-based Ethernet interface supports operation in 10/100 Mbps half/full-duplex mode.

Point-to-point Ethernet hardware co-simulation provides a co-simulation interface over a raw Ethernet connection. A raw Ethernet connection is a Layer 2 (data link layer) Ethernet connection between a supported FPGA development board and a PC host, with no routing network devices along the way. The point-to-point Ethernet interface supports operation in 10/100/1000 Mbps half/full duplex mode. It also supports huge frames over Gigabit Ethernet connections as long as the underlying connection supports it.

The VSK includes software, hardware, camera, cables, and a detailed user guide and reference designs. It includes a limited version of System Generator for DSP, ISE? software, and Embedded Design Kit (EDK) FPGA design tools, as well as a Xilinx ML402-SX35 development board, video I/O daughter card (VIODC), CMOS image sensor camera, power supply, and cables.

in conclusion

With this complete and easy-to-use solution, the Video Base Kit is an ideal hardware platform to evaluate Xilinx FPGAs for a wide range of video and imaging applications. With full integration and support from Xilinx System Generator for DSP software, the VSK can take full advantage of new high-speed Ethernet hardware co-simulation capabilities to enable real-time system integration, development, and verification of codecs, IP, and video algorithms.