Design of multi-channel audio/video acquisition and processing system based on DSP

1 Introduction

Currently, in digital image processing, real-time performance has become one of the technical difficulties due to the large amount of data and high algorithm difficulty. If a dedicated circuit is used, although real-time performance is guaranteed, the flexibility of the system is greatly reduced. Therefore, it is urgent to seek a high-speed general-purpose digital signal processing system.

The TMS320DM642 (hereinafter referred to as DM642) digital signal processor launched by II can process 4-channel analog video and audio input, 1-channel analog/digital video and 1-channel analog audio signal output in real time. It can adapt to the analog signal input of PAL/NTSC standard composite video CVBS or component video Y/C format, and can adapt to PAL/NTSC standard S terminal or digital RGB analog/digital signal output, and can adapt to standard microphone or stereo audio analog input and standard stereo audio analog output. It has the function of real-time processing and analysis of multi-channel acquisition data, and can realize data and image overlay display.

2 DM642 Introduction

The DM642 digital signal processor can operate at 500 MHz or 600 MHz, can complete up to 4.8 G operations per second, has online programming capabilities, and has a rich peripheral interface that can be connected to a variety of memories and can be directly connected to the network. It is an optimal device for high-speed image processing.

The DM642 CPU uses the second-generation VelociTI.2 core structure, with dual data paths, 8 computing units, and can execute

DM642 adopts a complete memory hierarchy architecture with 2-level memory. The cache controller in the 2-level memory can automatically manage and schedule the hierarchical memory architecture. External memory access and on-chip peripheral access are completed through EDMA.

DM642 has three video input and output ports and multiple audio signal input and output serial ports. The external memory interface EMIF provides a 64-bit width external bus data interface, supporting glueless interfaces with various devices. DM642 also has a host parallel interface, a peripheral device interconnection port, a multi-channel cache serial port and a general I/O port.

3 System Function and Hardware Circuit Design

3.1 Overall system structure

The overall structure of the system is shown in Figure 1. The image acquisition and preprocessing unit mainly completes the input of image signals, has a multi-channel signal multiplexing function, and digitizes the input analog video signal and converts the format. FPGA controls the logic of the system and the flow of image data, and can process the image data output by DM642 and then output it to the image encoding unit. The image processing unit uses DM642 for high-speed data processing and analysis. The image encoding unit encodes the image data to form a standard analog video signal, which can be directly output to the display device.

For 4-channel video and 4-channel audio signals, the CPU can only process 1 channel of data at a time. The CEO address space outside the DM642 chip located in the EMIF is expanded with 2 4 Mx32 bit SDRAMs, which can store image acquisition data and image processing data respectively, to improve data acquisition and storage speed.

3.2 Video port interface design

DM642 has 3 video ports, each of which can be configured as 2 channels, but both channels must be video input or output ports at the same time. Combined with practical applications, DM642 has 4 analog video inputs (cif format, resolution 352×288) and 1 analog video output.

VPO A channel is configured as 8-bit BT.656 video input or output port, connected to the first channel video input or video output. VPl A channel is configured as 8-bit BT.656 video input port, connected to the second channel video input. VP2 A and B channels are configured as 2 8-bit BT.656 video input ports, connected to the third and fourth channel video input. VP0 and VPI B channels are configured as MCASP, connected to 4 audio codecs.

The TVP5150 video encoder supports PAL/NTSC, CVBS or Y/C analog video input and 8-bit BT.656 digital video data stream output. The SAA7105 video decoder supports 8-bit BT.656 digital video data stream input and PAL/NTSC CVBS or Y/C analog video output. The internal registers of the video encoder/decoder are programmed through the I2C bus of the DM642 to achieve different input and output. The corresponding pin functions of the DM642 and TVP5150 are shown in Table 1.

The parameters of the video codec are configured through the I2C bus. Since the I2C slave address of TVP5150 has only two options, a CBT3257 2-to-1 switch is required to switch.

When used as a video input port, the line/field synchronization of the video data also includes the EAV and SAV time base signal control in the BT.656 digital video data stream. The video port only needs the video sampling clock and sampling enable signal (to control the start of sampling). TVP5150 uses the system clock SCLK to provide the sampling clock and the programmable pin GPCL to provide the sampling enable. When used as a video output port, the video port needs to provide the clock and line/field synchronization signals for SAA7105.

In the video output circuit, J1, J2, and J3 can be configured as RGB output signals, J2 and J3 can be connected to the S terminal, and J1, J2, J3, J4, and J5 can be directly output to the computer monitor. The specific interface circuit is shown in Figure 2 and Figure 3. The corresponding pin functions of DM642 and SAA7105 are listed in Table 2.

[page]

3.3 Multi-channel audio serial port interface design

The author uses 4 analog audio inputs and 1 analog audio output, and adopts the TLV320AIC23B audio codec, which supports microphone/stereo analog input/output and digital audio data stream output/input.

The PLL1708 programmable video/audio synchronous digital phase-locked loop provides clock signals for McASP and TLV320AIC23B. The SCK02 port is connected to the AHCLKX of McASP, and the SCK03 port is connected to the master clock MCLK of TLV320AIC23B. The clock input of PLL1708 is 27 MHz. The corresponding pin functions of DM642 and TLV320AIC23B are shown in Table 3.

The AIC23B data port is configured as a slave, and the 8 receive/transmit pins of McASP are configured as 4 receive/4 transmit, connected to the Dout/Din of the 4 codecs respectively. The receive frame synchronization of McASP is configured as an output, and is given to the LRCout of the 4 codecs at the same time. The transmit frame synchronization of McASP is configured as an output, and is given to the LRCin of the 4 codecs at the same time. The transmit bit clock ACLKX of McASP is configured as an output (divided by AHCLKX), and is given to the BCLK of the 4 codecs at the same time. The control port of AIC23B is configured as I2C, which is switched by the CBT3257 2-to-1 switch. The specific circuit is shown in Figure 4.

In addition, when powering DM642, attention should be paid to the power-on sequence of the CPU: the CHU core should be powered on before the I/O and powered off after the I/O. The CPU core and I/0 should be powered on at the same time as much as possible, and the time difference between the two should not be too large (<1 s), otherwise it will affect the life of the device or damage the device. Using a programmable clock circuit can well solve the clock problem of the entire circuit.

4 System Software Design

The system software includes system initialization settings, image processing algorithms and screen overlay programs. The specific software flow is shown in Figure 5.

4.1 System initialization setup procedure

Initialize the entire hardware system, including power-on initialization of DM642, setting of DM642 registers and system configuration pins, and setting of TVP5150 and SAA7105 registers using the I2C bus.

The boot mode of DM642 is booting from EMIFA, setting pin AEA[22:21] to ll, and initializing other configuration pins to default values. The peripheral configuration register (PERCFG) is used to configure the control video port, multi-channel buffered serial port, and multi-channel audio serial port, and is initialized to 0x0000 0079h. The device status register (DEVS-TAT) is used to control the status of each peripheral device of the circuit: EMAC, HPI, PCI, CPU clock frequency selection mode, circuit boot mode, and EMIFA input clock selection, and is initialized to 0x0000 005Ch.

4.2 Image Processing Procedure

The collected image data is processed and analyzed, and the video stream is formatted, which can be composite video or component video, or compressed and stored for later viewing.

4.3 Screen Overlay Program

The data in the FPGA internal FIFO and the data output by the video port are mixed to complete the screen display function. There are several ways to overlay the screen, the background can be transparent, semi-transparent, opaque, and the overlay position can be set arbitrarily. You only need to modify the starting point coordinates of the overlay image and add the corresponding image information to the corresponding video image queue.

4.4 Video/audio signal acquisition and storage and image data reading program

For 4-channel video signals and 4-channel audio signals, the CPU can only process 1 channel of data at a time. Therefore, 2 4Mx32bit SDRAMs are expanded outside the DM642 chip. At any time when the system is working, one is used for image acquisition, and the acquisition part writes image data to this storage area. The other is used for external reading of image data, and the DSP can read the image data in this storage area. The important feature of the dual SDRAM structure is that the DSP switches back and forth on the data operation of the storage area. When the A/D conversion data fills up SDRAM-1, the FPGA will send an interrupt signal to the DSP. At this time, while the DSP reads the data in SDRAM-1, the A/D conversion data is written to SDRAM-2. When the data in SDRAM-2 is full, the FPGA sends an interrupt signal to the DSP. At this time, the DSP reads the data in SDRAM-2, and at the same time, the A/D conversion data is written to SDRAM-1. In this way, data writing and reading are realized simultaneously. Since the speed at which DSP reads data from SDRAM is much faster than the speed at which A/D converter writes data, acquisition and external access can be performed simultaneously, and ping-pong switching between two storage areas is used to meet the requirements of real-time data exchange.

Since the DSP switches back and forth in the data operation of the storage area, it is necessary to use interrupts to realize data access. The specific implementation is to set interrupts inside the DSP, and INT6 is selected as the interrupt trigger pin. When the external interrupt signal arrives, the corresponding interrupt service program executes the interrupt response.

5 Conclusion

This system can process 4 channels of CIF digital video at the same time and can switch dynamically. The total sampling rate can reach 100 frames/second, and the rate of each channel is 25 frames/second, which can fully guarantee real-time image acquisition.

There are many ways to overlay images, and the overlay position can be set arbitrarily by modifying the coordinates of the starting point of the overlay image. DM642 also has a network interface, which can compress the output video signal and transmit it in real time over the network.

The image acquisition and processing system based on DM642 can be widely used in set-top boxes, IP videophones, network video conferencing and other fields.