Switching control of multiple video channels based on DSP+FPGA

With the rapid development of computer and digital image processing technology, video surveillance technology is widely used. Traditional video surveillance systems use a single camera to monitor a fixed scene. Not only is the video field of view limited, but it is also unable to monitor the same object in different directions. Here, a multi-channel video surveillance system is proposed, which can monitor different scenes by stably and reliably switching different video channels. This system not only makes up for the limited range of traditional surveillance videos, but also improves the utilization rate of surveillance resources and reduces surveillance costs.

1 System Hardware Structure

The DSP+FPGA hardware structure solution is adopted, and DSP and FPGA are used to control MAX4312 to select the required video channel, so as to achieve the purpose of switching between multiple video channels. The system structure block diagram is shown in Figure 1.

1.1 Selection of control components

According to actual needs, DSP uses ADSP_BF561 (hereinafter referred to as BF561), a high-performance processor of the Blackfin series launched by ADI, as the algorithm processing and control core. The BF561 processor contains two independent BF533 cores, each of which contains two multiplier/accumulators (MAC), two 40-bit ALUs, four video ALUs and one 40-bit shifter. The Blackfin processor uses an improved Harvard structure and a hierarchical memory structure. The L1 memory generally runs at full speed with no or little delay. L2 is another level of memory, which is distributed on or off the chip, and access to it consumes multiple processor cycles. At the L1 level, the instruction memory only stores instructions, two data memories store data, and a dedicated temporary data memory stores stacks and local variable information. At the L2 level, a unified storage space is used to store instructions and data. Here, the L2 level memory is used.

FPGA uses Altera's EP3C40F48C to pre-process the video signal. The chip has up to 24,624 logic units and an advanced external memory interface, allowing external single data rate (SDR) SDRAM, double data rate (DDR) SDRAM and DDR FCRAM devices to be integrated into complex system designs without reducing data access performance. It has 129 compatible LVDS channels, each with a data rate of up to 640 Mb/8, 4 programmable phase-locked loops and 8 global clock lines. In addition, the EP3C40F484C has low power consumption, with a total power consumption of about 300 mW when running globally.

1.2 Flash Design

According to the needs of system design, Flash uses CMOS type M29W640D, whose space is 18 Mx16 bit. In this system, the external data lines are D1~D15, and the address lines are A2~A22. Attention should be paid when connecting, because Flash is 16 bit wide, and the address needs to be misaligned in hardware connection, that is, DSP's A2 corresponds to Flash's A1, DSP's A3 corresponds to Flash's A2, and so on. In software programming, any operation on Flash, such as erasing and writing, must first write a certain number of fixed control command words to a specific address, that is, the sign-in instruction, to ensure that the memory will not be misoperated when the power is on and off.

1.3 Design of SDRAM

The external data memory used in this system is a large-capacity SDRAM MT48LC16M16, with a capacity of 4 Bankx8 Mx16 bit. According to the actual requirements of the system design, SDRAM stores real-time image data, benchmark data and program code when the system is running. Since the access speed of Flash is relatively slow, the benchmark data and program code stored in Flash should be moved to a faster memory in advance when the system is initialized, so that the DSP is not restricted by the memory reading time and can give full play to its high-speed performance.

1.4 Video acquisition module design

Since the switching time of the video channel is very short, the switching control module needs to have a very accurate judgment of the video signal in order to send the switching command in time, so a high-speed converter needs to be selected. In addition, since the power supply of this system is 5 V, a low-power device needs to be selected. Therefore, AD9203 is selected as the digital quantization device of the analog signal.

AD9203 is a single-channel, low-voltage, high-speed A/D converter produced by ADI, with a sampling rate of up to 40 Ms/s. Its accuracy is stable and reliable, and it basically maintains 10-bit accuracy within the full sampling bandwidth; at a sampling rate of 40 Ms/s, the ENOB (effective number of bits) still reaches 9.55 bits, the differential nonlinearity is ±0.25 LSB, and the signal-to-noise ratio and distortion remain at around 59 dB. The operating voltage of AD9203 is relatively flexible, allowing it to vary within the range of 2.7 to 3.6 V, which is particularly suitable for high-speed operation of portable devices at low voltage. Under a 3 V power supply, the power consumption is only 74 mW when working at full speed of 40 Ms/s; at 5 Ms/s, the power consumption will drop to 17 mW, and in standby mode, the power consumption is only 0.65 mW. For the peak-to-peak value of the input signal, it is usually set to 1 Vp-p or 2 Vp-p. In addition, AD9203 allows external voltage reference, and can flexibly set the peak-to-peak value of the input signal between 1 and 2 V according to design requirements. Figure 2 is the circuit application schematic of AD9203.

[page]

2 Control module design

The workflow of the whole system is to install multiple cameras at different monitoring points for image monitoring. Multiple channels of image video signals pass through MAX4312, and one of the video signals is selected for input at each moment. After being processed by AD8013AR, the video signal is output from the image monitor as a digital image signal. At the same time, the corresponding line (VIDEO_Hs) and field (VIDEO_Vs) signals are output from the line-field separation chip EL4583C, and the image back porch signal (AD_BRST) and video quantization signal are output from AD9203. Then the line (VIDEO_Hs) and field (VIDEO_Vs) signals are sent to DSP, and the back porch signal (AD_BRST) and video quantization signal are sent to FPGA. FPGA uses these signals to pre-process the image, and then sends the processed image to DSP through the PPI port for algorithm processing. Finally, DSP sends a video channel switching command to complete the switching between different channels, and sends the final processing result to the alarm and status indication device to realize the monitoring alarm function. The software control process is shown in Figure 3.

2.1 FPGA module function

The FPGA control part is used to pre-process the video signal and wait for the DSP to send a switching command. The basic idea is: after a certain video signal passes through AD9203, the output video quantization signal is sent to the FPGA for pre-processing, and then sent to the DSP for algorithm processing. At the same time, the FPGA waits for the video channel switching command sent by the DSP according to the field signal.

During the experiment, it was found that the DSP's search for images is triggered by the rising edge (or falling edge) of the row signal, so the row signal needs to have a stable rising edge (or falling edge). Since the signal may jitter during the separation and transmission process, the rising edge (or falling edge) of the signal is unstable, which affects the DSP's judgment of the video signal. If this unstable row signal (VIDEO_Hs) is still used as the mark for the DSP to search for images, the received image will be misaligned between two adjacent rows, and the image drift phenomenon can be seen with the naked eye. The image (a) in Figure 4 is an image received with the VIDEO_Hs signal as the mark. This requires the FPGA to de-jitter the row (VIDEO_Hs) signal, but the signal received by the FPGA is only the row signal separated by the EL4583, and the specific position of the rising edge (or falling edge) of the signal cannot be distinguished. Therefore, FPGA generates a line flag signal (H_flag) as needed to replace the unstable line signal (VIDEO_Hs), and then sends the line flag signal (H_flag) and field signal (VIDEO_Vs) to DSP to make accurate judgment on the video signal. The image (b) in Figure 4 is the image received with the H_flag signal as the flag.

2.2 PPI port settings

The data communication between FPGA and DSP is realized through PPI port. PPI (Parallel External Interface) is half-duplex, with bidirectional ports, and can input and output up to 16 bits of data. PPI has 5 memories to control its operation. Among them, the PPI control register (PPI_CONTROL) sets the operation mode of the PPI port, the polarity of the control signal, and the bandwidth of the port. In this design, PPI_CONTROL is set to 0x00EC, the working mode of PPI is set to ITU_656 mode, the width of the port is set to 8 bits, and it is only a mode for receiving data. Since each PPI_CLK_initiated event (i.e. input or output operation) on the DMA bus is processed as a 16-bit entity, that is, if the transmitted data is not 16 bits, the high bits of the data are padded with 0 to make 16 bits of data. In this case, a more effective way is to pack the data, that is, synthesize the 8-bit data transmitted by the two FPGAs into 16-bit data, and then transmit it, which effectively utilizes resources and improves transmission efficiency.

2.3 DSP control function implementation

The process of implementing the DSP control function is to first set the PF pin of the DSP to determine the interrupt signal on the PF pin. The communication of the interrupt signal between the FPGA and the DSP is defined to be transmitted through PF6 and PF7, and then set the DMA (direct memory access). When processing the video signal, the odd and even field signals are processed separately, that is, only the odd field or even field of the signal is processed at a certain moment, so as to increase the signal processing speed. This requires the odd and even fields of the video signal to be stored separately. When setting the DMA, the odd and even field signal data is stored under different addresses.

When DSP receives the signal sent by FPGA through PF pin, it starts to realize its control function. The specific control switching process is as follows: when DSP receives the first VIDEO_Vs signal transmitted by PPI port, it starts to count the video signal through field counter (VsCount), and then reads the value of VsCount. When the count of VsCount reaches the switching requirement, DSP will send a switching command to FPGA, that is, write a value under a certain address, so that FPGA can complete the switching of video channels through DSP_A and DSP_D reading commands. Because the field signals of the two channels may be out of sync before and after the video channel is switched, the image search starts in the second field after the channel is switched. Figure 5 shows the results of DSP receiving images from the first field and the second field respectively.

3 System Test Results

The experiment switches 24 video signals and sets the switching time of each video signal to 6 fields, that is, the video channel is switched once every 6 fields. The stabilization time when switching between different video channels will not exceed one field, that is, the two channels can be stable within one field after switching. The experimental results show that the system can stably and reliably switch between multiple video channels, and multiple cameras can be added to a set of monitoring equipment to monitor different scenes in real time. This not only effectively solves the shortcomings of the traditional monitoring system with a single monitoring target, but also improves the monitoring efficiency and reduces the monitoring cost.

4 Conclusion

First, the hardware architecture of the whole system and the performance of the chip are briefly introduced, and the workflow of the whole system is described. Then, the control and gating of multiple video channels by DSP and FPGA are emphasized. The whole FPGA design program is written in Verilog hardware description language on QuartusⅡ platform, and the DSP program is successfully compiled in VisualDSP++. Both have been debugged and verified many times. The system design has been successfully applied to multi-scenario monitoring and alarm systems to accurately and reliably switch different video channels.