Design of high performance video control system based on multi-core processor ADSPBF561

The increase in clock frequency and the high power consumption caused by leakage current of deep submicron semiconductor manufacturing process have made the design of processors begin to turn to multi- core integrated solutions. In fact, multi-core processor technology is an effective way to improve processor performance, because the actual performance of a processor is the total number of instructions that the processor can process in each clock cycle. Therefore, with each additional core, the number of executable units in each clock cycle of the processor will double. At the end of the last century, dual-core processors began to enter high-end server products. Since Intel and AMD have successively launched dual-core CPUs, the application of multi-core CPUs in personal computers has become an irreversible trend. At the same time, multi-core architecture has shown significant advantages in processor performance, low power consumption, and reduced system circuit board area.

In some ways, embedded applications place more stringent demands on processors, especially in terms of low power consumption and low cost.

In order to meet the increasingly high performance requirements of current embedded device applications and reduce power consumption as much as possible, a significant change in the field of high-end embedded processors is the shift from higher and higher frequencies to multi-core architectures. The demand for multi-core processors has increased significantly in many product areas, and it also provides another way to strengthen the competitive advantage of embedded processor products.

1 Multi-core processor and high-performance video system

For embedded system design engineers, in the design of consumer electronic products, single-core processors are still the absolute mainstream. However, in some applications that require higher processing speed and control capabilities, dual-core processors are no longer highbrow, and multimedia products account for a large proportion of such applications, such as video surveillance (especially surveillance equipment with intelligent analysis functions), video conferencing systems, digital camera systems, automotive visualization systems, etc. Most intelligent video surveillance devices are embedded devices.

Since intelligent video surveillance is an extension of traditional video surveillance, the requirements of traditional video surveillance on hardware platforms must also be met by intelligent video surveillance. However, compared with traditional video surveillance, the needs of users of intelligent video surveillance are more diverse. For example, the functions required by subway system users are to detect whether the yellow line of the waiting area is crossed, whether the crowd density is too large, whether there are suspicious objects left behind, etc.; while for bank system users, what they need is intelligent monitoring and analysis of ATM machines, including whether fake keyboards and card swallowers are installed, whether there are violent behaviors at ATM machines, whether the faces of criminals appear, etc. Therefore, the diversification of user needs can adjust the algorithm according to user needs. On the other hand, intelligent video processing requires the chip to have strong processing capabilities. In addition, many algorithms must also adopt parallel processing when they are implemented. Therefore, traditional solutions based on single-core microcontrollers will face great challenges, while heterogeneous dual-processor or multi-processor solutions integrating MCU and DSP, as well as multi-core processors with powerful computing capabilities, have become the preferred solutions for intelligent video surveillance embedded platforms.

2 Intelligent Video Surveillance System Design

Intelligent video surveillance equipment integrates a variety of advanced technologies such as video image processing technology, pattern recognition technology, embedded development and design. The system hardware equipment is compact, has excellent performance and low power consumption. At present, a variety of intelligent video analysis algorithms have been successfully transplanted to this hardware platform, including regional intrusion detection, line crossing detection, object abandonment detection, object movement or theft detection, directional regional intrusion detection, directional line crossing detection, traffic density detection, and headcount. Therefore, its product performance is very stable and the algorithm execution efficiency is very high.

Figure 1 shows a hardware block diagram of an intelligent video surveillance system developed based on the BF561 platform. The BF561 in the figure uses dual DSP cores, so it can implement very complex intelligent video processing algorithms. This Blackfin DSP adopts a low-power design, and its stability is outstanding among DSPs of the same price. In addition, on the Blackfin DSP platform, ADI also provides many source code development programs, which can be modified for specific applications, so it is very flexible.

Figure 1 Functional block diagram of video surveillance system based on BF561

The ADV7183B in this system can decode the CCD image signal to adapt it to the ADSP BF561 processor; the ADV7179 can encode the signal processed by the ADSP BF561 processor and then send it to the display for display. In addition, the UART interface and EBIU port of the ADSP BF561 processor can be used to interface with the console and various processor peripherals for data storage or to connect to the remote console via Ethernet.

2.1 Dual-core converged processor BF561

The ADSP BF561 processor is a high-performance device in the Blackfin product family. It includes two independent ADI processor cores, each of which contains a dual multiply-accumulate signal processor and a RISC-like instruction set. Each core uses a complete SIMD architecture and contains instructions for video acceleration and image processing. The characteristic of the convergent processor is to reduce or avoid the use of separate digital signal and control processors, thereby reducing material costs and simplifying the complexity of software and hardware development.

ADSP BF561 is conveniently interfaced, powerful, and has two easy-to-control video input/output ports, supports ITU-R656, and includes a convenient DMA controller and UART interface, as well as a large number of GPIOs. At the same time, the Blackfin processor also provides a comprehensive power management solution.

Its unique low power and low voltage design (Blackfin can be powered by an external voltage below 2V) can reduce the overall power consumption of the system, which is of great significance for intelligent video surveillance systems that run 24/7.

The dual-core BF561 has twice the processing performance of the BF533 processor and twice the on-chip memory, which greatly enhances the data bandwidth and is fully compatible with the code of the BF533 processor. The 328KB on-chip memory of the BF561 processor provides two parallel data ports, so it can directly connect parallel AD and DA converters or other general peripherals. In addition, the BF561 processor also has multiple independent DMA controllers built in and supports automatic data transmission, which can minimize the burden on the processor core. The BF561 continues the rich interface characteristics of the Blackfin processor, so it can bring great convenience to the design.

Most multi-core processors have been heterogeneous multi-core processors that integrate multiple DSP and CPU cores. They use DSP and CPU to serve signal processing tasks and control tasks respectively, which is very suitable for communication applications such as mobile phones with simple task division. The dual-core DSP of BF561 is an exception. Each core can complete signal processing and control processing tasks at the same time. Therefore, there is no need to allocate these two types of tasks between multiple cores. In this way, tasks can be allocated arbitrarily to balance the task processing capacity between cores. The dual-core processor of BF561 is independent of each other. Therefore, the tasks of the two processor cores can be allocated according to the application during design, and the two cores are the same in executing instructions. The two processor cores can be used to run the embedded operating system (OS) and signal processing respectively. For embedded systems with a large amount of signal processing tasks, BF561 can also use both cores for signal processing at the same time, and then use the BF533 (or BF536 processor) with the same core to run the embedded operating system.

2.2 ADV7183B Video Decoder

This intelligent video surveillance system uses the ADV7183B chip as the decoding chip for the system's CCD image signal. ADV7183B is a comprehensive video decoding chip that can process input video sources in three formats: CVBS, S-Video, and YPrPb under NTSC or PAL formats. It can be widely used in electronic products such as projectors, digital TVs, DVDs, and game consoles.

ADV7183B has internal line-locked system clock (LLC) and adaptive digital line length tracking (ADLLT) circuits, which can provide dual video locking functions, as well as real-time clock, information output functions and 3-line chroma comb filters. ADV7183B's complete AGC and clamp control functions can program video adjustments for chroma, brightness, saturation and contrast, and can automatically detect NTSC or PAL. It has 12 analog video input channels, which can be set to two-line continuous bidirectional port mode and is compatible with I2C; in addition, ADV7183B can also support different modes of video input and 16-bit wide bus digital output, and its input peak-to-peak value is 0.5 V to 2 V.

Through different configurations, the six analog video input channels of ADV7183B can support six CVBS input signals, three S-video input signals and two YPrPb analog video input signals, and the type and channel of the input source can be controlled by the register INSEL.

In addition, ADV7183B also supports three output interface modes: including LLC-compatible synchronous pixel interface, CAPI interface and SCAPI interface. The default mode of ADV7183B is LLC-compatible 8-bit CCIR656 data. Figure 2 shows the software configuration flow chart of ADV7183B.

Figure 2 ADV7183B software configuration process

2.3 Camera Selection

This design uses the automotive driver assistance visual sensor system launched by Aglaia GmbH of Germany, which has the function of preventing traffic accidents and improving traffic flow. Its real-time electronic eye solution consists of hardware and software, which can imitate the human visual system from the eyeball (image capture) to the brain (image analysis and interpretation), and can notify the driver of the traffic signs ahead, warn of potential traffic conflicts and automatically limit the speed. It can also issue a warning when the driver drives out of the lane line, and even provide "fatigue analysis" to help the driver avoid collisions. AglaiaGmbH's complete solution includes cameras, image sensors and image processing algorithms, and its traffic analysis application includes various functional modules that can be configured by software, such as lane line recognition, object recognition, traffic sign recognition, headlight and taillight recognition, etc.

Aglaia GmbH chose the BF561 processor for its automotive driver assistance vision sensor system due to its high performance, low cost, low power consumption and rich I/O interfaces. The core of the system is four BF561 processors, which consume less than 20W of energy at about 3A current, including the energy consumption of the embedded microcontroller and signal processor. The dual-core BF561 device's 1.2V design consumes less than 1A of current, making it easy to meet Aglaia's 20W limit.

3 Conclusion

The intelligent video surveillance system in this paper can be widely used in military, public security, finance, transportation, education, public facilities, community, personal, industrial and commercial fields. Based on the unique advantages of the BF561 processor, Blackfin DSP can do a good job in algorithm parallel processing, especially BF561 uses dual DSP cores, so it can implement very complex intelligent video processing algorithms. Because Blackfin DSP adopts a low-power design, its stability is also very good among DSPs of the same price.