Embedded vision applications of high-performance DSP: Leveraging machines for smart eyes

　The 3D version of "Titanic" is a global hit. After 15 years, this sad story still makes people sigh. Although there are still many differences on the cause of the sinking of the Titanic, the fact that it hit an iceberg is certain. The reason why the Titanic hit the iceberg was ultimately attributed to the crew's failure to find a telescope (because the only pair of binoculars on the ship was locked in a cabinet by the second officer, and the second officer who kept the key to the cabinet did not board the ship in the end), so the lookout had to observe with the naked eye. When the iceberg was discovered, it was too late.

　　The observation ability of the naked eye is limited. For thousands of years, people have been using various means to make up for or replace the defects of the naked eye. With the advancement of technology, such tools have gradually added more intelligent processing functions from the original simple visual simulation function. For example, in the field of intelligent security monitoring, monitoring equipment can not only replace the human eye to provide 7×24 hours all-weather, unblinking video recording, but also provide warnings for special events or improve the clarity of video recording when such events occur; in the automotive field, when you are driving on the road at a high speed and ignore the speed limit sign ahead, at this time, another pair of "eyes" in the car will issue a warning to remind you to drive at the speed limit...

　　Similar applications are rapidly entering our lives with the concept of " embedded vision " and are receiving more and more attention. At the International Consumer Electronics Show (CES) in Las Vegas in early 2012, embedded vision was named one of the top ten technologies that will change the consumer electronics market this year. According to IMS Research, by 2015, there will be 3.5 billion electronic systems with visual capabilities, including smartphones, TVs, and automotive driver assistance systems ( ADAS ).

　　The Past and Present of Embedded Vision Applications

　　In fact, the concept of "embedded vision" has not been proposed for long, but similar applications have gone through several years of research and development and application in many industries, including intelligent video surveillance, biometrics, intelligent robots, machine vision in the industrial and medical industries, and ADAS, which is currently receiving much attention in the automotive industry , etc.

　　ADI  is one of the semiconductor companies that is at the forefront of the industry's development and application of embedded vision technology. ADI believes that it is essential to provide components that enable products to "see" and respond to current and future industrial, medical, and consumer electronics visual application development. To this end, ADI provides a variety of embedded vision signal chain devices, such as digital signal processors ( DSPs ), encoders/decoders, HDMI transceivers, amplifiers, and special power management ICs, making it extremely convenient to deploy a variety of visual applications. In particular, Blackfin devices are widely used in early "embedded vision" applications. The following are four representative early "embedded vision" application success stories.　　

 

　　⒈ Six years ago, the German company Aglaia GmbH developed and launched a driver-assistance visual sensor system for cars. Its real-time electronic eye solution consists of hardware and software (the core processor uses a dual-core Blackfin A DSP  BF561), which mimics the human visual system from the eyeball (image capture) to the brain (image analysis and interpretation). Aglaia's system can notify drivers of traffic signs ahead, warn of potential traffic conflicts, automatically limit speed, and warn drivers when they leave lane lines. The system can even provide "fatigue analysis" to help drivers avoid collisions.

　　⒉ Enhanced Vision (EV), a leading company in vision-assistance technology that provides advanced technology solutions for the rapidly growing visually impaired population, has selected the Blackfin BF561 processor for its latest generation of electronic low-vision assistive devices that allow the visually impaired to take care of themselves at home or when shopping for food or medicine.

　　⒊ Qritek in Seoul, South Korea can help you "keep a close eye" with its new IRIBIO mouse biometric system - the system uses a micro camera and an embedded iris authentication engine circuit board, which is very convenient to install in a computer mouse. The software embedded in the mouse uses the user's unique iris pattern as a password to handle authentication. The registration and authentication data is also securely stored inside the mouse, separate from the PC, to avoid threats from hackers.

　　⒋ Nanjing Xin Yi Tian Technology Co., Ltd. and the University of Hong Kong have cooperated to develop an intelligent and autonomous video analysis device - ThinkSmart V1, which detects, tracks, and classifies targets by monitoring on-site video data, analyzes the target's behavior, and sends early warning information in real time.

　　As Jeff Bier, founder of the Embedded Vision Alliance, pointed out: "Vision algorithms have very high requirements for computing power, and they vary greatly." In the above applications, high-performance Blackfin processors are used, and the first high-performance dual-core Blackfin processor BF561 is the main application. Vision algorithms and applications have greatly extended the range of processor performance requirements, and processor performance is crucial.　　

 

　　"In many embedded systems, designers face stringent constraints such as size, power consumption and cost, so it is critical to provide sufficient processing power at low cost and low power consumption." Jeff Bier has a deep understanding of embedded vision applications, and this view is also one of the main trends in current embedded vision applications. ADI  's latest Blackfin ADSP-BF609 and ADSP-BF608 processors cater to these demand trends for current and future embedded vision applications in a timely manner, aiming to promote complex multi-functional analysis technologies to various levels of embedded vision applications with high cost performance and low power consumption.

　　ADSP-BF608/9 brings dual-core high-performance processors to embedded vision applications

　　Whether it is from the initial video surveillance application to the automotive vision ADAS system and various innovative embedded vision applications, the most fundamental requirement is "vision" - video processing that meets the required clarity. On the other hand, current embedded vision pays more attention to the integration of intelligent analysis features, which puts higher requirements on embedded system processors, requiring faster video processing capabilities and more powerful intelligent computing and analysis capabilities.

　　ADSP-BF608/9 was designed with these application requirements in mind and is optimized for embedded vision applications. Both are equipped with a high-performance video analysis accelerator called a "pipelined vision processor (PVP)" - PVP consists of a set of configurable processing modules designed to accelerate up to 5 parallel image algorithms to achieve extremely high analysis performance. It can perform 25 billion mathematical operations per second and can perform functions including object monitoring, tracking and recognition.

　　"There is no unified standard in the industry for video analysis algorithms at present, and each customer's algorithm varies greatly, so the processor needs to provide sufficient flexibility for different customer algorithms." ADI  DSP Asia Pacific Business Manager Lu Lei pointed out. "The PVP module of ADSP-BF608/9 provides sufficient flexibility and is a flexible video processing engine. Specifically, it includes 12 high-performance, flexible signal processing modules such as convolution, scaling, and arithmetic modules, which support various common algorithms." Lu Lei further shared, "After the video enters PVP, it will go through filters, Sophie edge detection, polar coordinate transformation, and edge splitting. Taking a video image with a resolution of 800×480 as an example, if this part of the operation is not in PVP but in Blackfin, three 500MHz Blackfin cores are required."

　　For current embedded vision applications, low power consumption is also an important feature requirement. The high performance of PVP not only saves core resources and improves the computing power of the processor, but also greatly reduces power consumption. In addition, special optimizations were made during the design of PVP to simplify memory storage and reading and writing. One of the important benefits is low power consumption, because the processor usually consumes more power when reading and writing memory.

　　In addition to the low power consumption advantage brought by high-performance PVP, the BF60x series has dynamic power management, and the processor can adjust different clock frequencies to provide corresponding power (for example, when the processor clock frequency is reduced by 25%, the power consumption of BF608/9 will also be reduced by 25%). In addition, BF608/9 is based on a gated clock core design, which can selectively cut off the power supply of functional units according to instructions, and also supports a variety of power saving modes for periods when the required CPU action is very little (or no CPU action is required at all). Therefore, the low power consumption characteristics of ADSP-BF608/9 are outstanding, with a typical power consumption of 400mW at 25°C.

　　In addition, BF608/9 has a rich set of peripherals and memory connection interfaces: DDR2, LPDDR, RSI (mobile storage interface, supporting MMC, SD, SDIO and CE-ATA) connection capabilities; USB2.0, Ethernet, 5-category serial interface, ePPI for CMOS sensors and LCD; Link Port for high-speed multi-processing. The rich interfaces provide good support for the design diversity of customers.

　　Cost-effectiveness is always a key weapon in market competition, especially for embedded systems that are just beginning to gain popularity. Lu Lei said: "When setting performance targets for BF608 and BF609, ADI focused not only on the clock frequency and processing power of the DSP, but also on the performance-price ratio, performance-power ratio, and performance-area ratio."

　　A Typical Application of DSP -BF608/9

　The PVP integrated in ADSP - BF608/9 has up to 5 video analysis functions. A single processor can meet all embedded vision applications and is a suitable choice for many current embedded vision applications, such as advanced automotive driver assistance systems, industrial machine vision, and security/surveillance systems.

　　⒈Security monitoring

　　The future development of the monitoring market lies in intelligent analysis, and intelligent analysis can only be fully utilized on the basis of high-definition monitoring. For security applications, the scope involved is very wide. Usually different manufacturers and end customers have different requirements, including clarity, real-time processing and differentiated functional requirements. For example, some users are in the subway system, and they need the function of detecting whether the yellow line of the waiting area is crossed, whether the crowd density is too large, whether there are suspicious objects left behind, etc.; some users are in the banking system, and they need intelligent monitoring of ATM machines, such as analyzing whether fake keyboards are installed, card swallowers are installed, whether there is violence at the ATM machine, whether the face of the criminal appears, etc.

　　Therefore, more and more security applications need to add intelligent recognition algorithms, such as the number of vehicles and pedestrians entering and exiting at a specified time, the detection of reverse driving, license plate recognition, etc. In this application, BF608/9 can be used as a coprocessor to analyze the video and then send the results back to the main processor. Compared with the general system that stores all videos, the latter requires relatively high bandwidth and storage, while the former can only store what is needed, thereby saving hard disk space and reducing the overall system cost. This application has higher and higher transmission requirements for DSP . On the one hand, it requires the image preprocessing algorithm to be optimized and streamlined as much as possible. On the other hand, it also puts forward higher requirements on the core processing capability, internal bus architecture, data transmission capability, peripheral interface, and hardware overall architecture and instruction set support for preprocessing algorithms of the image preprocessing main chip. In terms of video analysis capabilities, BF609 supports high-definition analysis and BF608 supports VGA video analysis. Users can flexibly choose according to their needs.　　

 

　　⒉Application in ADAS

　　According to the main cause analysis report of road traffic accidents published by the German INVENT-FAS (Intelligent Transportation and User-Friendly Technology - Driving Assistance Active Safety Technology) Association: about 28% of accidents are caused by merging or changing lanes during driving; about 24% of accidents are caused by rear-end collisions; about 15% of accidents are caused by driving off the road. Developing multi-video processing capabilities such as collision warning, lane departure warning, lane change warning, pedestrian detection, etc. for automobiles has become the design focus of　　 automotive driving assistance systems ( ADAS ).

 

　　Vision ADAS systems usually include video amplification, encoding, transmission, decoding, processing and display functions. The resolution of cameras tends to be higher and higher, the number of cameras is increasing, and the functions are increasing. Traditional solutions are facing greater challenges. Blackfin processors have been maturely applied in automotive vision ADAS, and can provide mature and reliable high-performance video processing solutions, suitable for various systems from single function to multi-function. The launch of BF608/9 provides engineers with newer options. "Although automobile manufacturers have unified goals for ADAS, different requirements for accuracy, sensitivity, etc. lead to different algorithm standards for each customer." Lu Lei pointed out, "There are many ADAS solutions on the market, but BF608/9 has unique advantages in implementing flexible algorithms. Blackfin's software-based structure helps users launch distinctive terminal products, allowing customers to add their own algorithms." In addition to the Blackfin series of processors, ADI also provides a wide variety of video encoders/decoders, video multiplexers/switches and video amplifiers with excellent performance, while the optimized compiler and "Image Toolbox" algorithm library can maximize efficiency and MIPS.　　

 

　　⒊Barcode Image Processing

　　Barcode image processing faces unclear images caused by color loss, reflection, bending, etc. How to define a barcode in a large plane usually involves complex algorithms. In the past, some high-end barcode calculations were basically based on FPGA, but now the BF608/9 processor can eliminate noise and distortion through calculation, so design engineers can completely omit FPGA, helping to achieve low-cost design.

　　References:
　　[1]ADSP-BF606~609[R/OL].http://www.analog.com/static/imported-files/data_sheets/ADSP-BF6xx_ds_PrF.pdf
　　[2]Martin C., Gray L. Emulator and Evaluation Hardware Troubleshooting Guide for CCES Users[R/OL].(2012-9-28).http://www.analog.com/static/imported-files/application_notes/EE356v1.pdf
　　[3]Pellkofer A. UART Enhancements on ADSP-BF60x Blackfin Processors[R/OL].(2012-8-1).http://www.analog.com/static/imported-files/application_notes/EE354.pdf
　　[4]Blackfin enhances ThinkSmart V1 function, providing intelligent video security and surveillance [R/OL]. http://www.analog.com/zh/content/Blackfin_Powers_ThinkSmart_V1/fca.html
　　[5] Aglaia's "electronic eye" based on Blackfin processor warns drivers of lane changes Deviation, adjustment speed [R/OL]. http://www.analog.com/zh/processors-dsp/blackfin/products/customer-case-studies/aglaia_visual_sensor_systems/resources/fca.html