Smart camera hardware architecture and algorithm type application

The term "smart camera" has a long history. From the earliest analog cameras, analog dome cameras, standard definition network cameras to today's high-definition cameras, they are all called smart cameras. There is no unified standard for the definition of "smart". For example, supporting adjustable CCD parameters, automatic white balance, support for motion detection, support for video occlusion detection, and support for perimeter protection can all be called smart. The above factors have caused a large variety of smart cameras on the market, making it difficult for users to distinguish them, like seeing flowers in the fog.

However, with the development of high-definition video surveillance and the increasing maturity of market applications in the past two years, the industry has begun to define smart cameras clearly, that is, high-definition cameras with intelligent video analysis functions. This concept has gradually been recognized by manufacturers and users. The hardware architecture of smart cameras is relatively simple, but they can be divided into many categories based on the differences in the embedded intelligent analysis algorithms and output interfaces.

Smart Camera Hardware Architecture

Compared with traditional analog cameras or network cameras, the biggest feature of smart cameras is that they have intelligent analysis functions. Therefore, there is usually an additional CPU chip in the hardware - DSP, ARM or FPGA. This chip is responsible for the calculation of the intelligent analysis algorithm inside the smart camera.

SENSOR and ISP part

SENSOR generally uses CCD or CMOS. At present, most of the smart cameras used for monitoring use CCD chips with more than 2 million pixels because of their critical application areas and relatively advanced technology. However, with the maturity of CMOS technology, there is a trend towards CMOS. The ISP part mainly refers to the post-processing of CCD signals, such as the adjustment of white balance and exposure time, which can be divided into two situations: Traditionally, an independent DSP is generally used to complete it, but in recent years, some chip manufacturers have integrated this part of the function into the video compression ASIC chip to improve the integration.

Intelligent analysis and computing CPU

The main function of this part is to calculate the intelligent analysis algorithm embedded in the camera, and it is mainly implemented by FPGA and DSP. At present, both solutions are used by manufacturers. In comparison, the latter is more widely used and has higher user recognition. It is reported that some video compression chip manufacturers have integrated it into video ASIC, using the increasingly powerful ARM chip in ASIC chips to implement the analysis function.

Video compression part

This part of technology is very mature at present, and it can be divided into three categories according to the implementation method: the first is to use FPGA to implement, but the cost is high and the difficulty is great, and it is rarely seen now; the second is to use DSP to implement, which is very mature, but due to the cost and development difficulty, it has gradually been replaced by ASIC; the third is ASIC, chip manufacturers will solidify the video compression algorithm and basic functional API interface inside the ASIC, which greatly reduces the difficulty of terminal manufacturers to develop. At the same time, due to the realization of dedicated chips and batch tape-out, the cost of chips is also greatly reduced, laying a good foundation for the popularization of network cameras. The performance of current ASIC chips is already quite strong. A single chip can already support 16 D1s, and compressing a standard 1080P video is not a problem. In addition, ASIC chips can now integrate more and more functions. The ISP and intelligent analysis CPU mentioned above can now be integrated into an ASIC chip. Smart camera solutions using video compression chips generally output network interfaces.

SDI conversion output section

This part is generally optional. If the SDI conversion part is selected, the above-mentioned video compression part is no longer needed. The video data processed and analyzed by the ISP and CPU can be converted into a standard HD-SDI signal through the SDI conversion chip and output to related transmission or terminal equipment.

Several types and applications of intelligent analysis algorithms

Intelligent analysis is the core function of smart cameras. Its functions and performance are completely dependent on its embedded intelligent analysis algorithm. According to the current industry development trend and market demand, we can roughly divide the currently used intelligent analysis algorithms into the following types:

License plate recognition function

At present, smart cameras that support license plate recognition are mainly used in the transportation field, such as red light running, checkpoints, parking lots, etc. Compared with the previous traditional camera and processing host system, the smart camera adopts an integrated design, which reduces the cost on the one hand, and reduces the system connection and the failure rate on the other hand. Smart cameras with built-in license plate recognition function are the trend of future transportation applications.

Traffic behavior analysis

It mainly refers to the detection and analysis of traffic violations other than running red lights, such as illegal parking, crossing yellow lines, driving in the wrong direction, driving in the wrong lane, driving left or right, etc. The previous system was limited by system performance and basically could not achieve the above functions. Due to the improvement of CPU chip performance and the use of high-definition cameras with a larger field of view, smart cameras can now achieve the above functions more perfectly.

Basic behavioral analysis

Basic behavioral analysis mainly includes tripwires, perimeter protection, items left behind, items lost, crowds of people, etc. The technology in this area is relatively mature. These functions have existed since intelligent analysis entered the security field. Therefore, these algorithms are currently widely used.

Video Quality Diagnosis

Video quality diagnosis refers to the monitoring of the camera's own image quality, such as snow, streaks, video loss, etc. Since the algorithm mainly plays a self-test role, it is usually used in conjunction with other functions.

Traffic counting statistics

The counting function is mainly divided into people counting and vehicle counting. People counting is mainly used for entrance and exit control or statistics, such as subways, airports, railway stations, supermarkets, etc. Vehicle counting is mainly used for checkpoints, traffic density monitoring, etc.

Face recognition function

The face recognition function is divided into face detection and face comparison. The latter needs to be implemented based on the former. The former is mainly used to assist counting and target detection, while the latter is mainly used for personnel identification to achieve access control, attendance, important person identification, etc.

Differences in external output interfaces

According to the application scenarios, the interfaces of smart cameras are divided into network interfaces and HD-SDI interfaces. At present, most smart cameras use network interfaces for transmission, and the transmission protocol uses the standard TCP/IP protocol. The difference is that some smart cameras use network interfaces to transmit pictures, while some smart cameras use network interfaces to transmit compressed video streams. At present, there is a trend of integration between these two types of cameras, that is, one camera supports multiple code streams, which can transmit both pictures and video streams. The advantages of network cameras are standard protocols, easy deployment, and relatively low requirements for transmission media.

The HD-SDI interface is an emerging demand. Since the network interface has inevitable image quality loss and transmission delay, in some specific occasions, smart cameras with HD-SDI interfaces can be used. The HD-SDI interface has the characteristics of excellent image quality and low transmission delay, but the HD-SDI has stricter transmission requirements - coaxial cables are used for short distances, while optical fibers must be used for long distances. Compared with the network interface, the cost of HD-SDI interface equipment when building the overall system is much higher.

Conclusion

With the rapid development of high definition and intelligence, high-definition smart cameras with combined advantages will inevitably become the representative products of future video surveillance. I believe that through the joint efforts of colleagues in the industry, the technology and market of smart cameras will achieve qualitative improvements, and smart cameras will become the main force of front-end equipment in the future monitoring market.