Analysis of chip technology related to megapixel HD cameras

As early as 2008, high-definition video surveillance was proposed by the industry and once became a hot topic. However, due to various reasons such as technology, cost and actual customer needs at that time, the market for high-definition video surveillance did not actually start. However, from now on, high-definition video surveillance is no longer just a slogan. Various manufacturers have begun to take it as the key product technology development direction and also as the focus of market promotion. Tiandi Weiye has also grasped the trend of the industry. While integrating cutting-edge technology, it has launched high-definition video surveillance solutions for various industries and fields in combination with mature project applications. As a typical representative of modern security technology, the new generation of video surveillance technology that combines the three characteristics of high definition, networking and intelligence is gaining more and more attention and practical application. It will gradually replace traditional video surveillance technology and play an irreplaceable role in the social security monitoring system.

Faced with an increasingly large market, the increase in demand will bring about the need for higher performance and functions. The innovation of the core chip technology and the improvement of the performance of HD camera products will play a decisive role in the status of the HD video surveillance market. First, let's briefly analyze the common component architecture of current HD camera products. Common HD cameras are divided into three common parts: image sensor, ISP (image signal processor), compression processor or digital video converter. Some newly released HD cameras on the market will also include an intelligent analysis processor between the ISP and the compression processor.

Image Sensor

Image sensors are an important part of digital cameras. According to the different components, they can be divided into two categories: CCD and CMOS. Such as SONY's ICX445, ICX274, IMX035/036, Aptina's MT9M033, MT9D131, MT9T031, Omnivison's OV2715, etc. The function of the image sensor is to collect optical images and convert them into electrical signals that can be used by the back-end ISP.

In the past, we often said that CCD was a high-end technical component used in imaging products, while CMOS was used in low-quality imaging products due to its limited technical performance such as low-light performance and exposure mode. Today, CMOS technology is developing rapidly. For example, SONY has launched the IMX035/036 image sensor using Exmor technology, which has greatly improved its low-light performance, image resolution and frame rate, and chip processing speed.

ISP Image Signal Processor

The main function of the ISP (Image Signal Processor) image signal processor is to perform post-processing on the signal output by the front-end image sensor. Different ISPs are used to match image sensors from different manufacturers. The excellence of the ISP is very important in the entire camera product, and it should be said that it directly affects the quality of the image presented to the user. After the image is collected by CCD or CMOS, it needs to be processed in the later stage to better adapt to different environments and restore the details of the scene under different optical conditions. In the ISP, it will complete the 2A (AWB/AE, automatic white balance/automatic exposure) or 3A (AWB/AE/AF, automatic white balance/automatic exposure/automatic focus) that we often mention. In the traditional mode, a DSP or an FPGA is generally used to complete the post-processing of the image. Some camera products support 3D noise reduction, wide dynamic, slow shutter, frame accumulation, strong light suppression and other functions are also completed by ISP.

The ISP currently used in high-definition camera products generally comes from the following sources:

Self-developed by manufacturers: In order to better cooperate with back-end compression and function development, high-definition camera equipment manufacturers develop ISP processing algorithms on their own, integrate the algorithms into FPGA or DSP chips, and connect them to front-end image sensors.

Third-party R&D: In 2010, a number of ISP solutions were launched by non-HD camera manufacturers. They directly sold different ISP chips to camera manufacturers to cooperate with sensors from different manufacturers.

Package mode: Sensor manufacturers combine their self-developed ISP with their own sensors to form image acquisition and processing solutions and push them to customers. The image processing algorithms and various debugging work have been completed. Camera manufacturers only need to do interface docking and back-end compression or conversion to digital video (HD-SDI). We call this mode Stand-Alone Devices or Camera System On Chip.

The first source method has high flexibility, allowing developers to have more combinations when choosing solutions (mainly choosing sensors from different manufacturers). However, for the product, on the one hand, it increases the power consumption and heat dissipation of the whole machine, and on the other hand, it also increases the investment in product development, which also means that manufacturers who develop high-definition camera products need stronger technical strength. The latter two source solutions undoubtedly greatly reduce the threshold for product development, but relative to independent research and development of ISP, some costs will definitely increase. Imagine if there is a more integrated high-definition camera chip solution, integrating ISP and video compression encoding functions together is undoubtedly a better choice.

Compression processor or digital video converter

First, let's understand the concepts of compression processing and digital video conversion. After the ISP processes the front-end image sensor signal, it is generally transmitted to the back-end processing chip in digital interface formats such as MPTE296M/BT.1120/YUV4:2:2. The compression processor (SoC or DSP) can encode and compress the video signal using the H.264/MPEG-4/MJPEG algorithm, and then transmit it to the user through the built-in network service using the TCP/IP protocol network signal. Digital video conversion is a non-compressed method. The digital signal output by the ISP is converted into SD-SDI/HD-SDI/3G-SDI signals through a digital video conversion chip, corresponding to standard definition (VGA), high-definition standard ( 1080P@30fps ), 1080P@60fps or even higher resolutions. HD-SDI high-definition video signals require the use of dedicated transmission peripheral equipment, such as SDI optical transceivers, SDI matrices, etc.

Here, we focus on the compression processor. When we see the parameters and performance of a high-definition camera product, we often see the word SoC. There are many definitions of SoC (System On Chip). Due to its wide range of applications and rich connotations, it is difficult to give a specific definition. Generally speaking, we call it a system-level chip, also known as a system on a chip, which means it is a system, a dedicated integrated circuit that contains a complete high-definition camera system and embedded software services.

Common manufacturers of SoC in HD cameras include Ti (Texas Instruments), Ambarella, Hislicon, NXP, etc. Hardware compression processors from different manufacturers have their own characteristics, such as low power consumption, high performance, high video compression quality, high compression ratio, etc. We mentioned the concept of integrating ISP image signal processing with back-end video compression in the previous article. Traditionally, the several important components of HD cameras introduced in the previous article are separated from each other, and different processing tasks are completed by different chips. With the development of SoC chip technology and the general trend of the market, more and more manufacturers are integrating ISP with video compression processing and even intelligent analysis technology, and solidifying the functions in one chip.

NXP ASC8850 and Hislicon Hi3516 used in high-definition cameras integrate 3A, 3D noise reduction, wide dynamic range (WDR), strong light suppression, backlight compensation, digital image stabilization and other image processing functions implemented in ISP. The Hi3516 chip also integrates an intelligent analysis acceleration engine to support intelligent motion detection, perimeter protection, face detection, lens protection and other analysis applications. In addition, these SoC chips also open ISP-related APIs (program interfaces) to us manufacturers, allowing us to develop and optimize the secondary adjustment of ISP effects. These chip technologies make the integration of high-definition cameras higher and higher, and the integration of chips will be a trend.

The author believes that the development of high-definition camera SoC chips will seek development by making technological breakthroughs in power consumption, encoding performance, ISP, and intelligent analysis, which will drive the development of high-definition camera products.

Lower chip power consumption can make HD cameras smaller in size, applicable to harsher environments, and lead to a variety of HD camera product forms.

Higher encoding performance can enable high-definition cameras to have higher pixel real-time compression capabilities. Most high-definition chips such as TI's DM368, HiSilicon's Hi3515/3516, NXP's ASC8850, etc. can achieve 1080P@30fps . 3-megapixel and 5-megapixel level cannot achieve real-time compression. Higher-pixel high-definition cameras can replace our traditional multi-area monitoring mode of multiple machines at one point, which can effectively reduce equipment investment costs and peripheral wiring costs.

Currently, most video chips with built-in ISP functions have relatively complete support for CMOS chips and can be directly connected to the main products of major sensor manufacturers. However, support for CCD chips is relatively limited and developers are required to perform secondary conversion.

The current high-definition cameras with intelligent analysis functions have a DSP or ARM core with a higher main frequency reserved in the device. Developers can choose to transplant part or all of the functions in the intelligent analysis algorithm to the DSP or ARM to form an intelligent high-definition camera with intelligent analysis.

At present, the major SoC manufacturers in the industry have realized the integration trend of main control chips and have invested heavily in R&D to develop a new generation of high-definition SoC chips. Whether it is NXP, which was the first to launch ASC8850, TI, which launched DM368, or HiSilicon Semiconductor, which newly released Hi3516, they have all begun to make achievements in highly integrated high-definition camera main control chips. At the same time, some influential IC manufacturers are also eager to try. It is believed that with the efforts and promotion of chip manufacturers, combined with the market promotion of downstream equipment manufacturers and the pull of user demand, the highly integrated new generation of high-definition camera SoC chips will achieve greater and greater development and will gradually become a mature industry.

Conclusion

The chip technology related to high-definition cameras is developing rapidly, and the market's performance requirements for high-definition surveillance products are also constantly increasing. The general 720P resolution is already a bit stretched, and the demand for 2-megapixel, 3-megapixel and even 5-megapixel high-definition cameras is constantly emerging. These demands have posed greater challenges to the performance of related chips for high-definition cameras. To truly realize high-definition video surveillance in all aspects, it is bound to support high-definition video surveillance from the high-definition of front-end products to network transmission, centralized storage, display control, platform management and other aspects.