A brief discussion on the technology of high-definition network cameras

HD cameras, as the name implies, require cameras with higher definition. Before starting the technical discussion, we naturally define the new star of surveillance, HD cameras. This leads to the topic of our discussion today, what is the core key of HD cameras, and what solutions can meet this core key demand and meet market demand.

As a collection terminal, the camera is the cornerstone of our video surveillance and an extension of our eyes. In the standard definition era, after more than 20 years of development, CCTV has developed to an extreme. Both the cost and effect of analog cameras have reached the best cost-effectiveness. However, in the high-definition era, due to the technological changes in the sensor industry and the increase in the amount of high-definition video data, new challenges have been raised for camera collection and transmission. This is also the core of our high-definition camera research and development and production: how to connect various high-definition sensors, debug the image quality that meets monitoring needs, and how to solve the video transmission problem of high-definition network cameras.

The first question is how to connect various high-definition sensors and debug the image quality to meet monitoring requirements.

This is the common key issue that all HD camera solutions need to solve. On the one hand, HD sensors have shifted from CCD technology to CMOS (this is determined by many factors such as process and cost). On the other hand, various CMOS sensor manufacturers do not provide dedicated ISP (also called DSP in the CCTV era), and ISP modules must be purchased separately from outside. This leads to relatively high technical requirements for developing and producing a HD camera and connecting sensor + ISP.

At present, many sensor companies (SONY, Panasonic, Aptina, Omnivision, Sharp, Altasens, etc.) have launched high-definition CMOS solutions. However, except for SONY which has its own ISP (ISP: Image Signal Processor), other companies are independent sensors that need to be configured with ISP externally. However, since high-definition surveillance has just become popular, there are few independent ISP solutions, and there is still a lack of flexibility in connecting various sensors, not to mention the negative factors such as cost and sensor board size. Therefore, a good ISP solution is the key to solving the problem of high-definition camera effects.

Wide dynamic range, color reproduction, noise reduction, low illumination and other factors are key factors affecting image quality. In different application environments such as smart transportation, finance, and safe cities, people have different requirements for high-definition cameras. The processing effect of CMOS sensor is somewhat different from that of CCD, so ISP processing is the guarantee for adaptability to various application scenarios.

When designing Hi3516, a dedicated chip for high-definition network cameras, HiSilicon decided to embed the ISP system in the chip, considering the final system cost, power consumption, effect debugging and other comprehensive factors. At the beginning of the design of this module, it worked closely with the original manufacturers of various sensors, matched the technical roadmap, and carried out targeted technical designs based on the characteristics of each sensor. Such as bad pixel correction, 2D/3D noise reduction, 3A processing, WDR, low illumination, edge enhancement, digital anti-shake, dynamic contrast enhancement and other image processing technologies. The application of these technologies corrects some shortcomings of the sensor and can effectively improve the high-definition effect. HiSilicon ISP has successfully connected with the high-definition sensors of the above-mentioned sensor manufacturers in advance, which greatly solves the customer's investment in this regard. In particular, the interfaces of various sensor companies are different. Matching in advance reduces the transfer of peripheral FPGAs, improves customer TTM and reduces customer BOM costs.

3A algorithm is a basic indicator for testing ISP. HiSilicon ISP not only provides customers with an appropriate 3A algorithm, but also provides a replaceable platform for companies with their own 3A processing algorithms. While lowering the threshold of the entire industry, it also provides differentiated protection for each company. After all, the needs of each market segment are different, and no algorithm can meet the needs of all markets. It is also a practical need for each company to adjust its 3A according to different manufacturers.

First of all, wide dynamic effect is the hotspot of most concern in various application scenarios, and it is also an important promotional focus of various sensors. HiSilicon's Hi3516 takes customers' needs into consideration and introduces two wide dynamic modes. One is a common algorithm for all sensors, and the other is an algorithm for sensors like aptina that have HDR-specific technology. The application of the two wide dynamic algorithms not only enhances the sensor selectivity, but also provides convenience for customers to pursue more differentiation.

Universal WDR algorithm off contrast (left - on, right - off)

Noise is a big taboo in video, especially the difference between CMOS technology and CCD technology itself leads to increased noise in low light conditions, which greatly affects the HD experience. HiSilicon ISP has two modules, 2D and 3D, which greatly improves the noise reduction process.

3D noise reduction switch comparison (left - on, right - off)

In addition to the ISP and sensor pairing algorithm, how to debug is actually the key. Due to the different layouts of the sensor board, the final effect of the sensor will be directly affected, so we need to debug the sensor and ISP. Since the sensor has thousands of registers and various combinations, it is very difficult to debug an effect that meets the needs. Considering the efficiency of the industry, HiSilicon provides an IPS TOOLS that can be used directly on a PC. This tool associates various effect adjustments with register combinations through various algorithms, avoiding the pain of customers adjusting registers right away. At the same time, on this basis, customers can debug registers based on their own experience, maintaining general scenarios while taking into account special applications. Therefore, the status of ISP and ISP tools is equally important.

In order to avoid advertising for HiSilicon, we will not elaborate on the effects of various algorithms one by one. We would like to share with our peers some of the precautions we have in selecting a good HD camera solution in terms of image quality. We also provide the algorithm effects of key ISPs and ISP debugging tools.
The second question is how to transmit HD.

There are two technical propositions on the market: SDI and IP. SDI technology originally belongs to the technology of broadcasting applications. Its advantages and disadvantages are very obvious, and this application is similar to the original CCTV. We will not discuss it in detail today. This article mainly explains the IP solution.

From megapixels to Full HD (1080P), network cameras have changed the dilemma of "good reviews but not sales" many years ago. Today, network cameras have benefited from the rapid development of the Internet and the maturity of encoding and decoding technology, and have ushered in the best opportunity for rapid development. After the front-end IP is changed, the original analog camera can only passively receive front-end data and it is difficult to interact in two directions (let's consider controlling the PTZ through the serial port as a simple interaction). The increase in interaction between the front-end and the back-end can better play the role of our distant "eyes".

The key to IP cameras is front-end video digitization and encoding. We put encoding at the front end, which not only greatly reduces transmission costs, but also solves the bottleneck problem of high-definition transmission, and provides a foundation for more interaction and intelligence. At the same time, it also greatly reduces the integrated high-definition encoding requirements of the back-end DVR.

Coding and Network Processing

Due to the limitation of display bandwidth and the cost of use of customers, although the HD experience is good, customers cannot get the HD experience at an unlimited cost, so the coding efficiency of HD cameras is required. Since Hi3510, HiSilicon has been committed to the research of video compression technology dedicated to surveillance, using standard H.264 and fully considering the video delay and network packaging convenience (for example, some tools of H.264 are used for broadcast-level video, and broadcast video does not need to consider the delay problem, but surveillance cannot be used directly in this way, and needs to be optimized in implementation). Hi3512 has launched a 2M 720P real-time HD solution. Hi3516 inherits the video advantages of HiSilicon's previous three generations of products in the H.264 algorithm, and upgrades H.264 from baseline to high profile level 5.0, which further improves the compression ratio. At the same time, it is backward compatible with Main profile and baseline, and is also compatible with compression algorithms such as MPEG4 and MJPEG, maximizing the compatibility of HD cameras.

Since network cameras run on IP networks, different implementation methods for network transmission will directly affect the user's visual experience. The encoding characteristics of unprocessed video are that the encoding bit stream size will change depending on the amount of video motion. Such bit streams are not a big problem in DVR recording applications, but in the network, because too large data packets mean network congestion, the internal control of the encoder needs to be very strict, and this control cannot affect the image quality.

HiSilicon has made use of its years of experience in the network field and has gradually applied its comparative advantages to its products. With the development of high-definition network cameras, the encoding requirements for the core chips are becoming more and more stringent, requiring a higher compression ratio to achieve a scene closer to reality. Especially with the rise of 3G and LTE wireless broadband networks, adaptability to various networks is an important development direction for chip manufacturers.

At the same time, the network is still a challenge for many engineering companies. The end-to-end solution of the plug-and-play function of network cameras and back-end NVR is also a contribution made by HiSilicon based on its own network technology to provide the industry with a better experience.

Real-time preview output

Traditional security personnel will have a headache when they get a network camera. It is really difficult to use. Every time they pick it up, they can't remember the network IP (the default IP attached to the body? Or was it changed during the construction process?). They also need to carry a portable device to repair the device. How can contractors like to use network products? Will they have to find engineers with "Huawei University" network certification to do wiring in the future?

Hi3516 has built-in CVBS output, which allows customers to use traditional analog monitors to view the internal parameters of the network camera and the lens effect lights of the network camera, just like using analog cameras, to facilitate engineering construction. We can simply and intuitively imagine that the construction workers are still digging trenches and wiring, the only difference is that there are fewer buried wires, and even WIFI is wireless. The workers still arrange and debug the cameras as before, regardless of whether they are IP or not. The only difference is that the analog video cable is changed to a Category 5 network cable when wiring.

At the same time, in order to meet the special requirements of public security and some other places that require lossless real-time images, such as command centers, Hi3516 also provides a BT1120 high-definition video digital port, which can complete various conversions such as HDMI, VGA, SDI, etc., and allow the camera to directly transmit unencoded high-definition video back to the public security command site, so that storage and preview can be carried out simultaneously.

Therefore, for network camera chips, the fundamental thing is to quickly meet customer needs at the current engineering level. We cannot transfer the savings in network transmission costs to the labor costs of construction workers. The second figure below is a schematic diagram of the Hi3515 structure and external connections. Readers may wish to browse it to deepen their understanding.
Front-end and back-end interaction, intelligence

The biggest benefit of front-end digitization is that it improves the original analog camera which only transmits data without paying attention to the needs of the back-end. For example, if you want to change the parameters, you can do it. If the lens is out of focus, the analog device will only take care of the out-of-focus content and will not tell the back-end device that the video is abnormal. After the front-end is IP-based, these problems can be greatly solved.

For example, Hi3516 integrates an intelligent acceleration module, providing everyone with an intelligent platform. At the same time, it combines with well-known domestic and foreign intelligent manufacturers to provide simple and practical intelligence for customers.

1. Moving objects crossing boundaries? Wandering? Going in the opposite direction? Invading an area? Leaving an area? Do you think these scenarios are very real in reality? Isn’t monitoring just monitoring suspicious moving objects (people)? HiSilicon platform makes these applications just draw a line, draw a box, and the rest is confirmed. Is it the same as the usage habits of motion detection? Yes, HiSilicon’s intelligent platform is to solve the problem that various previous intelligences still require engineers to go to the site to match the angle of the lens, installation direction and other requirements, liberate engineers, and let customers apply it at any time according to their own needs. Make 1. Intelligence simple and easy to use intelligence.

2. Face detection function: With such high-definition images, all monitored faces can be detected, and then captured in close-up. Based on the photos, they can be associated with the videos. In the future, all videos can be simply associated with a certain person. This not only provides high-definition photos for the platform's face recognition, but also provides convenience for monitoring management.

At the same time, face detection is combined with some statistical functions, such as the number of faces in each area, to achieve the concept of counting heads. Although the accuracy is still somewhat different from other professional algorithms, in some squares, the number of people in the circled area is clear, and in such a scene, there is actually no need to worry about whether it is 78 or 82 people. We know that the order of magnitude of about 80 people is enough. There are many combinations of such applications in actual development, which can be developed by various manufacturers. HiSilicon only provides the basic platform.

3. Video diagnosis: A small police station has to manage hundreds of cameras, which is too much to keep track of, let alone check their working status at all times. Hi3516 intelligent acceleration provides video diagnosis function operators, including dirty lens, out-of-focus, blocked, fogged, abnormal video color, lost video and other 12 diagnostic functions. You no longer have to worry about whether the hundreds of cameras you manage are normal. You can always find out whether the cameras are working properly. You can ask staff to perform maintenance at any time, reducing the possibility of video loss when a case occurs.

Through such an intelligent combination, the front-end and back-end are fully interactive. The back-end can control and modulate the front-end at will, and the front-end can pass various useful information to the back-end, achieving the real role of personnel.


The last question is back-end management, storage, management and display

This is a problem that all high-definition cameras are involved in. Here we only introduce the back-end solution for network cameras.

As mentioned above, network cameras can be arranged and debugged just like analog cameras. This should change the impression that network cameras are difficult to use, right? In order to achieve the matching of this system, only the front end is not enough, so HiSilicon also has a chip for the matching NVR management platform. Both the current Hi3520 and the soon-to-be-launched Hi3531 are designed to solve this problem.

If there is no large platform and you just need a plug-and-play application like Camera+DVR, then you will need to use NVR or the most popular hybrid DVR to solve the problem.

The network HD solution cannot be simply regarded as a network camera solution but an end-to-end overall solution. The key is the combination of application experience and reality, and the improvement of supporting products.

Due to limited space, we cannot further describe the development trends of other aspects of HD cameras. The biggest advantage of HD cameras is high definition, and HD network cameras have the advantages of front-end and back-end interaction, intelligence, and low transmission costs.

The NVR issue is mentioned here just to remind readers to systematically consider the backend solution when choosing the frontend. However, due to the limited space, we will not discuss the reality of NVR for the time being. We will write another article later. (The author of this article is Shenzhen HiSilicon Semiconductor Co., Ltd.)