Principle types and development characteristics of panoramic cameras

    When a technology never attracts people's attention, but still survives, this is the so-called "light as a feather" survival! There is only one thing worse than being talked about, and that is no one talking about you. Did you hear that? Panoramic camera fellow! 

    In those days when I was young and naive, I naively thought that in addition to the "gun camera", there should be something called a "cannon camera" that can see the whole picture at a glance. Later, when I found the panoramic camera, I was impressed: So you are that! 

    Multi-lens panoramic view, useless?

    This type of panoramic camera has multiple sensors encapsulated inside, and a panoramic effect is achieved by stitching the split screens. In other words, it is similar to pointing multiple gun cameras in different directions and then stitching the images together. The core technologies such as digital processing and compression are integrated into the front-end firmware, and the four separate images are integrated into a 180° or 360° high-definition panoramic image according to user needs, and then transmitted to the back-end management platform via the network. Compared with the mainstream fisheye panoramic camera, the advantage of the multi-lens stitching panoramic camera is that it is free from the limitation of focal length to a certain extent, and can see farther under the same conditions. At the same time, it also has a virtual PTZ function, which can achieve functions such as circumferential cropping, intermediate restoration, and maintaining 180° expansion, so that users can adjust the image according to the focus of monitoring.

    This type of product has a rather tasteless pricing: higher than a fisheye panoramic camera, but less than directly installing multiple traditional cameras. Moreover, the stitching technology has a higher demand for the entire solution. The choice of field of view, how to perfectly match the structure, and how to seamlessly stitch the algorithm all determine whether such products can achieve better development in the future. Because the lens coordination cannot achieve physical perfection, it is inevitable that a conical blind spot will appear directly below during hoisting. After improvements, the pitch angle of the lens inside the camera is adjustable, and the impact of this blind spot can be reduced to a certain extent during installation according to the on-site conditions. On the other hand, when the installation height reaches more than three meters (the optimal installation height is three to five meters), the size of the blind spot generated by the camera is basically similar to the size of the human head, which actually does not have much impact on actual monitoring.

   Fisheye panoramic camera

    Fisheye panoramic cameras are the mainstream products among panoramic cameras, accounting for more than 70% to 80% of the total market share. A set of lenses and sensors can achieve the purpose of panoramic viewing. The hardware design complexity of the camera is low, and the reliability and service life are long. Among them, the fisheye lens is named because its front end bulges like a fish eye. The focal length of the fisheye lens used in panoramic cameras must be less than 2mm. At present, such lens products on the market are relatively mature. The reason why fisheye panoramic cameras have become high-end technology products is the difficulty of processing the original distortion pie chart and how to better provide humanized services for back-end monitoring personnel, that is, the realization of the two core technologies of image correction and virtual ptz. It can be seen from the above that these two technologies determine the development level of fisheye panoramic cameras.

    Specifically, as long as there is a fisheye lens, you can get a panoramic picture in the background, but this original image is a distorted circular pie chart, which does not conform to the normal observation habits of the human eye and must be corrected to achieve the final two-screen or four-screen monitoring needs. Virtual PTZ technology gives users a fast-ball monitoring experience - without mechanical parts, it can realize the key observation, image rotation and magnification of any monitoring point within the observation range.

    Core Algorithm

    The core image processing algorithm directly affects the system resource usage, bitrate, bandwidth and storage. The quality of the core algorithm directly affects the correction of image edge curvature, scene segmentation, virtual PTZ, rotation and zooming, all of which are driven by the core algorithm.

    The image formed by the fisheye lens is severely distorted due to the influence of the depth of field curvature, which does not conform to normal visual perception. Therefore, image correction technology is needed to eliminate the distortion of the fisheye lens. The idea of ​​image correction technology is not complicated. Generally, a separate sector is pulled out from the original pie chart, and then a certain deformation and correction processing is performed according to the principles of proportion and perspective. The images that meet the monitoring requirements can be obtained by splicing these separately processed sector images. For products with low requirements, the circular pie chart can be simply stretched at the four corners to obtain an image effect suitable for human eyes.

    Although the core algorithms of each manufacturer are bound to be different, the central idea of ​​image correction technology is to use a certain algorithm to stretch and integrate images with severe edge distortion into normal-proportion images suitable for viewing. The rationality of the algorithm used, the level of programming, and the final system resource occupancy rate jointly determine the pros and cons of this algorithm. As for virtual PTZ, it is actually to capture pixels inside the image to achieve functions such as magnification and rotation. Compared with the correction algorithm, its main difficulty is not how to achieve better picture effects, but how to better integrate it into the front-end firmware and back-end software at the same time to achieve two-way virtual control of the actual and post-recording.

    Fisheye lens

    The choice of lens will determine the performance upper limit of image quality. If the collected data is not clear, no matter how powerful the post-processing is, it cannot "fabricate" pixels. In terms of characteristics, the fisheye lens is a special ultra-wide-angle lens. The front lens of this photographic lens is of a large diameter and bulges toward the front of the lens in a parabolic shape. Its visual effect is similar to that of a fisheye observing the scenery on the water surface. Although the field of view provides unparalleled enjoyment, it is obtained at the expense of the original monitoring screen. The deformed image of the fisheye lens is generally called barrel distortion. This distortion will become more serious as the field of view expands. However, this is also a reasonable phenomenon. Li Jie, general manager of Hangzhou Puwei Optoelectronics Technology Co., Ltd., said: "The fisheye lens is designed according to the principle of refractive optics. It is an ultra-wide-angle lens composed of multiple refractive lenses. It can shoot more than 2π solid angle space, but the design of the field of view and angular resolution is restricted. The typical performance is that the target angular resolution in the central area close to the optical axis is very high, but the target angular resolution in the edge area far away from the optical axis is too low (the edge image is severely compressed)"

    Electronic PTZ

    Electronic PTZ is a highlight feature of IP-based panoramic cameras. Different from the PTZ control of other cameras, it can enlarge or move the image area within the monitoring field of view. When changing direction to observe another image area, it will not make any noise, which is stealthy and difficult to detect. The PTZ of panoramic cameras does not require the participation of mechanical parts, which improves the life and reliability of the product. At the same time, its technical characteristics can ensure that multiple PTZ windows can be previewed at the same time, subverting the understanding that PTZ on traditional cameras can only observe different areas in time.

    Panorama of Tragedy

    Favoring some resolutions over others

    A big advantage of panoramic cameras over ordinary cameras is their large monitoring range. However, from the perspective of resolution density, cameras with the same pixel will cause pixel dispersion and degradation when monitoring a larger area. This is because the monitoring range is very large, and the imaging chip of the same size as the traditional monitoring lens needs to receive several times the image information, which causes the image resolution to decrease. Therefore, it can only be used when the quality requirements for video monitoring images are not too high or when high-resolution imaging devices are used. This explains why the image quality of a 3-megapixel panoramic camera looks like CIF resolution. Therefore, for panoramic cameras, improving the image resolution is an important research topic in the future.

    Difficult to popularize low light

    Low illumination has always been a very important indicator in surveillance cameras. With the increasing demand for surveillance, people are paying more and more attention to the application of low illumination. However, the performance of panoramic cameras using high-pixel image sensors in low illumination is not satisfactory.

    Constraints of wide dynamic range

    The panoramic camera's all-round shooting angles solve the problem of shooting in various positions, but for functions such as wide dynamic range, white balance, exposure, etc. that need to be adjusted in special scenes, it is much more complicated to handle in a panoramic camera.

    It is precisely based on these factors that panoramic cameras have relatively stringent requirements on the lighting requirements of the monitoring environment and the installation position angle, etc.

   Fisheye panoramic + high-speed ball, a match made in heaven

    Panoramic cameras are generally high-definition cameras, and people have high expectations for their image quality. However, due to the short focal length of the fisheye panoramic camera lens, the effective monitoring range is greatly limited. People's faces can be seen clearly within a radius of about 5 meters, and it will not be clear enough if it is farther away. In other words, although panoramic cameras meet the needs of "monitoring without blind spots", due to the problem of lens focal length, it is difficult to meet the need of "seeing all details clearly", which is a bottleneck encountered in the promotion of panoramic cameras. High-speed ball cameras have large zoom and rotatable functions, and they do a very good job in presenting monitoring details, but a single ball camera is still not convenient and fast enough in terms of rapid positioning and determining direction.

    It is an "excellent" method to link a fisheye panoramic camera and a high-speed ball camera by combining software and hardware. Based on the large-scene and wide-viewing field monitoring of the panoramic camera, the high-speed ball camera improves the ability of detailed monitoring. It is like the superior unit grasping the events detected in the entire picture, and the high-speed ball camera is like a subordinate. After receiving the instructions from the superior, it immediately tracks and monitors the suspicious target and can enlarge the target image to a resolution of 1080P or above through the high-zoom lens. Allowing users to monitor the entire environment through the fisheye camera while also using the high-speed ball camera to capture high-definition images of key areas.

    Conclusion

    In short, panoramic cameras are like cannons, and are powerful tools for point-to-point monitoring. Although they still have weaknesses in the pursuit of details, they have incomparable advantages in controlling the overall situation.