Evaluating the intelligence of monitoring products with intelligent analysis technology

With the continuous development of modern information society, people's demand for security is increasing, making video surveillance systems more and more widely used in the field of security prevention, so it is an important technical means to implement security monitoring. However, the video surveillance systems currently used lack intelligent analysis of video content and mainly rely on human vision, so that the video surveillance system can only complete the video storage record of time and only provide evidence for post-analysis. In this way, it cannot provide pre-warning/alarm in advance and cannot really play the role of ensuring safety.

With the development of high-tech science and technology such as optoelectronics, video surveillance systems are also moving from digitalization and networking to high-definition and intelligent development. The so-called intelligent video surveillance system means that the system can automatically detect, analyze and identify abnormal situations in the monitoring image without human intervention, and make early warnings/alarms in time. Since intelligent video surveillance systems can better meet people's needs for security, security monitoring companies with vision and R&D capabilities at home and abroad have invested in research and development. Now, intelligent video surveillance products have been launched in the domestic and foreign markets.

As foreign monitoring product suppliers continue to introduce intelligent products to China, and as domestic intelligent video monitoring products continue to emerge, are products with intelligent video analysis functions considered intelligent products? Can their intelligent video analysis and recognition work? Under what circumstances does it work? Can it meet our different levels of security needs in different occasions? Can it provide people with the greatest security benefits and truly ensure safety? What is considered an intelligent video monitoring system? What is the relationship between intelligent video monitoring products and intelligent video analysis technology? How can security engineering companies choose practical intelligent video monitoring products? These are all issues that we are extremely concerned about.

As we all know, after receiving the security monitoring system project, the security engineering contractor will select it according to the complete set of instructions provided by the product supplier. However, in line with the principle of pursuing the greatest economic benefits, the product supplier often unrealistically exaggerates its products in the instructions, and generally sells the most profitable products. The security engineering contractor does not understand the intelligence of the new products, nor is it clear how much security benefits it can achieve. As a result, many system equipment put into use cannot meet the actual security needs, and some even become a kind of decoration during the construction and use process, which is tantamount to nothing. This is undoubtedly a huge waste of the user's investment.

If users can conduct intelligent evaluation of the video surveillance products they use, or if these products have been evaluated by experts from authoritative evaluation and appraisal departments, the effect will be completely different. In this way, security engineering companies can not only select the most cost-effective and appropriate intelligent products according to the security needs of users to maximize investment cost savings. Moreover, they can also make full use of their intelligent functions to prompt monitoring personnel to pay attention to relevant monitoring images before security threats occur, and prepare for potential threats, thereby maximizing the satisfaction of users' security needs and building the best performance system. In this way, users will not blindly invest a lot of money in pursuit of high-end products, causing great waste, but will rationally make reasonable choices based on their own security needs in certain aspects, saving investment costs on the one hand and meeting higher-level security needs on the other.

Therefore, it is very necessary to conduct an intelligent evaluation of the role of video surveillance products and their intelligent video analysis technology. It can not only evaluate the effectiveness of the product's intelligent video analysis and recognition, but also importantly evaluate its level and grade of intelligence, for reference by security engineers and users. At present, the evaluation work in the security industry is progressing slowly, and there is still a lack of a complete set of standardized evaluation methods and steps for the evaluation of security systems. This article discusses the intelligent video analysis technology of intelligent video surveillance products and its role, the relationship between the level of intelligent video surveillance products and the role of intelligent video analysis technology, how to evaluate the role of intelligent video analysis technology and the methods and steps of evaluation, etc., for reference by security engineers and users.

Intelligent video analysis technology and its role

Video analysis technology is a technology that uses computers to extract useful information from videos through calculation and analysis. For humans, we can intelligently extract the required target information by observing video images. But for computers, it is just a data series, a series containing the grayscale value or color value of each pixel in each frame. Its target information is contained in the plane image sequence composed of the values ​​of those pixel points, and it needs to be "understood" from the "whole" to obtain. Video analysis technology is to allow computers to extract content information or individual motion information contained in video signals through specific core algorithm programs, so as to realize the computer's understanding of the video, so that the computer can "understand" what is shown in the video or what kind of thing happened, that is, "event".

Video analysis technology covers a wide range. As long as it extracts useful information from the video through computational processing, it can be called video analysis technology. Since they all belong to the "analysis, identification and understanding" of video content, the technologies that are relatively mature and have formed products and are applied in actual projects, such as license plate recognition technology, video retrieval technology, video face detection, etc., all belong to "video analysis technology". Because they all belong to the extraction of useful information from the video, such as extracting license plate numbers, extracting text or specific graphics from the video, etc. Strictly speaking, these technologies are just relatively simple contents in video analysis technology, but because these technologies have been well studied and applied, they have some special titles and are mentioned separately. At present, the commonly referred to video analysis technology generally refers to the analysis, extraction and identification of target motion behavior from the video. It is often used in the field of security monitoring, and its scope of reference has been greatly reduced compared to the literal meaning.

The role of video analysis technology is to let the computer know "what happened" in the video, and then compare and judge with the corresponding rules, so that the computer can naturally know the characteristics of these events. If we look at it from the perspective of analyzing, extracting and identifying individual movement behaviors in the video, the computer can determine what behaviors these individuals have performed, and then determine whether these behaviors conform to certain rules and whether they belong to "a certain type" of behavior. In this way, events that do not conform to the rules can be discovered and alarmed immediately, so as to get rid of manual intervention and judgment, and make the computer "replace" people to monitor, that is, to achieve automatic monitoring or "intelligent monitoring". A more vivid explanation is that the camera and video transmission technology in the video monitoring system solve the problem of "eyes" (retinal imaging and optic nerves), so that the monitoring personnel can see the scene on the scene through the camera without being on the scene. And this scene has also gotten rid of geographical restrictions due to the advancement of transmission technology, and can even be seen through digital network transmission thousands of miles away; and video analysis technology adds a "brain" to the video monitoring system, so that the security monitoring system can at least replace people to monitor these videos at any time to a certain extent. In this way, there is no need for humans to monitor these videos at any time.

In fact, intelligent video analysis technology originated from computer vision technology. Computer vision is one of the branches of artificial intelligence research. It can establish a mapping relationship between images and image descriptions, so that it can understand the content of the video through digital image processing and analysis. Computer vision technology can be divided into low-level vision technology and high-level vision technology. Low-level vision technology needs to solve basic problems such as feature extraction, motion detection, and multi-target root tracking; high-level vision technology needs to solve the semantic understanding of the scene, such as the determination of target motion trajectory, the final conclusion of abnormal behavior and abnormal events, etc. Therefore, in the field of computer vision, one of the most fundamental problems is how to obtain high-level semantic understanding from the underlying raw video data, and the research on intelligent video analysis technology in intelligent video surveillance technology is closely centered on this basic problem. Its research scope is not limited to a specific problem, but involves many basic problems from the bottom to the top in computer vision.

Intelligent video surveillance technology mainly analyzes videos automatically, extracts key information from videos, and discovers and identifies abnormal events of interest, thereby replacing or assisting human monitoring. Obviously, video analysis and recognition involve complex software algorithms that can be programmed to identify strange and abnormal behaviors. Video content analysis and recognition software can detect and identify suspicious activities, events, or behavior patterns by analyzing live or recorded video streams. The intelligence of video surveillance systems means that the system can automatically detect and identify abnormal situations in the monitoring screen without human intervention, and make warnings/alarms in a timely manner.

Video analysis technology usually uses background subtraction technology to detect image changes (such as intrusion, packet loss, reverse traffic, etc. are all a pattern of image changes), that is, the video frame is compared with the baseline background image, and the pixel (area) changes in the same position are considered to be the changed area. These areas are further processed, identified, and tracked to obtain basic morphological information and dynamic information including target position, size, shape, speed, and residence time. After completing the tracking of the target and understanding the behavior, the mapping relationship between the image and the image description is completed, so that the system can further perform rule judgment until the alarm is triggered.

At present, the commonly used method for detecting moving targets is background subtraction. Although its algorithm is relatively simple, it is still affected by natural conditions such as light and weather, as well as changes in the background itself (such as waves, cloud shadows, and shaking leaves). In response to different weather and natural interferences, a variety of additional algorithms (filters) have been applied to compensate for these interferences. With the development of optoelectronic image processing technology and microelectronics technology, it is believed that intelligent video analysis technology will become more mature.

Broadly speaking, we can also divide intelligent video analysis technology into the following three categories:

Video Analysis

This category mainly finds the target in the surveillance image and detects the target's motion characteristics, such as the relative pixel position of the target, the target's moving direction and relative pixel moving speed, the shape of the target itself in the picture and its changes. Based on the above basic functions, video analysis can be divided into the following functional modules:

①Perimeter intrusion detection and target movement direction detection;

② Detection of target movement and stop state changes;

③Target appearance and disappearance detection;

④ Statistics of pedestrian and vehicle flow;

⑤PTZ automatic tracking system;

⑥System intelligent self-check function, etc.

Video Recognition

This category includes face recognition, gait recognition and license plate recognition. Its main technology is to find the commonality of some parts of the video image. For example, a face must have two eyes. If we can find the position of both eyes, we can determine the position and size of the face. However, with the existing technology, the face recognition system must be able to perform face comparison when both eyes are visible. It mainly includes:

①Face recognition system;

②Gait recognition system;

③License plate recognition system;

④Photo comparison system;

⑤ Parts recognition in industrial automation, i.e. machine vision systems, etc.

Video Improvement

This category mainly optimizes some invisible, blurred, or vibrating images to increase the monitoring performance of the video. Specifically includes:

①Infrared night vision image enhancement processing;

② Deblurring of license plate recognition images;

③Light variation and shadow suppression processing;

④ Tidal and object size filtering;

⑤Video image stabilization system, etc.

The relationship between intelligent video surveillance systems or products and intelligent video analysis technology

The evaluation work of the security industry started late. At present, the evaluation in this field is mainly based on the evaluation methods in the field of safe production. For example, the expert scoring method, weighted average method, hierarchical analysis method, gray system method, or some foreign performance evaluation methods are used to evaluate the security system. However, the intelligent evaluation of the video surveillance system is crucial to whether the intelligent video analysis and recognition technology of the system can work and whether it can truly play a safety benefit. Therefore, it is necessary to study an effective method for intelligent evaluation of intelligent video surveillance systems or products and intelligent video analysis technology.

[page]

As one of the world's leading intelligent surveillance suppliers, iOmniscient, an Australian company, has established an intelligent IQ (Intelligence Quotient) rating system for video surveillance systems through discussion, imitating the classification standards of human intelligence levels. This intelligent IQ rating system is also a good attempt to establish a video surveillance system evaluation method. Some people in China have followed this idea and proposed to establish an intelligent evaluation method for video surveillance systems, namely the intelligent IQ rating method. The intelligent IQ rating method sets the intelligent IQ of the system between 60 and 180, and divides it into 7 levels according to the detection ability of the system. However, this method is not in line with the habits of Chinese people, because we Chinese are accustomed to the percentage system or the 5-level scoring system. Below, I will refer to the intelligent IQ rating method to propose a practical 5-level scoring intelligent evaluation method suitable for my country, for reference by the law makers of the intelligent evaluation method of video surveillance systems.

The 5-level scoring intelligent assessment method is divided into the following levels:

First level range of products

The intelligent IQ of products or systems in this level range is 1 point, or less than 50 points, which means that the intelligence level is extremely poor. Generally, video motion detection VMD is used to detect movement in the monitoring scene. That is, by comparing the pixels of one frame of image with the pixels of the next frame of image, it constitutes the simplest form of video motion detection. If there is a slight difference, it means that there are some changes in the monitoring scene, which is usually called movement. Obviously, a system that can only detect video movement is not very intelligent and is prone to false alarms.

Because in fact some interference, such as light changes, water reflections, shadows or other changes, can make the system think there is movement and send out false alarms. Therefore, for products in this level range, the role of intelligent video analysis is extremely poor, so this product is not practical at all.

Second level range of products

The intelligent IQ of products or systems in this level range is 2 points, or greater than or equal to 50 and less than 60 points, that is, the intelligence level is unsatisfactory. Products or systems in this level range can not only eliminate the interference of the first level range, but also analyze the characteristics of the target entering the monitoring screen (such as the size or shape of the target), so as to distinguish whether it is a person or other animals, or a car or a truck, etc.

Although products or systems in this level range can identify people and objects (i.e. the size or shape of the target), they cannot judge whether the behavior of people or objects is abnormal. The function of their intelligent video analysis is relatively poor, that is, it fails, so the product has not reached practical use.

Products in the third range

The intelligent IQ of products or systems in this level range is 3 points, or greater than or equal to 60 and less than 75 points, that is, the intelligence level is qualified or medium. In addition to being able to eliminate interference and clearly distinguish between people and objects, products or systems in this level range can also simply distinguish and identify abnormal behaviors of people and objects under room temperature conditions, such as realizing abandoned object detection, people and objects crossing the line detection, and can accurately perform statistical calculations on people and objects. Products or systems in this level range are set with better professional algorithms, so their calculations are more accurate.

Products or systems in this level range can identify certain abnormal behaviors of people and objects under indoor conditions such as room temperature, and their intelligent video analysis function is acceptable, that is, it reaches the passing level, so the product can still be used, but only under conditions such as normal indoor temperature.

Products in the fourth range

The intelligent IQ of products or systems in this level range is 4 points, or greater than or equal to 75 and less than 85, that is, the intelligence level is good. Products or systems in this level range can realize behavioral analysis, that is, they can distinguish and identify more abnormal behaviors of people and objects. For example, the system can detect abnormalities of the previous level in outdoor environments, and can also detect abnormal behaviors such as wandering, running, sliding and falling of targets (such as people falling from upstairs), fighting between people, car hitting people or two cars colliding, and fire precursors indoors and outdoors.

Methods and steps for evaluating intelligent video analysis technology

From the above, we can see that not all products with intelligent video analysis functions are "intelligent". We must conduct an intelligent evaluation to see how many intelligent video analysis functions it has and whether it can work, etc., so as to help users choose video surveillance system equipment products with the required level of intelligence.

Evaluation Methods

You can understand the role of intelligent video analysis by looking at the level of intelligent products

The specific evaluation method can be used to evaluate the role of intelligent video analysis technology by "grading of intelligent evaluation methods with 5 levels of scoring". Products in the first level range have extremely poor intelligent video analysis and are not practical at all; products in the second level range can identify people and objects (i.e. the size or shape of the target), but cannot judge the abnormal behavior of people and objects, and their intelligent video analysis role is not qualified, so the product cannot be practical; products in the third level range can identify certain abnormal behaviors of people and objects under indoor conditions such as room temperature, and their intelligent video analysis role is OK, that is, it reaches the qualified level, so the product can only be limited to application under indoor conditions such as normal temperature; products in the fourth level range can detect abandoned objects (discarded objects) in crowded scenes, and can very effectively detect luggage left alone at airports, or items that may be stolen in museums and warehouses. Generally, it can be linked with anti-theft, fire prevention, access control and other systems, and can accurately detect and identify people's faces, gait or voice. Products or systems in this level range can identify abnormal behaviors of people and objects under general outdoor conditions, and their intelligent video analysis works well and has good anti-interference ability, that is, it reaches good, so the product can be used indoors and outdoors; products in the fifth level range can detect abnormal behaviors of people and objects of the above levels under more severe climatic conditions than the fourth level range, and can detect more comprehensive and more abnormal behaviors of people and objects, such as people wearing masks, pointing knives or guns at people, etc., and can accurately link with anti-theft, fire prevention, access control and other systems, and can accurately detect and identify people's faces, gaits, voices, etc. Because it can eliminate all interference and can be used in extremely harsh environments, this level is excellent, and its intelligent video analysis works very well, that is, the level of intelligence is extremely high (this is the direction we are working towards).

Evaluating the role of intelligent video analysis technology from the perspective of intelligent software algorithms

The core of intelligent video analysis and recognition technology is intelligent software algorithm. Therefore, we can start from the intelligent software algorithm to evaluate whether the role of intelligent video analysis and recognition technology is truly practical. Generally speaking, the effective intelligent video analysis and recognition technology, that is, the practical intelligent software algorithm, mainly depends on whether it can meet the following requirements:

① Whether there is a good degree of consistency between different monitoring scenarios and the mathematical model of the intelligent software algorithm. Whether the mathematical model of the intelligent software algorithm used takes into account the influence of factors in different monitoring scenarios, that is, whether the monitoring environment and the intelligent software algorithm model achieve the greatest consistency. For example, when the target size is used as an important classification feature, usually in a scene with a large depth of field, the target size varies greatly, and the accuracy of target classification will be greatly reduced. This can be solved by reducing the depth of field of the scene, or adding a scene calibration algorithm, or reducing the weight of the size feature in the classification algorithm.

Are there some software modules added to the intelligent software algorithm framework according to different scenarios?

● For video images captured by cameras mounted on moving objects such as cars, an anti-shake software module must be added to improve the processing effect in the case of camera shake;

●In outdoor scenes with severe shadows, a shadow suppression software module should be added to improve the processing effect in outdoor scenes with severe shadows;

●In the case of drastic changes in lighting, it is necessary to add a light change suppression software module to improve the processing effect in the case of drastic changes in lighting;

● In scenarios where target images frequently occlude each other, an occlusion processing software module can be added to improve tracking accuracy in scenarios where target images frequently occlude each other.

Have you considered the impact of complex surveillance scenes such as low illumination, high disturbance, and high congestion on system performance?

If there is a tidal filter, it can filter out objects that often change in shape or move in an irregular direction or at too fast a speed, such as sunlight reflected from the water surface.

Are there object size filters (i.e. filtering out objects that are too large or too small) and object size sudden change filters (i.e. filtering out objects whose size changes suddenly) to avoid false alarms, etc.

Steps of Assessment

The intelligent evaluation of an intelligent video surveillance system or product and its intelligent video analysis technology can be carried out according to the following specific steps:

Expert evaluation and appraisal team for organizational intelligent evaluation

First of all, in order to conduct an accurate and reliable assessment, it is necessary to organize 5 or more real security monitoring technology or optoelectronic information technology experts with associate senior professional titles or above to form an expert assessment and appraisal group for intelligent assessment.

Evaluate the intelligent level of submitted materials

Secondly, security monitoring or optoelectronic information technology experts with associate senior professional titles or above will conduct a preliminary assessment of the theoretical level of intelligence of the submitted materials.

Test and monitor environmental conditions on site

Thirdly, security monitoring or optoelectronic information technology experts with associate senior professional titles or above will organize the testing of the environmental conditions of the monitoring site and evaluate the degree of consistency between their intelligent software algorithm model and the environmental conditions of the monitoring site.

Simulate abnormalities of people and objects to test the actual intelligent functions of video surveillance systems or products

The fourth step is to simulate the abnormalities of people and objects under the environmental conditions of the monitoring site, and have security monitoring or optoelectronic information technology experts with associate senior professional titles or above test the actual intelligent functions of the video monitoring system or product under different distance conditions, and conduct testing and evaluation according to the 5-level scoring evaluation method.

Comprehensive evaluation and scoring by experts

Finally, security monitoring or optoelectronic information technology experts with associate senior professional titles or above will conduct a comprehensive assessment and scoring based on the above situation using the 5-level scoring method, and finally sign and approve to form the final conclusive opinion of the evaluation and appraisal.

Conclusion

From the above, we can see that not all products with intelligent video analysis functions are "intelligent". We must conduct an intelligent evaluation of the intelligent video surveillance system or product and its intelligent video analysis technology to see which level of product it belongs to, how many intelligent video analysis functions it has, whether it can work, whether its intelligent software algorithm is consistent with the environment of the required monitoring scene, whether there are some required software modules attached, etc., so as to help users choose the video surveillance system equipment products with the practical intelligent level they need.

The five-level range method of intelligent evaluation proposed in this article is only a preliminary idea and may have many shortcomings. It is hoped that it will have certain reference value for the evaluation of security technology systems, or provide a way of thinking for the development of evaluation work in the entire security field.

With the continuous development of the security industry and the continuous progress of evaluation work, I believe that more complete and better evaluation methods will emerge to put the evaluation work of my country's security industry on the right track.

Products or systems in this level range can also detect abandoned objects (discarded objects) in crowded scenes, and can very effectively detect luggage left alone at airports, or items that may be stolen in museums and warehouses. They can generally be linked with anti-theft, fire prevention, access control and other systems, and can accurately detect and identify people's faces, gait, voice, etc.

Products or systems in this level range can identify abnormal behavior of people and objects under general outdoor conditions. Their intelligent video analysis function is good and their ability to eliminate interference is good, which means they can be used indoors and outdoors.

The fifth level of products

The intelligent IQ of products or systems in this level range is 5 points, or greater than or equal to 85 points to 100 points, that is, the intelligence level is excellent. Products or systems in this level range can detect crowded scenes (such as airport waiting halls and railway station ticket gates). The system can provide multiple functions, such as calculating how many people are in a very crowded scene during a certain period of time; if the number of people passing through the entrance per unit time is too large (the system can be pre-set), an alarm will be issued, etc.

This product or system can detect the above-mentioned abnormal behaviors of people and objects under more severe weather conditions than the fourth level, and can detect more comprehensive abnormal behaviors of people and objects, such as people wearing masks, pointing knives or guns at people, etc., and can accurately link with anti-theft, fire prevention, access control and other systems, and can accurately detect and identify people's faces, gait, voice, etc. In particular, systems with scores above 95 can eliminate all interference and detect and distinguish targets that are extremely small or have extremely low contrast and cannot be seen by the human eye.

Obviously, the intelligent video analysis of the video surveillance system at this level works very well, that is, it is at an excellent level. Because it can eliminate all interference and can be used in extremely harsh environments, its intelligence level is extremely high.

From the above, we can see that not all products with intelligent video analysis are good intelligent products. The key is to see which level it belongs to. For example, if a product only belongs to the first or second level, although it also has intelligent video analysis functions, the function is poor and cannot be used in security; while the intelligent video analysis function of products belonging to the third, fourth, and fifth levels is effective and good, that is, it is a practical product needed in security. Therefore, for products with intelligent video analysis functions, it is necessary to evaluate the intelligent level range. Only products belonging to the third, fourth, and fifth levels can be selected by security engineers and users.