Super useful information! One article to understand the privacy and security issues of facial recognition in App applications | CAICT report

Author | Liu Lin

"Scan your face to unlock your phone," "Scan your face to shop," "Scan your face to verify your identity when applying for a bank card"... The "face" we are born with is not only linked to our appearance, but is also gradually evolving into the unique identification mark of our identity, gradually penetrating into our daily lives and providing us with great convenience.

But when face recognition meets reality, privacy issues are still an insurmountable reality. So, when face recognition is applied to more scenarios, is our privacy still safe?

Today, let’s start with the privacy and security of facial recognition in App applications.

Recently, the Security Research Institute of the China Academy of Information and Communications Technology and Beijing Baidu Netcom Technology Co., Ltd. jointly released the "Privacy Security Research Report on Facial Recognition Technology in App Applications" (hereinafter referred to as the "Report").

The report sorted out the market situation, characteristics and difficulties of face recognition technology, as well as its application scenarios and purposes in apps, and analyzed the security issues of face recognition technology in the process of app application based on actual cases. In terms of security issues, apps with a large number of downloads using face recognition technology were selected and evaluated. Targeted suggestions were also put forward in terms of laws and regulations, regulatory systems, technical standards, industry self-discipline, etc., in combination with my country's actual situation.

If you want to get the full text pdf of this report, please reply to the keyword "706 report" in the Leifeng.com official account to extract it.

Face recognition is a biometric technology that identifies people based on their facial features. Specifically, the computer uses a video acquisition device to obtain the facial image of the object to be identified, and then uses the core algorithm to calculate and analyze the facial features, face shape, angle and other feature information, and then compares it with the existing templates in its own database, and finally determines the user's true identity.

1. Face recognition technology

From the entire process of face collection to face recognition, face recognition technology generally includes: face image collection and detection; face feature extraction; face regularization and face recognition comparison, etc.

• Face image acquisition and detection
  

Different facial images can be captured by the camera, such as static images, dynamic images, different positions, different expressions, etc. When the user is within the shooting range of the acquisition device, the acquisition device will automatically search and capture the user's facial image. In practical applications, face detection is mainly used for preprocessing of face recognition, that is, accurately calibrating the position and size of the face in the image.

• Face feature extraction

The features that can be used in face recognition systems are usually divided into visual features, pixel statistical features, face image variation coefficient features, face image algebraic features, etc. Face feature extraction is performed on certain features of the face. Face feature extraction, also known as face representation, is the process of modeling the features of the face. The methods of face feature extraction can be summarized into two categories: one is a knowledge-based representation method, and the other is a representation method based on algebraic features or statistical learning.

• Face regularization
  

The preprocessing of face images is based on the face detection results, and the image is processed and finally serves the process of feature extraction. The original image obtained by the system is often not used directly due to various restrictions and random interference. It must be preprocessed by grayscale correction, noise filtering and other image preprocessing in the early stage of image processing. For face images, the preprocessing process mainly includes light compensation, grayscale transformation, histogram equalization, normalization, geometric correction, filtering and sharpening of face images.

• Face recognition comparison
  

The extracted facial image feature data is searched and matched with the feature template stored in the database. By setting a threshold, when the similarity exceeds this threshold, the matching result is output. Face recognition is to compare the facial features to be identified with the obtained facial feature templates, and judge the identity information of the face according to the similarity. It can be divided into 1:1, 1:N, and attribute recognition. Among them, 1:1 is to compare the eigenvalue vectors corresponding to 2 faces, and 1:N is to compare the eigenvalue vectors of 1 face photo with the eigenvalue vectors corresponding to other N faces, and output the faces with the highest similarity or the top X similarity rankings.

2. Application of face recognition technology in identity verification

The authentication process of face recognition in the App is as follows:

The user takes a photo of his or her ID card information and uploads it to the App. The App obtains the user information and the ID card system photo through citizen identity information query, establishes a user profile and associates the user's face. When the App scans the user's portrait, it performs liveness detection, face quality detection, face image processing, etc., and compares the face with the previously obtained user portrait photo to complete the identity verification.

3. Features

• Natural
  

Naturalness means that the identification method is the same as the biological characteristics used by humans for individual identification. For example, in face recognition, humans also distinguish and confirm identities by observing and comparing faces. Other natural identification methods include voice recognition and body shape recognition. However, fingerprint recognition and iris recognition are not natural.

• Non-contact

Face recognition completely utilizes visible light to obtain face image information. Unlike fingerprint recognition which requires the use of finger contact sensors to collect fingerprints, users do not need direct contact between their faces and the device. It can simultaneously meet the needs of multiple people to continuously recognize and sort face image information, and can be used in some application scenarios such as hospital temperature measurement and community access control.

4. Difficulty

• Similarity
  

There is little difference between different individuals. All facial structures are similar, and even the structure and appearance of facial organs are very similar. This feature is beneficial for using faces for positioning, but it is not conducive to using faces to distinguish between human individuals. For example, the twin phenomenon, the average birth rate in the world is 1:89. Some twins have different faces, and some twins are even very similar in terms of facial features. This is a big challenge for face recognition systems.

• Volatility
  

The facial features of a person's face are unstable. A person can produce many expressions through changes in his face, and the visual image of the face can vary greatly depending on the observation angle. In addition, face recognition is also affected by lighting conditions, face coverings (masks, sunglasses, beards, hair), age, and other factors.

• Vulnerability
  

With the development of digital photography and video synthesis technology, it is becoming easier to obtain the facial information of a specific person or synthesize facial information. With the development of deep learning technology such as adversarial training, computers can synthesize the facial information of anyone with high precision.

1. Market prospects

In 2018, the scale of China's computer vision face recognition market was 15.17 billion yuan. According to the summary of six authoritative institutions by the Qianzhan Industry Research Institute, it is optimistically estimated that the scale of my country's computer vision face recognition market is expected to exceed 100 billion yuan in 2020.

According to a research report by iResearch, facial recognition market applications cover security, finance, smart parks, transportation, Internet services, etc. According to the statistics of iResearch, in 2018, security accounted for 61.1% of the facial recognition market share, finance accounted for 17.1%, smart parks accounted for 6.7%, Internet services accounted for 3.9%, transportation accounted for 3.3%, personal intelligence accounted for 2.9%, and others (smart cars, smart retail, government services, operator services) accounted for 5%.

2. Enterprise application

Facial recognition is currently developing rapidly in China, and various emerging companies have sprung up like mushrooms after a rain. Chinese facial recognition unicorns include Megvii Technology and SenseTime, while startups such as CloudWalk Technology and Yitu Technology are also continuing to make efforts to seize the market. From the perspective of the facial recognition industry chain, the upstream of the facial recognition industry is supported by hardware foundations, including high-definition cameras, processing chips (TPU\CPU\GPU), servers, and data and video transmission equipment; the midstream of the industry chain is mainly facial recognition algorithms and software services, and the downstream of the industry chain is specific scenario applications, namely application solutions, consumer terminals or services, etc.

Domestic Internet companies such as Baidu, Alibaba and Tencent attach great importance to facial recognition.

Alibaba controls Megvii Technology, SenseTime, and Yitu Technology, and has developed its own facial recognition interface. It has fully applied facial recognition technology to apps such as Alipay and Taobao, and has worked with other business units under the group to study application scenarios for facial recognition.

Tencent It has its own Youtu team, which provides image technology support for more than 50 apps including QQ Space, Tencent Maps, and Tencent Games.

Baidu Face recognition has also developed rapidly relying on huge data resources. Baidu has launched apps such as Baidu Image Recognition and FaceOptimizer by using face recognition technology.

3. Application scenarios

• Application scenarios in financial apps
  

The main purpose of integrating facial recognition into financial apps is to ensure the security of users' financial transactions during use. Taking Alipay as an example, when users use "Jiebei" to borrow money, in addition to entering the password, they generally need to perform facial detection to confirm that the App operator is themselves. Facial recognition can effectively prevent Alipay accounts from being stolen and causing user property losses. In addition, financial apps can also provide remote account opening, card binding and identity verification, account login, installment shopping, facial attendance, facial payment and other services through facial recognition technology.

In the process of facial recognition being implemented in the financial industry, major banks have also tried to introduce facial recognition into financial scenarios such as face payment, instant card opening, VYM, etc. However, from a technical perspective, technology is not omnipotent. Although facial recognition technology is now very mature, lighting conditions, weather, user plastic surgery, etc. still affect facial recognition results.

In addition, the application of facial recognition in high-security business applications such as transfer payment and instant card opening should be cautious. We cannot simply rely on facial recognition technology to solve the problem of user identity verification. We also need to adopt two-factor or even multi-factor authentication including facial recognition to improve security.

• Application scenarios in online education apps
  

One of the purposes of online education apps using facial recognition is to verify the identity of students and avoid multiple people using one account. Facial recognition can greatly reduce the problem of account sharing. By triggering the facial recognition mechanism at a certain frequency, it can verify whether the face currently using the online school account is the same person.

In addition, another major use of face recognition is to help teachers understand students' learning status. Online classes are different from offline classes. Teachers cannot identify students' acceptance of the course by observing their expressions. Facial expression recognition can help teachers better understand students' needs.

Online education apps mainly serve primary and secondary school students. As children’s cognitive ability, risk identification ability and self-protection ability are relatively weak, children’s personal biological protection information is the focus of all walks of life. According to the survey results of the “Face Recognition Landing Scenario Observation Report” released by the Nandu Personal Information Protection Research Center, 33.84% of the respondents disagreed with the application of face recognition technology to education-related systems. It can be seen that more caution should be exercised when using face recognition technology on minors.

• Application scenarios of telecommunications apps

The main purpose of telecommunications apps accessing facial recognition is to achieve real-person authentication during SIM card activation. Taking the "China Mobile App" as an example, after users purchase a SIM card on the China Mobile App, they need to complete real-person authentication in the "Card Number Activation" service function of the App. During the activation process, they upload their ID card information and then conduct a portrait video authentication. During the video authentication process, users are required to record a 6-second video, and the recorded content is to read aloud the 4-digit verification code randomly generated on the screen. The SIM card can only be successfully activated after the video review is passed.

On September 27, 2019, the General Office of the Ministry of Industry and Information Technology issued the "Notice on Further Improving the Management of Real-Name Registration of Telephone Users" to guide telecommunications companies to carry out real-name registration of telephone users. In order to ensure the consistency of the person and the certificate in the telephone network access process and to innovatively use technical means such as artificial intelligence, the Ministry of Industry and Information Technology requires telecommunications companies to fully implement portrait comparison technical measures in physical channels from December 1, 2019. Only after the portrait comparison is consistent can the network access procedures be processed. Therefore, in order to safeguard the legitimate rights and interests of citizens in cyberspace and effectively prevent telecommunications network fraud, face recognition is also required when activating SIM cards online.

• Application scenarios of travel apps

Travel apps that integrate facial recognition functions can maximize the safety of drivers, passengers, and cargo.

Taking the facial recognition app "Didi Chuxing" as an example, after the driver fills in various basic information in the app, he needs to perform the final step of facial image authentication before he can accept the order. On the one hand, it can protect the driver's identity information and property security and prevent account theft; on the other hand, it can also protect the personal safety of passengers and prevent them from encountering bad drivers.

On September 11, 2018, a special inspection team composed of the Ministry of Transport, the Central Cyberspace Affairs Commission, the Ministry of Public Security and other departments entered the online car-hailing and ride-sharing platform companies to carry out special safety inspections, and required relevant apps to use facial recognition and other technologies before dispatching orders to review the consistency of vehicles and drivers. At the same time, the application of facial recognition technology to travel apps can effectively protect the property and personal safety of drivers and passengers.

• Application scenarios of Meitu Entertainment App

In addition to ensuring account security, Meitu Entertainment Apps can also use the facial recognition function to implement various creative interactive marketing activities.

Taking the face recognition app "Meitu XiuXiu" as an example, after users download the Meitu XiuXiu software to take photos, they generally use the picture beautification function. At this time, the App can access the face key point positioning function to help users locate key parts of the face including eyebrows, eyes, chin, etc., making it easier for users to use the beauty function.

At the same time, users can also customize their own exaggerated, funny, and unique facial photos. For example, ZAO, which was very popular last year, can also provide services such as photo face swapping, video face swapping, the same emoticon package, and changing clothes and hairstyles through face recognition technology.

The use of facial recognition technology by Meitu Entertainment Apps is necessary for their business functions, but the collection and use of personal biometric information should be regulated. The new version of the "Information Security Technology Personal Information Security Specification" (hereinafter referred to as the "Specification") stipulates that before collecting personal biometric information, the purpose, method and scope of use should be separately informed, and the explicit consent of the subject of personal information should be obtained.

At the same time, the "Specification" also stipulates that original personal biometric information should not be stored in principle. Therefore, in the expansion of business functions of Meitu Entertainment App, facial recognition technology should be reasonably used in accordance with the principle of minimum necessity, and users should be informed separately and consent should be obtained in accordance with the "Specification". When users refuse to authorize the relevant permissions for the expansion of business functions to use facial recognition technology, the App shall not repeatedly seek authorization, nor affect the use of other business functions unrelated to the permission.

• Application scenarios of e-commerce apps

One of the main purposes of e-commerce apps integrating facial recognition is to ensure the security of user accounts. Usually, facial recognition is performed when logging into an account to achieve real-person authentication, preventing criminals from logging into user accounts by cracking passwords.

Secondly, in order to improve the user service experience, e-commerce apps use face recognition to provide online dressing and trial services. In addition, the face recognition application scenarios of e-commerce apps also include: background image data management, that is, the management of prohibited pictures and advertising pictures, live broadcasts, short videos, etc.

E-commerce apps use facial recognition technology to improve user service experience, enhance user stickiness or provide convenience for users, which is an extended business function of e-commerce apps. Therefore, when e-commerce apps use facial recognition technology to obtain facial feature information, they should inform and obtain the user's consent. Users have the right to refuse to use related services, and users cannot be forced to provide facial feature information.

• Application scenarios of smart park apps

Smart campus apps access facial recognition functions mainly for access control management, attendance management, meeting management, etc. In the access control management application scenario, employees can use the app to solve all permission management issues in the enterprise building campus with just one face.

In the attendance management application scenario, the App is based on facial recognition technology, combined with network and GPS positioning, which can eliminate the phenomenon of punching in for others and solve the problem of attendance difficulties for field staff.

In the conference management application scenario, participants register for the meeting by entering their faces. When signing in for the meeting, the face recognition function of the App will enter the guest's facial information and automatically compare it with the background information, so that the identity of the guest can be quickly identified.

Smart Park Apps use facial recognition technology to save labor costs for enterprises, and the operation is efficient, fast and easy to manage. However, there are certain risks in the process of using facial recognition technology in smart park apps, and security incidents of deceiving facial detection through deep forgery are emerging one after another. Therefore, important departments such as state agencies and confidentiality units should not rely solely on facial recognition technology for access control management.

1. Lack of network and data security mechanisms can easily lead to facial data leakage

The current security technical standards and usage specifications for facial recognition technology are not perfect. There are no relevant regulations on the responsibilities and obligations of facial data controllers, the rights of facial data subjects, and the security measures that should be taken in the collection, storage, and processing of facial data.

Therefore, the security measures that have been taken by most facial recognition technology developers and application service providers may be difficult to cope with the security threats faced by facial recognition technology, and security incidents such as facial data leakage are prone to occur.

In addition, the cybersecurity ecosystem continues to deteriorate, and system security vulnerabilities are almost inevitable, so face database leaks are not uncommon. What's more frightening is that since biometric information is unique and non-renewable, once it is lost or leaked, it is permanently leaked, which will cause endless harm.

2. Irregular application of facial recognition technology provides potential for abuse of facial data

As facial recognition technology is increasingly used in people's lives, facial features have gradually become one of people's identity documents, but there are some irregularities in the application of facial recognition technology. First, most apps do not clearly inform and obtain user consent in accordance with the "Specifications" when collecting facial data, and do not even explain the purpose, scope and method of using facial recognition technology in the privacy policy, making passive collection and use of facial data a normal practice.

Secondly, some social entertainment apps and online education apps fail to collect and use facial data in accordance with relevant laws and regulations, resulting in frequent abuse of facial recognition technology.

3. Deep fake facial technology seriously threatens users’ property and even personal safety

Due to the characteristics of face recognition technology such as non-contact, low cost, fast detection, and automatic learning, face recognition has become an important means of identity recognition. However, along with face recognition technology, there is also the "deep fake" technology that uses machine learning systems and images and videos to change faces. Since 2017, deep fake technology has become active on the Internet. As the algorithm of this technology becomes more and more mature, whether it is a portrait or a sound or video, it can be forged or synthesized to the extent that it is almost impossible to distinguish the authenticity. The success rate of identity deception is as high as 99.5%, and it has even become the nemesis of many face recognition systems.

In view of this, it has become possible to use deep fake technology to crack verification systems such as facial recognition, illegally steal other people's payment accounts, obtain other people's personal information, or engage in other illegal activities under other people's names, which seriously threatens the property and personal safety of citizens, and may even threaten national security and public safety, causing social anxiety and a crisis of trust.

1. Accelerate the development of laws and regulations related to facial recognition

At present, the relevant laws and regulations on the protection of citizens' biometric information and other personal information are scattered in the General Provisions of Civil Law, the Cybersecurity Law, the Consumer Rights Protection Law, and the relevant judicial interpretations and regulations promulgated by the Supreme People's Court, the Supreme People's Procuratorate, and the State Council. The content is only some principled provisions on the collection, use, storage, and transmission of personal information. Therefore, my country needs to improve the laws and regulations on the use of personal biometric information, including face recognition, as soon as possible, clarify the scope of citizens' personal biometric information to be protected by law, the obligated subjects of citizens' personal biometric information protection, strengthen accountability, ensure the security of personal biometric information, standardize the use, and increase the penalties for violations of citizens' personal privacy, especially the leakage and abuse of personal biometric information.

2. Accelerate the construction of a regulatory system for the application of facial recognition technology

Establish a necessity assessment system for the application of facial recognition technology. Before adopting facial recognition technology, enterprises or organizations need to conduct a necessity assessment of the application of technology based on the technology implementation method, business scenarios, and data collection and use; at the same time, relevant regulatory authorities can establish a "negative list" or "white list" for the application of facial recognition technology in advance, and strengthen ex ante supervision with a "list + assessment" regulatory approach.

In addition, the in-process evaluation and post-approval accountability system for the application of facial recognition technology should be improved and perfected. On the one hand, enterprises or organizations using facial recognition technology should be urged to follow relevant safety regulations, implement facial recognition technology safety prevention and control measures, and conduct regular safety assessments; on the other hand, enterprises or organizations involved in security incidents such as facial data leakage should be held accountable, and regular visits and continuous supervision should be conducted on the enterprises involved within three to five years.

3. Accelerate the development of a series of security standards for facial recognition technology

Face recognition technology is gradually maturing, and more and more apps are using face recognition technology. Various security standards for face recognition technology, including standards for protecting personal biometric information, should be issued as soon as possible. It is recommended to speed up the development of a series of standards such as the security of face recognition technology itself and the protection of personal biometric information in App applications, so as to guide the industry to regulate the use of face recognition technology in Apps based on standards, improve the level of security protection of face recognition technology itself, and reduce the security risks of face recognition technology in App applications, thereby ensuring the security of users' personal biometric information.

4. Encourage industry associations or social organizations to carry out industry self-discipline

Nowadays, artificial intelligence technology represented by face recognition technology is developing rapidly. However, due to the complexity of face recognition technology, it is difficult to ensure the security of face data. Therefore, it is beneficial to establish a face recognition technology enterprise alliance organization, encourage relevant industry associations or social organizations to actively play the role of industry self-discipline platform, promote all stakeholders to jointly formulate codes of conduct for the collection and use of face data, promote relevant best practices, and drive the improvement of the overall level of personal biometric information protection, which is conducive to the healthy development of the face recognition industry.

In addition, App operators should consciously regulate the application of facial recognition technology in Apps and conduct regular self-assessments or third-party assessments. Before collecting facial data, the purpose and possible risks must be informed to protect the user's right to know and right to choose. At the same time, when users no longer want to continue to authorize the use of their facial data, App operators must provide "exit" or "deletion" channels to ensure the user's right to delete.

We can change our phones, or forge ID cards and driver's licenses, but with current medical technology, we cannot "change our faces." Facial recognition technology has broad prospects, but also potential security risks. Will this technology be the beginning of a new revolution in human-computer interaction, or a collapse of personal privacy? We will have to wait and see.