AI face-changing recognition rate exceeds 99.6%, Microsoft technology breaks DeepFake false information

DeepFake face-swapping has caused a stir around the world. Starting with the creation of indecent celebrity videos that look real enough, many users have used this "face-swapping artifact" as a video-faking tool and spread false information to the world through social networks. The emergence of technologies such as DeepFake has not only improved the authenticity of face-swapping, but its open source approach has also lowered the threshold for abusing such technologies to create and spread false information.

In fact, about 30% of synthetic photos and videos that have undergone AI face-changing are not recognizable to the naked eye and can easily be re-circulated as real information. This has become a social problem that needs to be solved urgently. What should and can we do about this problem? Microsoft Research Asia has provided a solution.

In addition to DeepFake, there are many face-changing technologies on the market. The image results generated by different algorithms vary greatly, and it is difficult to use the same face-changing identification model to solve the attacks of all face-changing technologies. At the same time, the face-changing identification model also needs to be able to distinguish face-changing technologies that do not exist at present but may appear in the future. How to predict the development direction of future face-changing technologies and deploy defenses in advance is also an important topic.

Currently, there are three most commonly used AI face-changing algorithms: DeepFake, FaceSwap, and Face2Face. Among them, DeepFake is based on the well-known GAN technology. For the faces it generates, the human recognition rate is about 75%*. FaceSwap is a deep learning algorithm that learns to reconstruct facial features. It can replace the model of a given picture. The human recognition rate for this type of face-changing is also about 75%*. Face2Face replaces the original face with other real faces. It does not involve the generation of faces. For the faces it creates, the human recognition rate is only 41%*. As one of the largest synthetic video databases in academia, the FaceForensics database created by the Technical University of Munich covers public videos edited by the above three face-changing algorithms for academic research.

Over the years, Microsoft Research Asia has developed industry-leading algorithms and models in face recognition, image generation, and other fields. At CVPR 2018, Microsoft Research Asia Visual Computing Group published a paper titled "Towards Open-Set Identity Preserving Face Synthesis", in which the technology can use data from open datasets to realistically synthesize images that retain the identity information of the face in the image. The deep technical accumulation has enabled researchers to have a deeper understanding of the technical principles of the "attacker", and thus to develop face-changing identification algorithms in a more targeted manner.

Figure 1: The model developed by Microsoft Research Asia extracts identity information and attribute information from the Mona Lisa and Audrey Hepburn images for synthesis

Therefore, the face-swapping identification algorithm developed by Microsoft Research Asia, based on the test results of the FaceForensics database, has surpassed the recognition rate of the human eye and the previous best level in the industry*: the recognition rate for DeepFake reached 99.87%, the recognition rate for FaceSwap was 99.66%, and the recognition rate for Face2Face was 99.67%.

Table 1: Recognition test results for known face-changing algorithms

More importantly, general face-swapping identification solutions require the development of specialized face-swapping identification models for each face-swapping algorithm. To identify the authenticity of an image, all models need to be tried one by one. The algorithm from Microsoft Research Asia can use a universal model to identify faces created by different types of face-swapping algorithms. At the same time, the researchers also examined details that are difficult to handle when synthesizing faces, such as glasses, teeth, hair edges, and facial contours, and made them the focus of the algorithm to improve recognition accuracy. Compared with other similar technologies, the face-swapping identification algorithm from Microsoft Research Asia has solved the problem of dealing with images with large dynamic ranges, occlusions, and changes in expressions.

除了准确识别已知算法合成的图像，换脸鉴别的另一大挑战是应对尚未出现的新算法。将现有的换脸鉴别算法直接用于新算法时，它们的有效性往往会显著下降。为此，微软亚洲研究院提出了一种通用换脸鉴别方法。为了更好地考察这一算法对未知换脸算法的鉴别能力，研究团队用真实图像对模型进行了训练，再让其辨别多种未知换脸算法生成的图像。实验结果表明，与基线算法相比，新算法对各类换脸算法的识别率均有大幅提升。随着研究团队对模型的进一步优化，通用鉴别模型一定能越来越精确地帮助我们应对新算法所带来的问题和挑战。

Table 2: Recognition test results for unknown face-changing algorithms

In Microsoft's view, to build trustworthy AI, the following six principles must be followed: fairness, reliability and security, privacy, inclusion, transparency, and responsibility. Microsoft has also established an Artificial Intelligence Ethics Committee (AETHER) to help Microsoft deal with the ethical and social impacts of AI.