Embedded Engineer AI Challenge Camp (Advanced): Deploy InsightFace algorithm on RV1106 for real-time face recognition of multiple people

nmg

Embedded Engineer AI Challenge Camp (Advanced): Deploy InsightFace algorithm on RV1106 for real-time face recognition of multiple people [Copy link]

In the previous issue, we challenged to deploy MNIST real-time handwritten digit recognition

In the final stage, under the leadership of @knv and @luyism, most of the challengers successfully reached shore.

This time, we invite you to challenge a slightly more complex model: face recognition

Face recognition is composed of several models, and the entire pipeline is longer

Even those who have participated in digital recognition last time will still have some difficulty

Embedded engineers are welcome to continue to join us and master the engineering practice of face recognition algorithms on the chip side.

I also hope that everyone can succeed in the challenge in the end~

Screenshot of ArcFace face recognition video demonstration from the InsightFace project

Event Schedule

time	content
October 30 - November 20 (21 days)	Learn about the InsightFace face recognition open source project and apply for the RV1106 Linux development board (with camera)
November 21st - November 27th (1 week)	Selection, announcement of the list of Xinghu RV1106 development boards, and distribution of development boards
November 28 - January 12 (45 days)	Participate in the deployment of face recognition applications to RV1106, testing and debugging
January 12th - January 26th (2 weeks)	Event review and awards

Mission description for the second phase of the Challenge Camp:

Based on the InsightFace open source face recognition project, use RV1106 to drive the camera to achieve real-time face recognition of multiple people (at least 3 people) and mark their names in real time.

Who is suitable to participate?

Possess a basic knowledge of Python and C/C++ languages and be familiar with embedded programming; hope to master the engineering of face recognition algorithms on the chip side and become embedded engineers with AI skills; also welcome users who have participated in the Embedded Engineer AI Challenge Camp (beginner level) to continue the challenge. Users who have participated in the beginner level can participate in the final awards and the activity kits will no longer be distributed repeatedly.

How to participate

Phase 1 (from now until November 20 ) : Learn about the engineering of InsightFace algorithm and apply for RV1106 Linux development board (with camera)

1. Click here to learn about the open source face detection project InsightFace: Introduction, source code, ArcFace video demonstration (also capable of side face recognition), etc.

2. Submit the following information to register:

-Reason for application

- In your own words: briefly describe what you know about InsightFace, how to deploy the InsightFace algorithm on RV1106, and how to achieve real-time face recognition for multiple people

-What application do you plan to deploy on the RV1106 Linux development board (with camera)? It requires multi-person real-time face recognition function with InsightFace algorithm;

3. We will select the netizens who will get the RV1106 Linux development board (with camera) based on the content you submitted, and announce the list and arrange the distribution of kits before November 27.

*Note: (1) Considering that face recognition data is large and training is difficult, participants are not required to start from training their own models. (2) InsightFace is recommended but not limited to it. There are many open source face recognition projects on GitHub, such as

链接已隐藏，如需查看请登录或者注册

,

链接已隐藏，如需查看请登录或者注册

,

链接已隐藏，如需查看请登录或者注册

,

链接已隐藏，如需查看请登录或者注册

,

链接已隐藏，如需查看请登录或者注册

, etc. Netizens can also choose an open source project to convert, but they cannot use the ready-made RKNN model.

Pictured: RV1106 Linux development board (with camera) sponsored by Xinghu Technology

Phase 2 (November 28-January 12 ) : Practical InsightFace algorithm engineering and application deployment

The challenger must deploy the multi-person real-time face recognition model to the kit and complete their application development within the specified time. Requirements:

1. Explain the InsightFace algorithm engineering process, application deployment process, and open source related codes, and attach corresponding instructions and debugging process.

2. Test demonstration videos of the following scenarios must be submitted:

- Displays the accuracy score given by the model for detecting a person

- Choose at least 3 people you like, input their photos in advance, and find videos or photos of them to test recognition

- Video demonstrating the functionality of the application you plan to deploy.

The rewards for netizens who complete the task are shown below; netizens who fail to complete the task or do not share as required need to send back the used kit and pay for the postage themselves.

Event Awards

Special Contribution Award (1 person): The person who is the first to post a high-quality post to share the task completion process, and whose post has great reference value for other netizens to complete their tasks, and who promptly helps other netizens solve problems they encounter in completing their tasks; one of the following gifts will be given priority.

Excellence Award (2 winners): Complete the task well and post a detailed and conscientious post to share the task process; the following gifts will be randomly distributed.

Participation Prize (several): Netizens who complete the task conscientiously and share as required do not need to return the Xinghu RV1106 Linux Development Board (with camera) they use as a participation prize.

Special Thanks

The sponsor of this event, Luckfox Technology, provided 30 sets of Luckfox RV1106 Linux development boards (with cameras) for free. We would also like to express our special thanks to the engineers with relevant front-line experience for their technical guidance during the preparation period of this event.

Event WeChat Group

The real-time interaction and reminders of this activity will be conducted in the AI technology exchange group set up in the forum. Any questions can be communicated in the group, and the event administrator will respond in time. Netizens who participate in the event and are interested are welcome to add "helloeeworld" as a friend on WeChat, with the code: AI Challenge Camp, and you will be pulled into the group.

yyhhgg

Sign up to participate

nmg

yyhhgg posted on 2024-11-5 10:33 Sign up

Need to post a comment:

-Reason for application

- In your own words: briefly describe what you know about InsightFace, how to deploy the InsightFace algorithm on RV1106, and how to achieve real-time face recognition for multiple people

-What application do you plan to deploy on the RV1106 Linux development board (with camera)? It requires multi-person real-time face recognition function with InsightFace algorithm;

小火苗

InsightFace is an efficient face recognition , face detection and face alignment algorithm library developed by Megvii.

Set up the development environment, install Python and related dependency libraries, and develop face recognition algorithms on the Ubuntu PC system.

Use RKNN-Toolkit2 to convert the algorithm program, because RKNN is applied on the computer side and cannot be directly run on the embedded side. Therefore, it is necessary to use tools to convert the algorithm program.

In actual development, the face recognition algorithm is developed and trained on the PC side, and finally the algorithm program is deployed on the RV1106 circuit board for actual use and debugging.

kit7828

insightface is an open source 2D/3D face analysis tool based on Pytorch and MXNet. It implements multiple SOTA face recognition, face detection, and face alignment algorithms, and optimizes training and deployment. insightface uses 5 feature point detection, so the algorithm resource requirements are relatively small and it is easy to deploy and implement on embedded systems.

For specific faces, they need to be trained in a PC environment before being deployed on the embedded board RV106. The RKNNLite toolkit can be used to convert the InsightFace algorithm model and implement face recognition on RV1106. The application planned to be challenged and deployed is real-time human trajectory tracking, that is, real-time annotation of specific face motion trajectories.

At present, there is still a long way to go in terms of learning, but the prospects in this field are still very promising. I will work hard to overcome the difficulties in learning and strive to complete this embedded real-time face recognition project.

冬天的木鱼

-Reason for application

1. Basic idea

1.1 Brief introduction to InsightFace:

The InsightFace algorithm is a deep learning model for face analysis and recognition tasks, focusing mainly on face recognition and face verification. It is an integrated Python library for 2D and 3D face analysis, implementing a variety of advanced face recognition, face detection, and face alignment algorithms, and optimized for training and deployment.

1.2 Deploy InsightFace on the RV1106 development board

Step 1. Get an RV1106 development board with a camera;

Step 2. Get familiar with the SDK. Download the SDK, unzip it and view its directory structure.

Step 3. Cross-compile: Use Pico SDK for cross-compile. Install LuckFoxPicoSDK, set environment variables, and ensure that the cmake version is greater than 3.15.

Step 4. Deploy InsightFace algorithm: Deploy the InsightFace algorithm on RV1106 to achieve real-time face recognition of multiple people.

Step 5. Testing and debugging: After deployment, test and debug to ensure that the algorithm runs normally on RV1106 and realizes the real-time face recognition function of multiple people

2. Specific Applications

2.1 Real-time face recognition of multiple people in complex scenes

Try to use the real-time multi-face recognition scenario in the home scene. If time permits, use multi-face recognition in the meeting scene to test and verify the algorithm and model.

waterman

-Reasons for application
I have been deeply involved in Linux embedded design and neural network training and quantization for many years, and have rich development and design experience. I have achieved good results in many artificial intelligence competitions held by Tianchi. In addition, I have also participated in the design of various embedded edge computing projects, including face recognition access control based on ESP32S3, cat and dog classifier based on ESP32S3, and intelligent conversation robot with large language model based on RK3576. Familiar with Linux, C programming and model quantization deployment of rknn. In addition, I am also keen on project technology sharing. I have shared many project design processes and experiences on CSDN and other blog sites, and have open-sourced multiple projects on github. I want to take advantage of this opportunity provided by the world of electronic engineering, make full use of the powerful features of RV1106, further study insightface, design and open-source a real-time monitoring and wanted fugitive/blacklist personnel identification system for a large traffic area.

-Brief description in your own words:
InsightFace is an open source 2D/3D face analysis tool based on Pytorch and MXNet. It contains multiple cutting-edge algorithms for face recognition, face detection and face alignment, which are optimized for efficient training and deployment. In order to implement InsightFace's multi-person real-time face recognition on the RV1106 platform, you first need to complete the verification of the algorithm process on the PC: that is, use InsightFace to perform the entire process of face detection, face alignment and face recognition in sequence to achieve the expected effect. Next, use RKNN Toolkit to quantize and convert the model to the RKNN format compatible with RV1106. This quantization process not only reduces the model size, but also improves the running speed, making it more suitable for running on resource-constrained embedded devices.
After completing the model optimization, you can start developing embedded applications on RV1106. First, implement real-time acquisition of camera data and stream the video into the face recognition system. Then load the converted RKNN model and perform real-time inference on RV1106. At the same time, the OpenCV-Mobile library is used for image preprocessing and cropping of face areas. Finally, face alignment is performed and face feature vectors are extracted and compared, thereby achieving accurate real-time face recognition of multiple people.

-打算在RV1106 Linux开发板（带摄像头）部署什么应用？
我希望基于RV1106 Linux开发板，开发部署一套大流量区域的实时监控与通缉在逃人员/黑名单人员识别系统。因为在大流量区域，如机场、火车站、地铁站、大型商场等，每天都有大量人员进出。传统的安全监控手段依赖于人工观察和手动核查，效率低下且容易出现疏漏。使用多人实时人脸识别技术可以显著提升安全监控的效率和准确性，并且能够及时发现并处理通缉在逃人员或黑名单人员，保障公共安全。

qsxc

-Reasons for Application -In your own words: Briefly describe what you know about InsightFace, how to deploy the InsightFace algorithm on RV1106, and how to implement multi-person real-time face recognition. InsightFace is a 2D/3D face analysis project based on Pytorch and MXNet, which can implement face recognition, face detection, and face alignment algorithms. Use the code provided by the project to convert the MXNet model to be tested into ONNX format. Then, use the Netron tool to verify whether the model is compatible with the RV1106 platform, and further convert it to an RKNN model and try to deploy it on RV1106. -What application do you plan to deploy on the RV1106 Linux development board (with camera)? It requires multi-person real-time face recognition with the InsightFace algorithm; Deploy an intelligent monitoring system that can capture video streams in real time and use InsightFace to detect, identify, and track faces in the video. Using this method, early warnings for specific features can be achieved.

lijinlei

【Reasons for Application】My major is optics, and I mainly engage in the development and maintenance of optical devices and equipment. I am proficient in and use various software and hardware development environments and embedded development, and have rich experience in experimental instrument operation and data collection. Therefore, I want to participate in this challenge camp, combine RV1106 and InsightFace database, realize real-time face recognition and personnel flow monitoring of multiple people in the laboratory, and upload data to the EMQX platform through the MQTT protocol to realize Home Assistant remote data monitoring, providing strong support for experimental safety and personnel flow status acquisition.
【Deployment Plan】InsightFace is an advanced deep face analysis database, including solutions and custom program SDK development for high-quality face recognition, and provides advanced tools to promote creativity and innovation in various industries.
Hardware: Use NVIDIA series GPUs, because InsightFace's training efficiency on GPU is much higher than CPU;
Software: Linux operating system, Python, databases include MXNet (InsightFace's native framework), NumPy, OpenCV, etc.
Data: Database collection: Collect data sets containing multiple face images. The dataset should contain multiple photos of different people for training and verification
Data processing: Face alignment, use tools such as MTCNN to align faces, and ensure that each face is in a similar position and size. The InsightFace source code contains the implementation of MTCNN, which can be used directly
Data conversion: Convert the processed data to the format required by InsightFace, including lst files and rec files.
Model training: Modify the configuration file and execute the training command
Model verification and testing: 1. Establish a target set that needs to be verified; 2. Use the InsightFace script to verify the target set; 3. Analyze the verification results and optimize the model
Application: Access control system, monitoring recognition, etc., the camera captures face information in real time, and recognizes and authenticates through the model.
[Challenge Plan] The high-performance RV1106 Linux development board cooperates with the operating system and database deployment, combines the peripheral camera to collect face data streams in real time, matches with the target face model in real time, performs face recognition and detection, cooperates with the access control system to realize personnel flow control; combines monitoring and alarm systems, drones, etc., to track the target person in real time.

zhuxirui

I have run the official RKNN routines and RKLLM large model on the RK3588 development board, and have also deployed the yolov5 model I trained myself. I understand the RKNN API call, but considering the high cost and power requirements of rk3588, I have not been able to find a practical embedded application scenario. This application is also to learn more about the full process of embedded AI deployment, understand more technical details, and find a foothold for edge AI.

The InsightFace framework provides a wealth of face recognition, face detection, and face alignment algorithms, and is optimized for training and deployment. I personally think that its biggest advantage is that it achieves millisecond-level real-time detection, which is very friendly to embedded devices. With the help of Rockchip's RKNN toolchain, the reasoning performance can be further improved. In actual deployment, the pre-trained model can be exported as a pt model through the pytorch framework and then converted to an onnx model (it can also be directly converted to an rknn model) and then converted to an rknn model through the toolchain. After the algorithm is verified on the PC, it can be uploaded to the development board for running tests. The corresponding program functions are expected to be implemented using python and opencv packages.

InsightFace's facial recognition algorithm can be used to implement a smart doorbell system. When the entered face is recognized, the light will turn on and the door will open. When a stranger's face approaches, the buzzer will sound an alarm and capture the face.