Google's human image segmentation model Bodypix has been updated again, optimized for Coral development board, running smoothly at 720p/30fps

Xiaocha from Aofei Temple
Quantum Bit Report | Public Account QbitAI

AI development boards are indeed a hot topic for major manufacturers. Raspberry Pi 4 has just released an 8GB version, and the almost forgotten Google Coral development board has welcomed a new machine learning model and has been listed on the GitHub open source hot list recently.

Last year, Google released the Bodypix 2.0 model for identifying 24 parts of the human body. Now Google has ported it to the Coral device and made special optimizations for the API of this hardware.

Coral BodyPix has many uses. In addition to the most basic recognition and segmentation of different parts of the human body, it can also realize anonymous traffic statistics.

Of course, these are just two applications listed on the official GitHub, and there are many more actual uses.

What is Bodypix

Bodypix was first released by Google in February 2019 as an AI model for human body segmentation.

BodyPix can segment images into pixels belonging to people and pixels not belonging to people. The human parts can be further classified into one of 24 body parts, including the head, right upper arm, left calf, torso, etc.

Last November, Google released its upgraded version 2.0, which added support for multi-person images, ResNet-50-based models, new APIs, weight quantization, and support for images of different sizes.

As a Tensorflow.Js project, Bodypix can run on the browser side and is very smooth even on mobile phones. According to official tests by Google, the Bodypix model can run smoothly at 21fps on an iPhone X.

Now, after official optimization by Google, even the Coral development board with lower hardware specifications can be used smoothly, and the frame rate is even comparable to that of the iPhone.

Installation and Usage

BodyPix depends on the latest version of the Coral API, and for the Coral development board, you need to install the latest Mendel OS. Then run the install_requirements.sh in the project to install other dependencies.

python3 bodypix.py

The above command can also be added with different options for more detailed operations, such as adding --width and --height to control the resolution of the input image, so that you can choose between running speed and accuracy.

python3 bodypix.py --width 480 --height 360  # fast but low res
python3 bodypix.py --width 640 --height 480  # default
python3 bodypix.py --width 1280 --height 720 # slower but high res

In addition to changing the resolution, the project also includes two models, MobileNet and ResNet, for users to choose from.

If you are facing the monitor and want to create a mirror effect, you can add the parameter:

python3 bodypix.py --mirror

In addition, the encoding format of the input frame can be defined:

python3 bodypix.py --h264
python3 bodypix.py --jpeg

There is also the anonymous mode we mentioned at the beginning. By adding the parameter -anonymize, you can hide the character's physical features and skeleton and only show the external outline.

python3 bodypix.py --anonymize

About Google Coral

Google launched a series of edge AI devices at the TensorFlow Developer Conference in early 2019, including not only development boards similar to the Raspberry Pi, but also AI acceleration computing sticks. They all have built-in edge TPUs, which are more suitable for AI computing than the Raspberry Pi.

At the time, Google claimed that the Coral development board could run deep neural networks on high-resolution video at 30 frames per second.

Now, the Coral Bodypi released by Google is optimized for the iMX8MQ GPU and VPU on the Coral development board, enabling real-time processing and rendering on the platform.

On the project homepage, Google stated that the optimized Bodypix can reach 30fps on the Coral development board even when running at 1280x720 resolution, finally fulfilling its original promise.

Open source address:
https://github.com/google-coral/project-bodypix