Based on Enclustra XCZU15EG core board and PE1 baseboard

AI: Robotic rock-paper-scissors, based on Enclustra XCZU15EG core board and PE1 baseboard

Artificial intelligence (AI) is taking over more and more applications and life scenarios, such as image detection and classification, translation and recommendation systems, etc. The number of applications based on machine learning technology is huge and growing. With Enclustra's core board module combining FPGA and ARM processor, it is easier than ever to use AI offline and at the edge.

In order to demonstrate "FPGA-based AI, machine vision, and motion control", Enclustra created a demo of a robot arm and a human interacting and playing rock-paper-scissors.

The low latency and low power consumption of FPGAs are demonstrated in this demo. The neural network with Synthara IP core runs on the Enclustra Mercury+ XU7-15EG core module (SoM based on XCZU15EG) based on Xilinx Zynq UltraScale+ and Mercury+ PE1 baseboard to analyze the streaming data collected by the camera. Gestures are detected in real time, and the reasoning of each image only takes 8~9 ms, with a sensitivity of more than 100 fps. Common cameras are 30 or 60 fps, so the bottleneck is the frame rate of the camera. The robot servo motor responds according to the rules of the game.

It is worth mentioning that this demo realizes the processing of various sensors and signals in a single device: image processing, neural network reasoning, and robotic arms are all controlled in one FPGA.

The images of player gestures were divided into five categories: rock, scissors, paper, illegal gestures, and no gestures. In order to enable the neural network to accurately classify the images, Enclustra collected a variety of images from the videos recorded by several players: different players, male/female players, different accessories (watches, wristbands, long sleeves...), different brightness, left/right-handed players, different movement speeds, different distances, etc. After the neural network learned nearly 10,000 images, it had the ability to quickly defeat new players.