TinyissimoYOLO lightweight visual inspection algorithm for MCU

While artificial intelligence (AI) algorithms running on larger, more powerful hardware often get the spotlight, the importance of AI at the edge should not be underestimated. Edge AI refers to deploying artificial intelligence algorithms on local devices such as smartphones, cameras, sensors and other IoT devices, rather than relying solely on cloud-based solutions. This decentralized approach offers many benefits and unlocks a wider application space.

One of the key benefits of edge AI is reduced latency. By processing data locally, edge AI eliminates the need for trips to the cloud, resulting in faster response times. This real-time capability is critical in immediate decision-making scenarios such as self-driving vehicles, industrial automation, and critical infrastructure monitoring. Additionally, edge AI enhances privacy and security as sensitive data remains local on the device, reducing the risk of data breaches and ensuring user privacy.

Despite the numerous advantages, running resource-intensive algorithms, such as complex object detection or deep learning models, on edge devices remains a significant challenge. Edge computing devices typically have limited computing power, memory, and power consumption compared to cloud-based hardware. Striking a balance between algorithm accuracy and device limitations is critical to ensuring efficient operation. Optimizations such as model compression, quantization, and efficient inference techniques are necessary to make these algorithms run well on edge devices.

Since understanding and identifying objects in images or videos is a fundamental task in visual perception, object detection algorithms are of special importance across various industries and applications. Currently, great progress has been made in object detection in the field of resource-constrained edge devices, such as Edge Impulse's FOMO algorithm, which runs 30 times faster than MobileNet SSD and requires less than 200 KB of memory. 

For such an important and diverse application area, there is still a lot of room for development.

The latest entry into the field is a study from ETH Zurich. They developed a highly flexible, memory-efficient, and ultra-lightweight object detection network they call TinyissimoYOLO. The optimizations applied to this model make it ideal for running on low-power microcontrollers.

TinyissimoYOLO is a convolutional neural network (CNN) based on the popular YOLO algorithm architecture. It consists of quantized convolutional layers with 3 x 3 kernels and a fully connected output layer. Both convolutional layers and fully connected linear layers have been heavily optimized in hardware and software toolchains for modern devices, giving TinyissimoYOLO improvements in speed and efficiency. It is a general object detection network that can be applied to a wide range of tasks and requires no more than 512 KB of space to store model parameters.

The model can be deployed on almost any hardware that meets its very modest requirements, including platforms with Arm Cortex-M processors or AI hardware accelerators. A variety of devices were tested using TinyissimoYOLO, including ADI MAX78000, Greenwaves GAP9, Sony Spresense and Syntiant TinyML.

While evaluating their approach, the team found that they could run object detection on the MAX78000 board at an astonishing 180 frames per second. This outstanding performance is accompanied by ultra-low power consumption of only 196 µJ per inference. Surprisingly, the tiny model also performed on par with larger object detection algorithms.

However, achieving this functionality requires reducing the size of the target, for example, the image input size is limited to 88 x 88 pixels. This may not be enough resolution for many purposes. Furthermore, as the number of objects increases, the multi-class object detection problem becomes more difficult, so each image supports up to three detected objects.

Despite these limitations, TinyissimoYOLO's versatility, accuracy, and minimal hardware requirements make it one of the options for those looking to perform object detection at the edge.