You read that right: Machine learning on MCU is now very powerful!

Running machine learning models on embedded devices is often referred to as embedded machine learning. Machine learning in embedded devices has many benefits:

TinyML: A new business opportunity for MCU manufacturers

The initial success of deep learning models is mainly attributed to large servers with large amounts of memory and GPU clusters. Although cloud-based deep learning is very successful, it is not suitable for all situations because many applications require inference on devices. Most of today's AI applications are based on machine learning technology. If machine learning models can run smoothly on resource-constrained devices, it will open a technical door for many emerging applications. This is also an important reason why edge computing and embedded machine learning have become increasingly popular in recent years.

Embedded machine learning is a field of machine learning, and these models, called Tiny Machine Learning (TinyML), are ideal for edge devices with limited memory and processing power, and non-existent or limited Internet connectivity. TinyML has now become a rapidly developing field in machine learning, and through the organic combination of hardware, algorithms, and software, it enables the analysis of sensor data at mW or less power, realizing the process of embedding AI on small hardware.

Although TinyML is a new concept, applying machine learning to smart devices is nothing new. For example, most smartphones have some kind of neural network, and music recognition and many camera modes (such as night vision and portrait mode) are examples that rely on embedded deep learning. These are all areas where TinyML comes in handy and take Edge AI a step further.

Edge AI chipsets bring AI to countless endpoints, including mobile devices, cars, smart speakers, and wireless cameras. However, these devices often cannot fully utilize all the data generated because they cannot support high computing performance and high data throughput. The emergence of TinyML makes it possible to run machine learning models on MCUs. These MCUs are generally inexpensive, small in size, with hundreds of KB of low-power memory (SRAM) and several megabytes of storage space built in, low power consumption, and wide application. The main goal of the TinyML chipset is to solve the cost and energy efficiency issues. They achieve data analysis performance on low-power, low-processing and small-memory hardware through software designed for small inference workloads. This technology has the potential to revolutionize the future of the Internet of Things.

Famous companies gather in the TinyML track

Since the birth of TinyML, the innovation market has been hot, and many products have attracted much attention. For example: Industrial AI smart camera based on NVIDIA Jetson Xavier NX, which is the industry's first industrial smart camera launched by Adlink . The camera is based on NVIDIA's Jetson Xavier NX, with high performance, small size, and about ten times the efficiency of its predecessor. It is a compact, reliable, and powerful Edge AI application product that opens the door to artificial intelligence innovation in manufacturing, logistics, healthcare, agriculture, and many other commercial fields.

TinyML focuses on optimizing machine learning workloads so that they can run on low-power microcontrollers. The surge in TinyML will lead to the expansion of Edge AI outside of traditional key markets, with more end users benefiting from smart connected sensors and IoT devices based on sound waves, temperature, pressure, vibration, and other data sources. Today, TinyML is at the intersection of machine learning and embedded IoT, with the potential to bring disruptive changes to many industries. The potential applications of TinyML are almost boundless, such as: industrial robots that can predict when service is needed, sensors that can monitor crops for harmful insects, in-store shelves that can request restocking when inventory is low, and medical monitors that can track vital signs while maintaining privacy.

Audio analysis, pattern recognition, and voice human-machine interface are the most widely used areas for TinyML today. NXP 's EdgeReady MCU-based 3D face recognition solution uses the i.MX RT117F crossover MCU to help developers quickly add 3D face recognition and advanced liveness detection to their products, even in outdoor lighting conditions. The solution's 3D liveness detection function can also identify and resist fraud using photos or 3D models, using only a high-performance 3D structured light camera module (SLM) and an optional RGB camera based on a low-cost CMOS sensor, without the need for an expensive, power-hungry, Linux-based MPU.

The i.MX RT1170 used in the solution is a cross-border MCU that uses a Cortex-M7 core with a main frequency of 1GHz and an Arm Cortex-M4 with a main frequency of 400MHz. It has excellent computing power, multiple media functions and real-time functions. Face recognition and liveness detection can be performed completely offline on the i.MX RT117F MCU without relying on the cloud, which not only eliminates latency issues but also effectively protects consumers' privacy.

Figure 1: i.MX RT117F 3D face recognition hardware block diagram

(Image source: NXP)

Figure 2: Execution flow of the food recognition model

(Source: ST)

As the IoT market expands, the amount of data at the edge grows rapidly, and AIoT, enabled by TinyML, has emerged. According to the analysis data of Markets and Markets, the AIoT market size in 2019 was approximately US$5.1 billion, and is expected to grow to US$16.2 billion by 2024, with a compound annual growth rate (CAGR) of up to 26%. The main function of AIoT is to enable networked devices to have machine learning capabilities, thereby performing complex intelligent calculations.

Infineon launched ModusToolbox ML in June 2021 , with the goal of enabling the company's PSoC MCU to have deep learning capabilities. ModusToolbox ML is a new feature based on ModusToolbox software that provides developers with the middleware, software libraries, and specialized tools required for deep learning-based ML models. ML can be seamlessly integrated with existing software frameworks in ModusToolbox, making it very convenient to integrate into secure AIoT systems. ModusToolbox ML allows developers to use their preferred deep learning frameworks, such as TensorFlow, and deploy them directly to PSoC MCUs. In addition, ML helps engineers optimize models for embedded platforms, reduce platform complexity, and provide performance verification capabilities based on test data.

Figure 3: Infineon PSoC6 MCU internal architecture

(Image source: Infineon)

The growing TinyML ecosystem

Founded in 2019, the TinyML community is a community of researchers and industry engineers dedicated to bringing ML capabilities to microcontroller devices. TinyML consists of machine learning architectures, techniques, tools, and methods that can perform analysis on various sensor modes (visual, audio, motion, chemical, and others) on low-power target devices, mainly battery-powered devices. Evgeni Gousev, one of the founders of TinyML, believes: "We are in the digital transformation revolution, and TinyML performs on-device machine intelligence and analysis at low cost, combined with inherent privacy features, providing great energy-saving advantages."

TinyML will be popularized in many industries, and it will affect almost every industry including retail, healthcare, transportation, health, agriculture, fitness, and manufacturing. At the same time, industry players have quickly recognized the value of TinyML and have taken quick action to create a supportive ecosystem.

Arm is a staunch supporter of TinyML and a leader in TinyML technology. With more than 180 billion Arm-based chips shipped, its IP, tools and more than 1,100 software partners have built billions of tiny smart IoT devices.

Today, the Arm ^® Cortex ^® -M series MCUs have become the most widely used platform for TinyML. They are able to perform real-time calculations quickly and efficiently, are inexpensive, highly reliable, responsive, and consume very little power. The Cortex-M55 processor is Arm's most AI-capable Cortex-M processor, which provides enhanced, energy-efficient DSP and ML performance. The Ethos-U55 NPU is a new ML processor called a microNPU, specifically designed to accelerate ML reasoning in area-constrained embedded and IoT devices. The Ethos-U55, combined with the AI-enabled Cortex-M55 processor, delivers 480 times the ML performance over existing Cortex-M-based systems.

In fact, at the beginning of 2021, Raspberry Pi released its first microcontroller board, which is one of the most affordable development boards on the market, priced at only $4. Called Raspberry Pi Pico, it is based on the RP2040 MCU and has a powerful dual-core Cortex-M0+ processor built in, capable of running TensorFlow Lite Micro, and we will soon see various TinyML use cases for this board.

For decision makers drowning in massive amounts of data, TinyML is like a savior, making full use of data at the edge, enabling people to get the right information faster. In addition, TinyML also improves the privacy issues that people generally worry about by processing data on the device and transmitting only key information.

Next, we will see a new world with trillions of smart devices, powered by TinyML technology, that can sense, analyze, and act autonomously, and will create a healthier and more sustainable environment for us.

This article is an exclusive original article. Please indicate the source when reprinting it. We reserve the right to pursue legal liability for unauthorized copying and non-compliant reprinting.