Embedded artificial intelligence/machine learning (AI/ML) develops differently with “ecosystem + integration + customization”

With the continuous improvement of embedded processor capabilities, the continuous introduction of ultra-miniaturized hardware accelerators, and the continuous emergence of original and commercial development environments and tools, embedded artificial intelligence/machine learning (AI/ML) technology has developed rapidly in recent years. At the same time, because these technologies are very close to various application needs, they are entering a new space for differentiated development. In the future, their growth rate will be comparable to or even exceed that of cloud-based artificial intelligence applications that require a strong resource system, are based on good communication conditions, and are based on cloud.

Artificial intelligence is not a term that has only been proposed in recent years. However, with the promotion of events such as Google AlphaGo's victory over the human Go world champion in recent years, technologies such as convolutional neural networks, deep learning and machine learning have entered the public eye, and the model of "artificial intelligence = data + algorithm + computing power" has also been widely recognized.

As a result, in many people's impressions, artificial intelligence and machine learning are new and huge algorithm applications based on Intel's latest server processors or Nvidia's GPU acceleration modules. In particular, artificial intelligence training is a resource-consuming war and has become an emerging field with a high entry threshold. Chip companies that traditionally design MCUs or SoCs have basically no chance of noble AI/ML.

However, people soon discovered that the solutions provided by the leading artificial intelligence companies include not only complex functions such as autonomous driving road condition analysis, natural language processing, rapid medical image recognition and high-frequency financial transactions, but also a larger number of lite-level applications such as license plate recognition, smart speaker wake-up word recognition, portable smart health monitoring equipment, face recognition startup and smart home security.

Driven by strong market demand, with the launch of Google's open source TensorFlow Lite embedded machine learning architecture and similar products, and the commercial use of hardware accelerators such as Imagination's PowerVR Neural Network Accelerator (NNA) on mobile or embedded devices, various embedded AI/ML functional solutions with lower power consumption and cost, as well as more compactness, are emerging.

Through analysis, Beijing Huaxing Wanbang Management Consulting Co., Ltd. believes that the widespread rise of embedded AI/ML has brought about a new model that is different from the development paradigm of traditional AI technology centered on "artificial intelligence = data + algorithm + computing power". The focus of embedded AI/ML for specific or some applications and functions has shifted to "ecology + integration + customization". Below we analyze from three aspects: integration into the IoT ecosystem, integration of hardware and commercial development tools, and development of customized processors based on RISC-V:

Bringing the latest Matter protocol and IoT ecosystem to embedded AI/ML

Silicon Labs (also known as "Silicon Labs") is a leading global provider of IoT chips, software and solutions. The company is well-known in the industry for supporting the most comprehensive IoT communication protocols and providing excellent product performance. Its customers include leading manufacturers in smart homes, smart cities, industrial and commercial, smart healthcare and energy.

Earlier this year, the company announced the launch of its BG24 and MG24 series of 2.4 GHz wireless SoCs, which not only support the latest Matter IoT communication protocol, but also support Bluetooth and multi-protocol operations respectively. They also provide artificial intelligence/machine learning capabilities for battery-powered edge devices and applications, and bring high-performance wireless capabilities and a large IoT ecosystem.

BG24 and MG24 wireless SoCs represent the industry's cutting-edge ecosystem, features, and technology portfolio, including support for wireless multi-protocols, long battery life (low power consumption), machine learning, and security for IoT edge applications. Silicon Labs' new software toolkits enable developers to quickly build and deploy AI/ML algorithms using some commonly used toolkits (such as TensorFlow).

To achieve AI/ML computing power, the BG24 and MG24 series are the first to integrate dedicated AI/ML accelerators to help developers deploy artificial intelligence or machine learning functions and solve power consumption problems. This dedicated hardware is designed to handle complex calculations quickly and efficiently. Internal tests show that its performance is improved by up to 4 times and energy efficiency is improved by up to 6 times. Because machine learning calculations are performed on local devices rather than in the cloud, network latency is eliminated and decision-making and action are accelerated.

In addition, the BG24 and MG24 series also have the largest flash and random access memory (RAM) capacity in the Silicon Labs portfolio, enabling support for multiple protocols, Matter, and training ML algorithms with large data sets. These chips are equipped with PSA Level 3 certified Secure VaultTM IoT security technology, which can provide the high security required for door locks, medical devices and other products that need to be deployed with caution.

Highly integrated embedded AI/ML with leading commercial development tools

IAR Systems is a global leader in embedded development software and services, and its leading IAR Embedded Workbench® tool chain has been widely adopted around the world. IAR Systems' development tools support Alif Semiconductor™'s highly integrated Ensemble™ and Crescendo™ series chips, creating AI-based, efficient microcontrollers (MCUs) and fusion processors to enable the next generation of embedded connected applications.

The integration of more functions represents a development direction for embedded AI/ML. These energy-efficient product lines from Alif Semiconductor provide up to 4 processing cores, as well as artificial intelligence/machine learning (AI/ML) acceleration, multi-layer security, integrated LTE Cat-M1 and NB-IoT connectivity, and global navigation satellite system (GNSS) positioning, which greatly expands its application range.

In order to make better use of these functions, it is necessary to use industry-proven, leading compiler technologies such as IAR Systems' Arm development tools to optimize code size and speed, while also providing high-performance debugging capabilities, thus providing a good platform for enterprises.

In November 2021, IAR Systems announced that its latest version of IAR Embedded Workbench for Arm® added support for the Arm Cortex®-M55 processor. This processor is a Cortex-M series processor that supports AI technology, bringing energy-saving digital signal processing (DSP) and machine learning capabilities.

The collaboration enables application developers of Ensemble or Crescendo devices to leverage the IAR Embedded Workbench® for Arm development toolchain to achieve high-performance and powerful code optimization capabilities to fully realize the AI/ML potential of the device while maintaining maximum energy efficiency.

RISC-V enables embedded AI/ML to be customized for edge applications

Diversified needs are one of the characteristics of embedded applications. MCU suppliers have long been meeting users' personalized needs by matching different processor cores with peripherals. The rise of RISC-V has brought a new trend of customized processors, which will continue to extend to the embedded AI/ML field and be supported by leading manufacturers in the industry.

Codasip is a leading provider of RISC-V processor IP and advanced processor design tools, providing IC designers with all the advantages of the RISC-V open ISA and the unique ability to customize processor IP. In February this year, Codasip launched two new low-power embedded RISC-V processor cores, L31 and L11, optimized for custom processors.

Based on these new cores, customers can easily use the Codasip Studio tool to customize processor designs to support challenging applications such as neural networks, AI/ML, including extremely small, power-constrained applications such as IoT edge computing. Codasip's core customizability is the cornerstone of its success. Currently, more than 2 billion processors around the world use Codasip's IP.

The Codasip L31/L11 embedded core runs on Google's TensorFlow Lite for Microcontrollers (TFLite Micro) and uses the Codasip Studio tool to customize a new class of embedded AI cores, which can provide sufficient performance for AI/ML compute-intensive and internal resource-limited embedded systems and other applications. Different applications also have huge differences in the requirements for devices, and existing processors cannot load AI/ML applications well.

Codasip provides a "create differentiated design" model, which means that customers using its Studio tools can customize processors according to the requirements of their specific systems, software and applications. By combining TFLite Micro), RISC-V custom instructions and Codasip processor design tools, it can bring advantages such as low latency, high security, fast communication and low power consumption to embedded and efficient edge neural network processing functions.

Looking to the future: New applications and new technologies will continue to emerge

With the development of the industry, embedded AI/ML technologies and applications will be further developed. Based on the new paradigm of "ecosystem + integration + customization" proposed by Huaxing Wanbang and the continuous innovation of edge applications, we can see that some new technologies deserve high attention in the future, such as new hardware accelerators and security technologies suitable for edge applications.