AI moves from the cloud to the terminal, driving the demand for ASICs

The trend of artificial intelligence is sweeping the world. In order to accelerate the popularization of AI applications, reduce cloud computing workloads, and realize more innovative applications, the demand for edge computing is increasing day by day. AI is beginning to move from the "cloud" to the "terminal", thus pushing up the demand for ASICs.

According to market research firm Ovum, ASIC's market share will increase significantly from 11% to 48% from 2018 to 2025. Ovum's survey report pointed out that in 2016, the cloud (including enterprises, data centers, etc.) was the main revenue area for deep learning chips, accounting for 80%. However, by 2025, this ratio will change, with the edge accounting for 80%, while the cloud will drop to 20%. The edge here refers to terminal devices, and is centered on consumer products (rather than small servers or routers), including mobile devices (mobile phones, tablets), head-mounted displays (HMDs), such as AR/VR/MR, smart speakers, robots, drones, cars, security cameras, etc.

Aditya Kaul, research director of Tractica/Ovum, said that most AI processors today, such as GPUs, are used in cloud servers and data centers to perform AI training and inference in the cloud. However, with the increasing demand for privacy and security, coupled with factors such as reducing costs, latency and breaking bandwidth limitations, distributed AI has emerged, and more and more AI edge application cases have emerged. For example, Apple's A12 Bionic chip has a new generation of "neural network engine" that uses real-time machine learning technology to change the user experience of smartphones.

Kaul pointed out that, in short, the shift of AI from the cloud to the edge is ongoing. Of course, AI on edge devices is still mainly based on inference rather than training. However, with the increase in innovative AI applications, more and more chip manufacturers are trying to improve the computing performance of terminal device processors in order to avoid sending data to the cloud for data computing, inference and training. Therefore, various processors have been introduced, such as CPU, FPGA, GPU, ASIC, NPU or SoC Accelerator.

Among them, the market share of ASIC is expected to rise significantly as the demand for edge computing increases, from 11% in 2018 to 52% in 2025. Kaul further explained that the reason why ASIC is popular is that the emerging deep learning processor architecture is mostly based on graph or Tensorflow; and the aforementioned AI edge computing is limited by power consumption and computing performance, so it is mostly based on inference rather than training. However, if it is assumed that by 2021, terminal devices will introduce a large number of AI chips, what is needed is an IC that can perform inference and training on the same chip, can cope with distributed computing and has low power consumption, so the demand for ASIC will continue to rise, realizing more AI edge application cases.