What you should know about AI edge computing!

Source: Content from "New Electronics", thank you.

Artificial intelligence (AI) is developing at an increasingly rapid pace and has begun to make inroads into terminal devices. Computational analysis has begun to shift from the cloud to terminal nodes. The development of edge computing can be said to be a hot topic in the current semiconductor industry. The 2018 Taipei International Computer Show (Computex 2018) has also become an excellent occasion for companies in various fields (such as IP, chips, and storage) to showcase their strengths, and they have released new solutions or market layout plans during the exhibition.

Rene Haas, president of Arm IP Products Group (Figure 1), said that with the rapid development of the Internet of Things, Arm predicts that by 2035, there will be more than 1 trillion connected devices in the world, used in medical, automotive, lighting and road fields, and the substantial growth in the number of connected devices will also lead to the continued development of terminal and cloud computing. Intelligent computing will continue to promote a new era of the Internet of Things, lead the AI ​​revolution, and make intelligent computing in the Internet of Things ubiquitous.

Figure 1 Rene Haas, president of Arm IP Products Group, said that the booming development of the Internet of Things will lead to a rapid increase in connected devices, and intelligent computing will be ubiquitous in the future.

In response to this trend, and to integrate the AI/machine learning (ML) applications, calculations and frameworks of the ecosystem, and combine software optimization with hardware IP products, so that various devices and platforms can support the most commonly used machine learning frameworks, Arm recently announced the launch of three new IP products, namely Cortex-A76 CPU, Mali-G76 GPU, and Mali-V76 VPU, to enhance gaming and AR/VR experience, AI and machine learning capabilities. Through these three new products, Arm will continue to strengthen the company's competitive advantage in the mobile field, and once again enhance the computing performance of mobile terminal devices such as smartphones, tablets, and PCs.

Nandan Nayampally, vice president and general manager of the client business unit at Arm (Figure 2), said that 5G will drive innovation in the entire mobile industry in the future. The upcoming 5G network applications, including VR, AI, or mobile games, will drive more computing growth, and there will be more different computing needs in the future.

Figure 2 Nandan Nayampally, vice president and general manager of the Client Business Unit at Arm, pointed out that 5G plus AI will drive innovation in the entire mobile industry. Arm has launched new IP products to meet market demand.

Nayampally further pointed out that gaming is also one of the key factors driving the continued rise in mobile computing. The gaming industry has become one of the world's largest revenue markets, with an estimated output value of US$137.9 billion in 2018, which has also driven consumers' demand for computing performance.

It is reported that the Cortex-A76 is built based on Arm's DynamIQ technology. Compared with the Cortex-A75 released last year, it has improved performance by 35% and efficiency by 40%. It can provide 4 times the computing performance for AI/ML on terminal devices, and achieve a fast and secure experience on PCs and smartphones.

Mali-G76 improves computing performance by 30% and increases performance density by 30% over the previous generation Mali-G72 GPU. It not only meets consumers' needs to play high-end games at any time, but also provides developers with more performance space, allowing them to write more new applications, bring more high-end games to mobile applications, or integrate AR/VR into life.

Finally, as the demand for UHD 8K increases, to ensure that the IP can support encoding and decoding operations on smartphones and other devices, Arm launched the Mali-V76, which can support 8K resolution up to 60fps or four 4K streaming videos at 60fps. Consumers can stream four 4K resolution movies at the same time, record videos in video conferences, or watch four games in 4K; or at a lower resolution, it can still present high-resolution image quality (Full HD) and support up to 16 streaming videos to form a 4×4 TV wall.

At the same time, in order to improve the performance of machine learning in terminal devices, Arm also released the Project Trillium platform in early 2018, including a new machine learning processor (ML Processor), an object detection processor (Objects Processor), and Arm neural network software (Arm NN). Compared with independent CPUs, GPUs and accelerators, the performance of the Project Trillium platform far exceeds the programmable logic of traditional DSPs.

Jem Davies, vice president, academician and general manager of the Machine Learning Business Group at Arm (Figure 3), pointed out that the development potential of edge computing is huge, and there are indeed many independent solutions on the market, such as ASIC accelerators, CPU/GPU, etc. Terminal operators can certainly choose the solution they want, but the disadvantage is that they have to spend time integrating hardware and software (TensorFlow, Caffe) by themselves.

Figure 3 Jem Davies, Arm vice president, academician and general manager of the machine learning business group, believes that Project Trillium is expected to create a complete machine learning ecosystem for terminal devices.

Davies explained that the advantage of Project Trillium is that it is presented in the form of a platform architecture. In terms of hardware, not only are there ML Processors and Objects Processors to choose from, but Arm NN software can also be used to help users simplify the connection and integration between neural network frameworks such as TensorFlow, Caffe and Android NN and Arm Cortex CPUs, Arm Mali GPUs and machine learning processors.

Davies further pointed out that software integration is one of the key elements in the development of machine learning. Many accelerator companies may be able to provide relevant hardware processors (CPU, GPU), but few have the resources to provide a complete platform architecture to assist customers in software and hardware integration or to improve ML model operations. Project Trillium includes new Arm IP processors and neural network software, which can meet current market needs from both hardware and software aspects. This approach will also help Arm build a complete edge computing ecosystem.

In addition, Davies also observed that the demand for machine learning in MCUs is also very strong. He revealed that on the first day when Project Trillium was launched and the Arm NN software development kit was available for download, more than 5,000 users began using CMSIS NN to try to execute machine learning algorithms on Cortex-M.

Davies said that this result was actually beyond Arm's expectations, and it also shows that the demand and interest of the MCU user group for machine learning cannot be ignored. This also prompted Arm to decide to further enhance the efficiency of this type of core in executing ML algorithms in the new version of the Cortex-M core to be launched in the future.

CMSIS NN is a compute library under the Arm neural network software development kit Arm NN SDK, which can improve the efficiency of Cortex-M in executing machine learning algorithms. Even the existing Cortex-M core can perform some very simple machine learning inferences with the help of CMSIS NN, such as judging the meaning of sensor output data. Of course, since the MCU's computing performance and memory space are not very abundant, it is impossible to perform very complex machine learning inferences, but if a simple interpretation of the data output by a single sensor node is made, it is still possible.

Davies pointed out that if MCUs cannot support certain basic ML algorithms, the future of ubiquitous AI applications will be difficult to achieve. Currently, the artificial intelligence application services provided through cloud data centers actually have obvious application limitations. Only by continuously pushing AI to the edge can AI applications become more popular. In order to enable MCUs to execute ML algorithms more efficiently, the efficiency of Cortex-M in executing ML will be further improved in Arm's future product development roadmap.

On the other hand, the automotive industry will also be one of the key application areas of edge computing in the future. According to Arm's forecast, by 2020, an average car will be embedded with more than 200 sensors and processed by more than 100 engine control units (ECUs) or microcontrollers (MCUs). How to quickly process such a large amount of data, respond in real time, and maintain system stability and security, and build an autonomous driving car that meets user needs, will become a major challenge for the future automotive electronics market.

In this regard, John Ronco (Figure 4), vice president of Arm and general manager of the Embedded and Automotive Business Unit, pointed out that the rise of edge computing means that terminal devices no longer need to send large amounts of data back to the cloud for processing, but this also means that general CPUs or machine learning chips require higher processing performance, which is why Arm launched Project Trillium and Cortex-A76, and these products are also very suitable for use in automotive electronic components.

Figure 4 John Ronco, vice president of Arm and general manager of the Embedded and Automotive Business Unit, said that processors such as CPUs and GPUs must have higher performance to meet the safety requirements of autonomous driving.

In addition, to achieve autonomous driving, a car is often equipped with visual sensors in addition to radar and lidar, and therefore requires a higher GPU to cope with the huge image calculations.

Ronco said that the visual computing requirements of autonomous driving are different from those of general IP network cameras in that most IP network cameras have a single lens and are not often moved, usually placed in a corner inside or outside the house for monitoring. However, for cars, they need to be equipped with several cameras to detect road conditions and the environment. The image information received is very large, and because the car is always moving, the surrounding scenery will also change constantly, which will make the calculation more complicated, so a complete solution is needed.

Ronco revealed that the object detection processor in Project Trillium is mainly used for IP network cameras. To meet the visual computing needs of automobiles, it is necessary to rely on high-efficiency GPUs such as Mali-G76, which have higher computing performance and can respond to rapid environmental changes when the car is driving and avoid accidents.

In short, the AI ​​era brings new business opportunities to various application fields, and edge computing is bound to enter the automotive industry. However, if edge computing is to be built into cars, higher-level technologies must be embedded to achieve better performance, making cars smarter, safer, and more efficient.

The rise of edge computing has not only increased processor performance, but also storage demand, so storage companies have accelerated their product layout. Christopher Bergey, vice president of Western Digital's embedded application solutions division (Figure 5), pointed out that technologies such as edge computing and machine learning have made storage and computing quite complex.

Figure 5 Christopher Bergey, vice president of Western Digital's embedded application solutions division, said that in response to the edge computing market, the company's one-stop production model is a market competitive advantage.

Bergey further explained that edge computing will have different requirements for storage products depending on different application scenarios. For example, in automobiles, special attention is paid to temperature and reliability. In recent years, cost and stable supply for five years have been added as considerations. In mobile device applications, taking smartphones as an example, consumers now have higher and higher requirements for taking photos. As the pixels of photos increase, the storage capacity of mobile phones must also increase. The demand for edge storage will also increase, so the performance of related embedded flash memory (EFD) products will also improve accordingly.

In response to this trend, Western Digital has launched a new iNAND product series - iNAND8521/iNAND7550, which uses the company's 64-layer 3D NAND technology and advanced UFS and e.MMC interfaces to provide better data performance and huge storage capacity. When used in smartphones and thin computing devices, these two products can accelerate the realization of various data-centric applications, including augmented reality (AR), high-resolution video capture, social media experience, and the recently emerging AI and IoT edge experience.

Bergey revealed that the development trend of mobile devices in the future will undoubtedly move towards higher performance, because after the arrival of the 5G era, data transmission will become faster and faster, and there will be more and more innovative applications. In addition, with the rise of AI, the combination of the two will increase the workload requirements and storage capacity. The company will also continue to work closely with mobile phone manufacturers to provide suitable products according to needs.

Bergey also pointed out that the company actually has a good strategic advantage in responding to the development of edge computing. The reason is that WDC has a complete product line (from low-end products to high-performance products). In addition, WDC adopts a one-stop production strategy, from wafers, controllers, firmware and software, etc., all of which are self-responsible. Therefore, it can quickly launch products in response to market changes or meet the customization needs of equipment manufacturers. This is WDC's advantage in the fiercely competitive edge computing market.

As for NXP, it starts with security and works with ecosystem partners such as NEXCOM, IMAGO, Accton Technology, and Senzhun Technology to jointly invest in the deployment of edge computing security infrastructure to support emerging AI and machine learning connected at the edge, as well as secure edge processing deployed in the cloud.

The system suppliers will develop products based on NXP's Layerscape and i.MX application processor families to meet various applications that require local processing power and cloud connectivity. The developed platform provides a perfect balance between computing power, connectivity and storage capacity, suitable for both enterprise and industrial environments.

NXP's EdgeScale technology and open source software for Docker and Kubernetes enable a variety of edge applications to run on common cloud architectures, including Amazon Web Services (AWS), Greengrass, Google Cloud IoT, Microsoft Azure IoT, Alibaba and private cloud architectures.

NXP pointed out that EdgeScale is a suite of devices and cloud services that can simplify the deployment of secure computing resources at the edge of the network. NXP will work with these partners to provide scalability, security, and ease of deployment for IoT and on-premises computing platforms to achieve secure deployment and management.

Tareq Bustami, senior vice president and general manager of NXP Digital Networking, said that building secure edge solutions is extremely important for the successful development of the Internet of Things and Industry 4.0. Therefore, the company is committed to working with many equipment manufacturers to provide easy-to-use and cloud-connected secure edge computing solutions. Through cooperation, the company will help launch smarter and more functional edge solutions, adding powerful security features that can be deployed and managed on a large scale.

In summary, it can be seen that whether it is IP vendors, storage companies or chip suppliers, they are all actively deploying in the edge computing market, developing open platforms and hardware architectures, hoping to allow AI to enter various terminal devices and build a complete ecosystem.

Today is the 1635th issue of content shared by "Semiconductor Industry Observer" for you, welcome to follow.