Intel's Zhang Yu: How the Internet of Things can embrace the era of software-defined platforms

Recently, at Intel's annual Artificial Intelligence and Internet of Things Ecosystem Partner Summit, Dr. Zhang Yu, Chief Engineer and Chief Technology Officer of Intel China's Internet of Things Division, introduced Intel's multi-application integration and practice in the fields of AI and Internet of Things, and the integration of several applications around Intel's single platform. This is also Intel's greatest effort in responding to the AIoT era.

"In the context of digital transformation, IoT devices will have more and more functions, and most of these functions need to be implemented through software. The problem and challenge now is how to share several software on a single platform," said Zhang Yu. "For IoT platforms, the open platform of general-purpose processors has become the mainstream architecture of IoT devices. Coupled with the maturity of virtualization software, we can share a hardware platform for several different applications, while providing independence for the applications to ensure quality."

Zhang Yu said that driven by Moore's Law, the improvement in computing performance is amazing. For example, the supercomputer performance in 1994 was 1,300 times per second, and now the computing power of the supercomputers on the latest list has reached 2 billion times per second, an increase of more than a million times. While computing power is increasing, algorithms are also constantly expanding, and even the iteration speed has long exceeded the hardware itself. For this reason, the concept of software-defined systems frequently appears in the Internet of Things. Zhang Yu pointed out that the essence of software-defined systems is to digitize and standardize hardware systems and realize various virtualized and diverse platforms through software empowerment.

The foundation of a unified platform or software-defined system is that the hardware platform has high enough performance and is flexible enough. For example, in the industrial field, applications such as machine vision, motion control, and human-machine interface can already be implemented on a unified platform. In the field of smart cities, in addition to advertising push, electronic billboards also have video acquisition, video analysis, city hotspots, and more functions, and have also spawned more applications and services. In the field of transportation, virtualization technology can be used to achieve software-defined cockpits, integrating vehicle infotainment systems, integrated instrument panels, and rear-seat entertainment systems on a unified platform.

Key technologies of software-defined systems

Virtualization is the most critical element of the software-defined platform, which includes the virtualization of hardware resources and platform-based system software.

Virtualization of hardware resources is the abstraction of hardware to achieve the reallocation and reconstruction of hardware resources and improve the utilization of hardware resources. Platform-based system software needs to realize the unified allocation of hardware resources through software and provide a standardized middle layer to break the relationship between upper-level applications and underlying hardware resources, thereby improving the utilization of the system.

Today, concepts such as software-defined networking (SDN), network function virtualization (NFV), and software-defined storage (SDS) have become a reality. In fact, these emerging applications use virtualization technology to integrate several applications on a unified platform to improve the efficiency of the entire platform and reduce the overall application cost of the system.

Virtualization technology is inseparable from the virtual machine monitor. This technology realizes the conversion between physical hardware resources and virtual resources, such as the conversion from physical CPU to virtual CPU, physical storage to virtual storage, and physical network to virtual network. At the same time, the virtual machine monitor can also monitor the running status of the virtual machine created on it. There are two types of virtual machine monitors in the industry, namely Type 1 and Type 2. The characteristic of Type 1 is that it can directly operate on hardware resources without the support of the underlying operating system, which can achieve faster response and reduce response delay. It is very suitable for scenarios with high real-time requirements, including industrial control. Type 2 virtual machines need to be built on top of the existing host operating system to access hardware resources.

In addition to virtual machine monitors, containers are also a new technology that has become popular in recent years. Container technology can be simply understood as a lightweight virtual machine. Developers can package applications and related dependencies in containers and publish them through containers. Different containers share the host operating system, so there is no need to establish a separate operating system for each container.

In comparison, virtual machines can virtualize operating systems, so they have better isolation. Containers have poorer isolation, but they take up less system resources and start up faster.

Software-defined systems cannot do without the support of underlying hardware, such as CPU virtualization, I/O virtualization or network virtualization, and must use virtualization technologies such as VTd, VTc, VTx, etc. "Hardware virtualization technology is mature, but with the development of artificial intelligence, these hardware are needed to support artificial intelligence applications, so Intel launched the HDDL (VAD) accelerator card, which integrates 8 Movidius Myraid X chips, provides 8T computing power, is based on the PCI-e bus, and has a total power consumption of only 20 watts." Zhang Yu introduced.

The accelerator card is equipped with OpenVINO software, which can help developers quickly deploy artificial intelligence networks to the accelerator card.

Intel's Multi-Convergence Practice

Zhang Yu said that Intel is currently developing multi-application fusion software, which includes two aspects. One is focused on the edge side, which is an extension of Intel's existing tools such as OpenVINO to better support multi-application environments. The other end is a cloud solution that provides a software management platform reference practice, which can configure system parameters on the web and automatically generate installation scripts. Developers can directly deploy it to the local multi-application environment for operation. At the same time, we also provide reference practices including application stores.

OpenVINO is a project that Zhang Yu emphasized. OpenVINO includes two core components, one is the model optimizer and the other is the inference engine. The model optimizer can help developers download the network model trained on the artificial intelligence framework to the corresponding platform. The workflow of the model optimizer is as follows: Through the model optimizer, the model is optimized while ensuring the quality, and the optimization results are converted into intermediate representation files. The intermediate representation files are read by the inference engine. After the reading is completed, they can be downloaded to the specified platform for execution through the hardware plug-in. The current hardware plug-ins include CPU plug-ins, FPGA plug-ins, GPU plug-ins, and Movidus' Visual plug-ins.

OpenVINO also includes a resource scheduler that can assign artificial intelligence networks to different VPU chips for execution. If two applications use the same artificial intelligence network, they can share VPU resources and only need to download once, thereby reducing the amount of model data transmitted.

OpenVINO cannot achieve hardware isolation between different applications. If isolation is required, certain expansions are needed. The resource scheduler can maintain the internal resource distribution and allocate the required hardware to the application. It can also record the number, type and location of resources allocated to different applications.

At the same time, HDDL supports flexible configuration and isolation. For example, 4 chips are allocated to application 1 and 4 chips are allocated to application 2 to realize different artificial intelligence networks. At this time, if the application requires more model processing, load consolidation can be used to integrate VPU resources and free up new model applications. Using 8 chips, physical isolation between different applications is achieved while resource sharing is also achieved.

Intel provides a graphical UI to facilitate user management system, graphical monitoring software on the edge side to display the application running status, and graphical configuration tools on the cloud. You can easily complete the setup step by step according to the system requirements.

Zhang Yu gave several actual development screenshots on site to demonstrate the efficiency of Intel's software-defined platform methodology, including script installation, virtual machine selection, resource configuration, container settings, system monitoring, and other actions.

"The multi-task fusion software tools we are currently working on are aimed at helping developers more easily build multi-application fusion systems. Specific measures include providing more management tools to facilitate customers in load integration and monitoring, optimizing software to maximize the overall performance of the system, while also ensuring the isolation of the system," Zhang Yu concluded.