In the new stage of development of deep learning framework, we need to be vigilant against the "name and reality trap"

Recently, the series of technological blockade policies promoted by the United States have been jaw-dropping. The world technological order that we are accustomed to is undoubtedly being changed and even reconstructed at a speed visible to the naked eye.

In such a discourse atmosphere, the independent and controllable underlying technology has begun to become the consensus and driving direction of all sectors of society. In the process of the science and technology industry, semiconductors and computer operating systems represent the technical categories that were lacking in the past and are eager to fill today, that is, the technologies that we can clearly feel are "stuck in the neck" today. The other category is "future technology" that China and the United States are generally in a stage of keeping pace and need to be actively developed. The so-called yesterday's cause is today's result. If we don't want to be stuck in the neck again in the future, then today's technology must also remain independent and controllable, and even develop at a high speed to surpass Europe and the United States-AI technology is the representative of this field.

(Last year, GitHub suddenly restricted access to users in Crimea, Cuba, North Korea, Iran, and Syria.)

In the process of developing AI technology, I believe readers are already familiar with the most critical development base, which is the deep learning framework. A few years ago, we specifically discussed the strategic significance of the independence of the deep learning framework, and today it has been fulfilled one by one. At this stage of the development of domestic deep learning frameworks, many new trends have emerged. At this stage, what we need may not only be a simple benchmarking and support of the Chinese and American frameworks, but also a more intelligent, strategic, and reasonable industrial rhythm to promote the healthy development of the technology base.

In this process, blindly implementing egalitarian resource allocation and industry promotion may be the biggest problem. Entering a new stage of deep learning framework development, the appropriate industry strategy should be to expect and promote the flourishing of a hundred flowers, while also finding the best trees that can become pillars.

All trees and flowers are not scenery. Only an ecosystem with a sound internal system and a reasonable development mechanism can foster a thriving Chinese AI jungle.

The name-reality trap: What should we be wary of in the new stage of development of deep learning frameworks?

As industrial AI continues to develop, AI's underlying technical tools and platforms have received widespread attention from all walks of life and even countries around the world. However, as relevant industrial achievements continue to emerge, AI technology is a new thing and lacks industry-wide judgment standards, which leads to inconsistent expressions of government, capital, media, industry, and other fields, and great differences in understanding, which leads to different bases for subsequent actions.

For example, many technology platforms and tools that enter the public eye are basically labeled with words such as first, exclusive, and breakthrough. Coupled with the seemingly strong capital and team background, it is easy for people outside the industry to have too high psychological expectations. Whether in chips, operating systems, or AI,In the field of big data , we have seen too many similar problems, even farces. This phenomenon can be called the "name and reality trap" in the independent technology industry.

This is like iron and gold are both metals, but their values ​​are obviously different. However, due to the lack of professional knowledge of the technology industry at the general level, it is difficult to distinguish the actual differences between platforms with similar names.

This has also begun to be reflected in the development of deep learning frameworks. Since the beginning of this year, not only have the three major American frameworks, Google TensorFlow, Facebook's PyTorch, and Amazon's MXNet, been continuously updated, but many domestic deep learning frameworks have also announced that they will be open source to the industry. For example, the Oneflow deep learning framework developed by Yiliu Technology, the deep learning framework MegEngine of Megvii Technology, and so on. Of course, each framework is full of developers' dreams and efforts, and will also show unique advantages in some training tasks. But if we compare these new frameworks horizontally, we will find that they have some uneven deficiencies in core technology, tool integrity, and ecological layout, and they even fail to meet the standards of the domestic big brother "PaddlePaddle". Then if the same empowerment is given from the perspectives of policy, capital, and industrial cooperation, it will obviously cause the problem of indiscriminate flooding.

(The PaddlePaddle Panorama reflects the comprehensive core technology and complete tool set)

In fact, how to define what is a complete deep learning framework is a very professional industry issue. For example, as early as 2018, Alibaba open-sourced its X-Deep Learning, calling it "the industry's first deep learning open-source framework for high-dimensional sparse data scenarios." In fact, XDL can only meet the needs of high-dimensional sparse data scenarios, with a narrow range of capabilities, and soon disappeared among developers.

The same dilemma is particularly evident in the recently open-sourced deep learning frameworks. Technical light cavalry such as Oneflow often only target a single industry demand, and lack the scope of capabilities and tooling, ecology, and industry characteristics. As the operating system of the AI ​​era, the deep learning framework is precisely a field that requires long-term accumulation of small amounts. If the industry and developers are guided to enter indiscriminately, it will obviously cause an imbalance in internal cohesion and the inability to form an effective ecology.

So is there a way to establish a judgment mechanism for deep learning frameworks in the industry cycle?

Three-dimensional model: How to establish a value judgment mechanism for the AI ​​industry?

We certainly don’t think that old deep learning frameworks are necessarily better and new ones are worse, or that the strong will always be strong and the big ones will take all. We also don’t think that there is no need for new deep learning frameworks to appear, or that there is a lack of industrial opportunities. On the contrary, the flourishing industry is a sign of the beginning of prosperity in the industry. It’s just that we believe that whether in industrial policies, capital support, or industrial cooperation, a standardized and diversified value judgment mechanism should be established to guide the development of deep learning frameworks towards rationality and high efficiency. Avoid egalitarianism that causes waste of resources, and avoid the public from seeing innovation and independence as the same thing, which leads to the misguidance of public sentiment.

Returning to the field of deep learning frameworks, a healthy value judgment model needs to examine multiple dimensions to examine the development of a product or platform:

First, in the current scientific research environment, the value of core technology autonomy and independent control of underlying technology has become very obvious. Policymakers, industry players and the media should firmly support the development of domestic frameworks and even promote the migration of the AI ​​industry ecosystem from American frameworks to domestic frameworks. After all, under the current situation, no one can predict whether TensorFlow will be the next to be cut off.

However, it is not enough to just download and use it for support. Just like the operating system, the deep learning framework itself is very difficult, but what is even more challenging is the development and convergence of the ecosystem. Relevant policies and industry standards should also pay more attention to the AI ​​developer ecosystem and formulate targeted promotion strategies around ecological needs. In this regard, a significant indicator is the number of developers, which can intuitively reflect the degree of ecological prosperity. Take PaddlePaddle as an example. It has gathered more than 2.1 million developers and is expanding AI capabilities to various industries through its own ecological scale. At the industrial stage, it deserves higher attention.

The most important thing is that the deep learning framework should reflect the driving force for the transformation and upgrading of the national economy. According to the current policy, whether the deep learning framework can bear the industrial-grade carrying requirements brought by industrial intelligence to the deep learning framework in today's new infrastructure cycle. This standard includes several parts, such as the integrity and leading of core technologies, which can bear large-scale deep learning; on the other hand, it is the support ability of the platform system and technical capabilities for the field of industrialized large-scale production. However, the transcripts handed in by most deep learning frameworks have not yet met the requirements, either because of the narrow coverage or because the companies are dissatisfied. If you look at PaddlePaddle, it has covered many industries and fields such as communications, electricity, urban management, people's livelihood, industry, agriculture, forestry, and public welfare. It has also been put into use during the epidemic. At present, 90,000 companies have created more than 295,000 models through PaddlePaddle. These industrial indicators are obviously the best criteria for judging the value of deep learning frameworks, and they are also the direction that latecomers urgently need to break through.

As deep learning frameworks flourish, we hope to avoid trying to do everything at once and arrive at a support strategy and industry guidance behavior with clear priorities and industry logic.