Utilization rate is less than 15%: computing power shortage or computing power surplus?
Latest update time：2024-11-18
Reads：
Author | Zhao Yan
Source | Communications Industry Network
In November 2022, GPT3.5 and ChatGPT, which were trained based on the A100 GPU series, quickly attracted the world's imagination of AI and started the AI competition. After that, it was hard to get a card. Now, the rental price of NVIDIA's two popular chips has dropped by 50% in 10 months. Many industry chain insiders said that the price of computing cards is close to the sales cost, and the shortage of AI computing power supply has eased. At the same time, by July 2024, 140 intelligent computing center projects appeared in China, triggering thoughts about excess computing power.
"The overall lack of computing power is indeed a problem, but at the same time, the structural shortage of computing power is also an objective reality. That is to say, while the supply of computing power is tight, a large amount of computing power is not effectively utilized," said An Hui, deputy chief engineer of CCID Research Institute.
Three reasons for insufficient computing power supply
At present, there is a problem of mismatch between supply and demand in my country's computing power development. Although the scale of computing power continues to grow, there is a large gap in computing power for high-end applications such as artificial intelligence and high-performance computing. According to relevant reports, in 2023, China's demand for intelligent computing power will reach 123.6EFLOPS, but the supply scale will only be 57.9EFLOPS, and the gap between supply and demand is significant.
In the process of using computing power, due to the imbalance and mismatch between the existing supply structure and the actual computing power needs
of users, there is a mismatch between supply and demand, which also leads to a large amount of idle and wasted computing power. Among them, there are many aspects of misalignment. For example, category misalignment, domestic computing power industry chain companies are relatively scattered, and most chip manufacturers and AI technology companies have different technical paths, resulting in a mismatch between chips and AI applications; spatial misalignment, such as the excess computing power in central and western China and insufficient application demand, leading to supply imbalance.
The factors that cause the shortage of computing power supply mainly include the development of AI, regional imbalances, and lack of core technologies. With the widespread application of large AI models, the original general computing power can no longer meet the needs of high-end and complex application scenarios.
Advanced computing power such as intelligent computing power has become a new demand hotspot, but the supply has not kept up.
The differences in economic development levels and industrial structures between regions also lead to differences in computing power demand. Due to the high degree of digitalization in the eastern region, the demand for computing power is more vigorous, while the supply is relatively insufficient, especially during peak hours or in specific application scenarios, the computing power shortage problem is particularly prominent.
The technological gap in high-performance computing and AI hardware in China is also one of the important reasons for the shortage of computing power supply.
As the core component of computing power infrastructure, the cost of computing power chips accounts for the vast majority of server costs. However, in the current computing power chip field, Chinese manufacturers have a certain gap in R&D capabilities compared with foreign manufacturers, and the initial development costs are relatively high. This not only leads to expensive computing power services, but also makes it difficult to increase computing power supply rapidly. AI hardware such as high-performance GPUs mainly rely on imports, which increases the uncertainty of computing power supply. Once the import channels are blocked or prices fluctuate, it will directly affect the stability of computing power supply.
Low computing power utilization
Corresponding to the insufficient supply of computing power is the problem of low computing power utilization efficiency. According to IDC data, the utilization rate of general computing centers with enterprises as the main users is
currently only 10%~15%
, which shows that the utilization rate of small or enterprise-level computing centers is relatively low. The resource utilization rates of national and large-scale intelligent computing centers also vary. For example, the computing power utilization rate of the artificial intelligence computing center of Xi'an Ascend Intelligent Technology Co., Ltd. is as high as 98.5%, and the resource utilization rates of the National Supercomputing Center in Shenzhen and the National Supercomputing Center in Jinan are also high. However, these only account for a small number of computing centers in China.
Industry insiders analyzed that most computing power centers are still facing problems in the three dimensions of "construction, application, and ecology" such as "scale first, single architecture; focusing on hardware development and neglecting software construction, and separation of technology and scenario requirements; lack of compatibility and coordination." As a
result,
nearly 50% of China's computing power centers have uneven distribution and management of computing power, and it is difficult to keep up with utilization rates.
Blind investment is one of the reasons for the excess computing power.
Some local governments and enterprises blindly invested in the construction of intelligent computing centers. After the completion of these intelligent computing centers, there was a lack of sufficient application scenarios and market demand, resulting in a waste of computing resources. At the China High Performance Computing Academic Annual Conference held in September this year, Chen Jian, vice president of the China Computer Federation (CCF), pointed out that if hardware resources are simply increased without considering their practical performance, and if the demand side is not ensured to obtain high-quality computing power conveniently and quickly, then such an approach is tantamount to a huge waste of resources and financial resources. He said that at present, except for the single large-scale computing power cluster that is in short supply, the computing power supply for AI reasoning can actually meet or basically cover the demand. The key lies in how to develop and utilize existing resources more effectively.
In addition, the computing power leasing market is sluggish, resulting in the ineffective use of computing power resources.
Some companies have idle computing power resources due to lack of application scenarios, resulting in waste. For example, a cloud computing platform invested a lot of money in the construction of a high-performance computing cluster in the early stage, but due to insufficient market demand, the computing power resources of the cluster have been idle for a long time. In order to reduce operating costs, the platform had to rent out part of the computing power resources to third parties, but the rental price was far below the cost price, making it difficult to make a profit.
Is excess computing power becoming the norm?
Is excess computing power becoming the norm? Industry insiders analyzed that judging from the current market conditions and price trends,
the computing power market is generally in excess.
This oversupply could lead to increased price competition in the coming years, affecting the profit margins of some computing power providers.
At the specific application level, such as the training and reasoning of large language models (LLM), the current computing power supply basically meets the demand. Some planned computing centers may not be fully put into use in the end, resulting in a waste of resources. In terms of price trends, the current price of computing power cards is close to or lower than their sales cost, indicating an oversupply in the market. This price trend is expected to continue, resulting in some computing power suppliers facing price reduction pressure.
The difficulty in connecting computing power supply and demand is the main bottleneck for the future development of computing power.
Affected by China's economic and social development pattern, the computing power resources deployed dispersedly across the country cannot fully match the more intensive computing power demand in the eastern region. At the same time, the constraints of the computing power nodes' ability to flexibly and efficiently allocate resources through the network have led to improper connection between supply and demand, resulting in the coexistence of "idle resources" and "difficulty in obtaining computing power".
Recently, Lin Yuan, CEO of QingCloud Technology, said that everyone was worried about whether there would be an oversupply of computing power. In the short term, supply and demand will fluctuate, and all applications will take time to land. But in the medium and long term, there will be demand for the previous round of supply. The development of AI has stages. The earliest stage was relatively simple and crude, which was to solve the problem of computing power. Since the fourth quarter of last year, customers have changed. Customers want more than just resources. They need a platform to manage resources.
In the longer term, computing power will become an important factor in driving economic development, just like water and electricity, and this will be irreversible. Demand and supply have entered a state of mutual promotion. The abundance of computing power will make the model progress faster, and the progress of the model will require more computing power.
Local, short-term, and structural computing power surplus or shortage will become an episode in the development of AI.
Note: The cover image is from the Douban movie "Dunkirk" stills
-END-
The content of this article is for communication and learning purposes only and does not constitute any investment advice. Some pictures are from the Internet and the copyright ownership has not been verified. It is not for commercial use. If there is any infringement, please contact us at info@gsi24.com.
▼
Review of past highlights
▼
京公网安备 11010802033920号