Xiong Dapeng, CEO of Yizhu Technology: Welcome the new turning point of computing power growth with AI chip architecture innovation

October 16, 2024 - At the SEMiBAY2024 "HBM and Memory Technology and Application Forum", Xiong Dapeng, founder, chairman and CEO of Yizhu Technology, delivered a speech entitled "Beyond the Limits: Technical Challenges and Solutions Facing High-Computing Power Chips".

Dr. Xiong Dapeng proposed that driven by AI big model technology, computing power is reaching a turning point in demand, and hardware architecture will become one of the key paths to meet computing power demand. Future computing power growth will be centered on storage units.

Opportunities and Challenges in the Era of Big Models

In the era of AI big models, with the continuous improvement of data, computing power, and parameter volume, model capabilities have been significantly enhanced. Dr. Xiong Dapeng pointed out that big models have gradually evolved from quantitative changes to qualitative changes. When the model size is large enough, emergent capabilities similar to human "enlightenment" will appear, and the reasoning ability of big models will be significantly improved. This change indicates that the last mile of AI application is about to be opened up, and the implementation of business will drive the demand for AI computing power to a turning point.

Omdia's latest report, "Cloud Computing and Data Center Artificial Intelligence Processor Forecast", shows that the market size of GPUs and other acceleration chips used for cloud computing and data center artificial intelligence has grown from less than $10 billion in 2022 to $78 billion in 2024, and is expected to reach $151 billion by 2029. However, the market may see a clear inflection point in 2026, and the growth momentum will shift from technology adoption to changes in demand for artificial intelligence applications.

In addition, IDC predicts that future AI servers will focus on improving computing power and processing efficiency (energy efficiency ratio) to adapt to more complex and larger-scale AI applications. It is expected that by 2027, the proportion of AI computing power used for reasoning will reach 72.6%, and in the future it is expected to reach 95% for reasoning and 5% for training.

Application implementation requires hardware architecture breakthroughs

However, the speed of improving the performance of existing chip hardware can no longer meet the rapidly growing computing power requirements of algorithm models. Moore's Law, the golden rule that once guided the development of the semiconductor industry, is now facing unprecedented challenges. A report from the Economic Research Institute of Guosen Securities pointed out that the parameter scale of large models increases 35 times every 18 months, while chips under Moore's Law only increase by 2 times. Therefore, exploring and developing new hardware architectures has become one of the key paths to breakthroughs in computing power.

Dr. Xiong Dapeng emphasized that under the existing hardware architecture, AI chips are currently facing the "three wall" problem: storage wall, energy consumption wall and compilation wall. The storage wall refers to the problem that the data access speed of the memory cannot keep up with the data processing speed of the computing unit, resulting in a performance bottleneck.

At the same time, the existence of the storage wall brings about the problems of energy consumption wall and compilation wall. The energy consumption wall refers to the fact that as chip performance improves, energy consumption and heat dissipation issues become the main factors limiting further performance improvement. The compilation wall refers to the fact that as the complexity of AI models increases, the amount of data and computing tasks that the compiler needs to process also increase dramatically, which makes static compilation optimization very difficult, and manual optimization consumes a lot of time and cost.

Storage and computing integration opens up the second growth curve of computing power

Faced with this challenge, Yizhu Technology chose to innovate and used a new chip design concept, the "storage-computing integrated super-heterogeneous" architecture, which greatly reduced the delay in data transfer and improved the overall computing efficiency and energy efficiency.

Dr. Xiong Dapeng pointed out that if we want to break the "three walls" of AI chips, we need to start from the first principle of computing power (Amdahl's law) and significantly reduce the amount of data moved so that the F value is close to 0, so as to ensure the linear growth of effective computing power density. Currently, there are two main solutions in the industry: one is in-memory computing, and the other is near-memory computing.

In-memory computing integrates storage and computing functions to reduce data transfer latency and improve performance and energy efficiency. In an ideal state, F=0, which enables seamless integration of storage and computing. Near-memory computing integrates storage units and computing units through advanced packaging to increase memory access bandwidth, reduce data transfer latency, and improve overall computing efficiency.

Dr. Xiong Dapeng emphasized that through technologies such as storage-computing integrated architecture, we can break through the bottleneck of traditional computing models, achieve higher effective computing power, and break the ceiling of effective computing power. In the future, the era centered on computing power units will come to an end, and the second growth curve of computing power will be centered on storage units.

Conclusion

Dr. Xiong Dapeng said that since its establishment, Yizhu Technology has always been committed to providing a new path for the development of AI high-computing chips that are more cost-effective, more energy-efficient, and have greater computing power development space through storage and computing integration. In March 2023, in the face of AI computing challenges brought by large models such as ChatGPT, Yizhu Technology first proposed "storage and computing integrated super heterogeneity", providing a new idea for the development of AI high-computing chips in the era of large models.

In the future, with the continuous advancement of AI technology, the demand for computing power is also growing. Yizhu Technology will provide a new direction for the development of AI chips through an innovative storage-computing integrated architecture. In the era of large models, Yizhu Technology's technology and products will provide strong support for the development of AI technology and drive the entire industry forward. With the continuous maturity of Yizhu Technology's technology and the continuous expansion of its applications, we have reason to expect that AI chip technology will usher in a new stage of development and make greater contributions to scientific and technological progress!