Nvidia’s six major customers develop self-developed AI chips

According to technology media The Information yesterday (7th), citing people familiar with the matter, six major technology companies: Microsoft, OpenAI, Tesla, Google, Amazon, and Meta are developing their own AI chips, which are expected to be integrated with Nvidia’s flagship Class microprocessor H100 GPU competition, and may help reduce costs and reduce reliance on Nvidia AI chips.

NVIDIA is now the well-deserved "King of AI computing power". The A100/H100 series chips occupy the top position of the pyramid. However, users face NVIDIA's dominance and suffer from high costs. According to analysis by Wall Street investment bank Bernstein, the cost of a single ChatGPT query is about 4 cents (approximately NT$1.2). If the search volume is to increase to one-tenth of Google's, it will cost about 48.1 billion per year. dollars of GPUs, and about $16 billion in chips to keep them running. Therefore, whether it is to reduce costs, reduce dependence on Nvidia, and improve bargaining power, technology giants have begun to develop their own AI chips.

Microsoft plans to unveil its first chip designed for artificial intelligence (AI) at its annual developer conference next month. In fact, The Information has reported that Microsoft has been developing a dedicated chip code-named Athena since 2019 to power large language models. The chip will be manufactured by TSMC, using an advanced 5nm process, and is planned to be launched as early as next year.

Some analysts said that developing a chip similar to Athena may cost about US$100 million per year. The daily operating cost of ChatGPT is about US$700,000, most of which comes from expensive servers. If the Athena chip has equal competition with Nvidia's products, Power, the cost of each chip will be reduced by one-third.

OpenAI is also exploring manufacturing self-developed artificial intelligence chips and has begun evaluating potential acquisition targets. According to reports, OpenAI has been discussing various solutions to solve the shortage of AI chips since at least last year. OpenAI has worked more closely with other chipmakers, including Nvidia, as well as diversifying its suppliers beyond Nvidia.

Tesla, the electric car manufacturer, is based on intelligent driving. It has launched two self-developed chips, namely fully autonomous driving (FSD) and Dojo D1 chip. The FSD chip is used for the autopilot system on Tesla cars; the Dojo D1 chip is used for the Tesla supercomputer Dojo chip, which is a general-purpose CPU designed to accelerate the training and improvement of Tesla's autopilot system. .

As early as 2013, Google had secretly developed a chip focusing on AI machine learning algorithms and used it in its internal cloud computing data center to replace Nvidia's GPU. This self-developed chip "TPU" was made public in 2016 and can perform large-scale matrix operations for deep learning models, such as models used in natural language processing, computer vision and recommendation systems. Google has actually deployed the AI ​​chip TPU v4 in its data center in 2020. However, the technical details were not disclosed for the first time until April this year.

Since Amazon launched its first Nitro1 chip in 2013, AWS has been the first cloud vendor to get involved in self-developed chips. It already has three product lines: network chips, server chips, and self-developed chips for artificial intelligence and machine learning. AWS's self-developed AI chip layout includes inference chip Inferentia and training chip Trainium. In early 2023, Inferentia 2 (Inf2), specially built for artificial intelligence, was released, which triples the computing performance and increases the total accelerator memory by a quarter. It can support distributed reasoning through direct ultra-high-speed connections between chips, with up to Supports 175 billion parameters, making it a strong contender for large-scale model inference.

Until 2022, Meta mainly used a combination of CPUs and custom chips designed to accelerate AI algorithms to run its AI workloads, but CPUs were often not as efficient as GPUs. Later, Meta canceled its plan to launch large-scale custom chips in 2022 and instead ordered billions of dollars worth of Nvidia GPUs. To reverse the situation, Meta is already developing in-house chips and announced the AI ​​training and inference chip project on May 19. According to reports, the chip consumes only 25 watts of power, accounting for a small portion of the power consumption of chips from market-leading suppliers such as NVIDIA, and uses the RISC-V (fifth-generation reduced instruction processor) open source architecture.

*Disclaimer: This article is original by the author. The content of the article is the personal opinion of the author. The reprinting by Semiconductor Industry Watch is only to convey a different point of view. It does not mean that Semiconductor Industry Watch agrees or supports the view. If you have any objections, please contact Semiconductor Industry Watch.

Today is the 3549th issue shared by "Semiconductor Industry Observation" with you. Welcome to pay attention.