Artificial intelligence chips consume too much power

Latest update time：2024-07-11

Reads：

????If you hope to meet more often, please mark the star ?????? and add it to your collection~

Source: The content is compiled from hpcwire by Semiconductor Industry Observer (ID: ic bank ), thank you.

Generative AI will consume a larger portion of the world’s electricity to meet the massive hardware demands required to run the applications. “In the next five years, AI chips will account for 1.5% of global electricity use, accounting for a significant portion of global energy,” semiconductor research firm TechInsights said in a research note last month.

TechInsights benchmarked total global electricity consumption of 153,000 TWh between 2025 and 2029, using data from the U.S. Energy Information Administration.

The research firm estimates that AI accelerators will consume 2,318 TWh of electricity worldwide in the same time frame, accounting for 1.5% of global electricity consumption.

The measurement is based on each GPU using 700W of power, which is the power consumption of Nvidia's flagship Hopper GPU. Nvidia's upcoming GPU, called Blackwell, is faster but consumes 1,200W of power.

TechInsights’ assumptions include only the amount of power consumed by the chip and do not include measurements of storage, memory, networking and other components used to generate the AI.

"Such high utilization is feasible given the huge demand for GPU capacity and the investment required to generate a return on these expensive assets," Owen Rogers, an analyst at TechInsights, said in a research note.

Green or not green?

According to the McKinsey AI Survey, 65% of respondents intend to adopt generative AI.

To keep up with demand, cloud providers and hyperscalers are investing billions of dollars to expand GPU capacity. Microsoft relies on Nvidia GPUs to run its AI infrastructure, and Meta will have a computing environment equivalent to "nearly 600,000 H100" GPUs, the company said.

According to TechInsights, Nvidia will ship about 3.76 million GPUs in 2023, up from about 2.6 million in 2022.

Last year, Gartner made a more aggressive prediction about power consumption, saying AI “could consume 3.5% of global electricity.” Gartner’s methodology is unclear, but it likely includes networking, storage, and memory.

The race for AI is characterized by companies delivering the fastest infrastructure and better results. The rush to implement AI in the business world has disrupted long-standing corporate sustainability plans.

Microsoft, Google and Amazon are spending billions of dollars to build massive data centers equipped with GPUs and AI chips to train and serve ever-larger models, adding to the power burden.

Cost Challenge

Rogers pointed out in the research report that although the purchase price of the server is $20,000, enterprises need to consider the growing electricity costs and the challenges facing the power grid.

Data centers will also need to be designed to meet the power requirements of AI, which may depend on grid capacity and the availability of backup power capacity.

Energy providers also have a responsibility to prepare their electricity infrastructure, including power stations, solar farms and transmission lines, for the AI era.

“If demand cannot be met, energy suppliers will take a market-based approach to managing capacity – i.e. raising prices to reduce consumption – rather than cutting capacity. Again, this could have cost implications for users of AI technologies,” Rogers said.

The US government’s primary goal is to achieve 100% clean energy by 2035, which will reduce the burden on the power grid. This will also open the door to more AI data centers.

Efficient use of energy

The power consumed by AI mirrors an earlier trend of cryptocurrency mining taxing the grid, which accounts for about 2.3% of U.S. electricity consumption, according to a February report from the U.S. Energy Information Administration.

However, energy industry observers agree that AI can use energy more efficiently than bitcoin mining.

Nvidia's AI focus is also on efficient energy use. To reduce power consumption, Nvidia's GPUs use its own chip technology. The company is switching from air cooling to liquid cooling on Hopper.

“The opportunity here is to help them get the maximum performance at the lowest possible cost through fixed megawatt data centers,” Ian Buck, vice president and general manager of Nvidia’s hyperscale and HPC computing business, said at an investor event last month.

HPC Providers, AI, and Sustainability

Panelists at the recent ISC 24 supercomputing conference mocked Nvidia for claiming its 1,000-watt GPUs are "sustainable."

The government lab also said that GPUs and direct liquid cooling have provided better performance scaling than CPUs in the past.

Lawrence Livermore National Laboratory is building the upcoming 2 exaflop supercomputer El Capitan, which will increase cooling pressure to 28,000 tons, add an additional 18,000 tons, and increase power to 85 megawatts for current and future systems.

“El Capitan will produce less than 40 megawatts, about 30 megawatts, but that’s a lot of power,” LLNL Chief Technology Officer Bronis de Supinski said during a breakout session.

He acknowledged that the El Capitan supercomputer might not be considered environmentally friendly, but also that attention should be paid to the results achieved within the performance and power envelope. For example, if the supercomputer solves a climate problem, then the energy it consumes might be worth it.

"A 30-megawatt supercomputer? I'm not going to tell you that's a sustainable resource, but it could go a long way toward solving the societal problems we want to solve," Deszubinski said.

Labs are also turning to renewable energy and liquid cooling. Liquid cooling, for example, “can save about 50 percent of cooling energy,” LRZ President Dieter Kranzlmüller said at the ISC 24 conference.

Sustainable computing environments are also considering carbon offsets, capturing and reusing waste heat, and reusing materials.

HPC's past drives the future

There are efforts underway to improve the energy efficiency of supercomputers so that every watt of power consumed by AI processing can be better utilized.

At the HPE Discover conference last month, CEO Antonio Neri said the company is porting energy-saving techniques used in Frontier and El Capitan to AI systems powered by Nvidia GPUs.

"HPE has one of the largest water cooling manufacturing capabilities in the world. Why? Because we have to make it for supercomputers," Neri said.

Nvidia CEO Jensen Huang, who also took the stage, quipped: "The future of liquid cooling will lead to better performance, lower infrastructure costs, and lower operating costs."

Uninstall AI

Consumer device makers are aggressively promoting PCs and mobile devices equipped with neural chips to enable on-device AI. Neural chips can run AI models locally, offloading pressure from GPUs in the cloud.

Apple provided a complete vision of its on-device and cloud AI strategy — if an iPhone or Mac determines that an AI task can’t be done on the device, it will reroute the query to a cloud server in an Apple data center.

Apple users can also choose to run AI on the device or through the cloud.

Microsoft encourages the use of AI chips in Windows devices. Qualcomm's AI Hub allows users to run benchmarks to see how well AI models run on devices. This allows users to decide whether to run inference on the device or in the cloud.

However, there is no killer app for AI computers that provides a tangible example of computers offloading the burden of AI to GPUs in the cloud.

Reference Links

https://www.hpcwire.com/2024/07/08/generative-ai-to-account-for-1-5-of-worlds-power-consumption-by-2029/

Click here???? to follow us and lock in more original content

END

*Disclaimer: This article is originally written by the author. The content of the article is the author's personal opinion. Semiconductor Industry Observer reprints it only to convey a different point of view. It does not mean that Semiconductor Industry Observer agrees or supports this point of view. If you have any objections, please contact Semiconductor Industry Observer.

Today is the 3823rd content shared by "Semiconductor Industry Observer" for you, welcome to follow.