Arm CEO says Arm is better suited to data center power savings than other architectures

Arm CEO Rene Haas said recently that the current explosive growth of artificial intelligence (AI), especially the training and operation of generative artificial intelligence (gen AI), has "insatiable energy requirements", but he said that Arm is in a unique position to help, and pointed out that the latest Neoverse CPU IP can provide better energy efficiency than competitors.

“AI has the potential to surpass all transformative innovations created over the last century. The benefits to society in healthcare, productivity, education, and many other areas will be beyond our imagination,” Haas claimed. “To run these complex AI workloads, the amount of compute required in the world’s data centers will need to grow exponentially. However, this insatiable demand for compute exposes a critical challenge: data centers require enormous amounts of power to power this groundbreaking technology.”

Today, data centers consume about 460 terawatt hours of electricity per year, equivalent to Germany's annual energy consumption, Haas said. He claimed that by 2030, demand for artificial intelligence will triple, exceeding the total electricity consumption of India, the world's most populous country. "Companies need to rethink everything about addressing energy efficiency, and that thinking should naturally include moving to Arm technology."

Currently, most data centers run x86 processors from Intel or AMD, along with accelerators from Intel, AMD, or NVIDIA. While Arm has had considerable success in embedded and portable computing, it has long struggled to break into the data center — but Haas said that’s changing with the release of its latest Neoverse processor technology, which has been adopted by Amazon, Microsoft, Google, and Oracle.

“As Arm deployments scale, these companies can save up to 15% of their total data center power,” Haas said. “Rather than adding to the energy problem, these massive savings can be used to drive additional AI capacity within the same power envelope. To put this into perspective, these energy savings could run 2 billion additional ChatGPT queries, power a quarter of daily web searches, light 20% of U.S. homes, and power a country the size of Costa Rica.”

To back up these claims with hard numbers, Haas cited some of the company’s products, such as

AWS Arm-based Graviton: Amazon Sagemaker delivers 25% faster AI inference performance, 30% faster web applications, 40% faster databases, and 60% greater efficiency than competitors.

Google Cloud's Arm-based Axion: Delivers 50% better performance and 60% better energy efficiency than competing architectures, powering CPU-based AI inference and training, YouTube, Google Earth, and more.

Microsoft Azure’s Arm-based Cobalt: Delivers 40% better performance than competitors, powers services like Microsoft Teams, and combines with the Maia accelerator to drive Azure’s end-to-end AI architecture.

Oracle Cloud’s Arm-based Ampere Altra Max: delivers 2.5x higher performance per rack server and 2.8x lower power consumption than traditional competitive offerings and is used to generate AI inference models - data aggregation, tokenization, and batch inference use cases for LLM training.

Arm isn’t the only company that thinks it can meet the growing energy demands of the AI ​​transformation, though: Intel just completed deployment of its Loihi 2-based Hala Point system at Sandia National Laboratories, which uses brain-inspired neuromorphic computing to achieve greater efficiency than traditional CPU and GPU parts, while NVIDIA’s Blackwell platform promises 25 times greater energy efficiency than the company’s previous generation of similar products, which uses Arm’s processor IP with NVIDIA’s GPUs.