Latest MLCommons results announced: Sharing AI achievements of the fifth-generation Xeon Scalable Processor

The latest MLPerf test results for the 5th Gen Xeon Scalable processors showcase the progress Intel and its ecosystem partners have made in improving generative AI performance.

Recently, MLCommons announced the results of the MLPerf v4.0 benchmark for AI reasoning. Among them, the fifth-generation Intel® Xeon® Scalable Processor (hereinafter referred to as the "fifth-generation Xeon") with built-in Intel® Advanced Matrix Extensions (Intel® AMX) performed well in the test, further demonstrating Intel's commitment to promoting "AI everywhere" through rich and competitive solutions. As of now, Intel is still the only CPU manufacturer that has submitted MLPerf test results. Compared with the results of the fourth-generation Xeon in the MLPerf reasoning v3.1 benchmark, the test results of the fifth-generation Xeon are an average of 1.42 times higher.

“We continue to improve AI performance across our broad portfolio of CPUs and accelerators in industry benchmarks,” said Zane Ball, Intel vice president and general manager of product management for the Data Center and AI Group. “This new MLCommons result shows that the AI ​​solutions we offer are able to meet the evolving and diverse AI needs of our customers. At the same time, Intel Xeon processors provide customers with a cost-effective option for rapid AI deployment.”

Intel products have demonstrated leading training and inference performance in multiple rounds of MLPerf benchmark tests to date. The test results have also established an industry standard for customers to evaluate the AI ​​performance of products.

Test results for the fifth-generation Xeon:

Compared with the performance of the fourth-generation Xeon in the MLPerf inference v3.1 performance benchmark, the fifth-generation Xeon, which has been optimized by hardware and software, has an average performance improvement of 1.42 times. Among them, for the GPT-J model with software optimizations such as continuous batching, the performance of the fifth-generation Xeon has increased by about 1.8 times compared with the test results of v3.1; similarly, thanks to MergedEmbeddingBag and other optimizations based on Intel AMX, the test results of DLRMv2 show a performance improvement of about 1.8 times and an accuracy of 99.9%.

5th Generation Intel® Xeon® Scalable Processors

At the same time, Intel is very proud to work with a wide range of OEM partners including Cisco, Dell, Quanta, Supermicro and Wiwynn to help them submit MLPerf test results based on their own products. Intel not only began submitting test results based on the fourth-generation Xeon in 2020, but the Xeon Scalable processor is also the host CPU for many accelerators in the products participating in MLPerf testing.

In addition, the 5th Generation Xeon can be evaluated on the Intel® Developer Cloud, where users can perform small and large-scale AI training (such as large language models or generative AI), run large-scale inference workloads, and manage AI computing resources.

Notes: Workload and related configuration descriptions. Results may vary.