New breakthrough! Ultra-fast memory accelerates Intel Xeon 6-core processors

Intel has worked with industry partners to use an innovative approach to double the memory bandwidth of standard DRAM modules. This plug-and-play solution can unleash the potential of the Xeon 6-core processor.

Intel has been deeply involved in the processor market, which is regarded as the "brain" that ensures the excellent operation of computers. However, it cannot be ignored that system memory (DRAM) also plays an important role in improving overall performance. Especially in the server field, the growth rate of the number of processor cores exceeds the increase in memory bandwidth, which means that the actual memory bandwidth available to each core is reduced accordingly.

Weather modeling, computational fluid dynamics, and certain AI scenarios often require running a large amount of workloads, so the imbalance between processor cores and memory bandwidth may lead to computing bottlenecks. Now, technological innovation has brought us new solutions.

After years of collaboration with industry partners, Intel engineers have created an innovative solution that will break through performance bottlenecks and lead to new open industry standards. The recently released Intel® Xeon® 6 Data Center processor is the first to use a new type of memory called MRDIMM, which significantly improves performance in a plug-and-play manner.

“A large portion of scientific computing workloads are memory bandwidth limited,” which is where MRDIMMs excel, said Bhanu Jaiswal, product manager for Xeon processors in Intel’s Data Center and Artificial Intelligence (DCAI) Group.

Next, we will reveal the story behind the development of DDR5 Multiplexed Dual In-line Memory Module (MRDIMM).

Inspired by the team, bringing parallelism to system memory

In data centers, the most commonly used memory modules, called RDIMMs, have onboard parallel resources just like modern processors, but they use them differently.

"Most DIMMs have two arrays: performance and capacity," said George Vergis, senior principal engineer for memory development at DCAI. "This design helps achieve a balance between performance and capacity."

The arrays are like the banks of a river, with one set of memory chips on a module belonging to one array and the rest to another. RDIMMs allow independent storage and data access to occur between multiple arrays, but not simultaneously.

This reminded Vergis of the moment when inspiration struck. He said, "At that time, we thought, since the parallel resources are not being used, why not bring them together?" So Vergis proposed an innovative solution, which is to place a small interface chip - a multiplexer (Mux) on the DRAM module, so that data can be transmitted across two memory arrays at the same time.

The multiplexing buffer consolidates the power load of each MRDIMM, which allows the interface to run faster than RDIMMs. The bandwidth is also doubled by being able to access both memory arrays in parallel.

The result is Intel’s fastest system memory ever, with nearly 40 percent more peak bandwidth, jumping from 6,400 MT/s to 8,800 MT/s, something that previously took multiple iterations to achieve.

Same memory module standard, but faster

Based on its past technological accumulation in the storage field, Intel continues to innovate in technology to enable different computing components to be interoperable and achieve performance improvements.

Vergis is Intel’s representative on the board of the Joint Electron Device Engineering Council (JEDEC), which sets open standards for the microelectronics industry and is particularly well-known for its work on memory standards. Vergis’ work on DDR5 standardization earned him a JEDEC award in 2018, and now he’s working full-time on DDR6. (This year, JEDEC also honored Intel CEO Pat Gelsinger, praising him as a “strong supporter of open standards” and noting that “Intel’s historical leadership in standards development is a testament to this.”)

Vergis led the team to start research and development in 2018 and verified the concept through a prototype in 2021. Intel worked with ecological partners in the storage field to manufacture the first batch of components and donated the component specifications to JEDEC as a new open standard at the end of 2022.

The outstanding advantage of MRDIMM is its ease of use. By adopting the same connector and form factor as regular RDIMM, the small multiplexer chip can also be adapted to the empty position on the previous module, so no changes are required to the motherboard.

MRDIMMs also have the same error correction and reliability, availability and serviceability (RAS) features as RDIMMs, Vergis said, maintaining data integrity regardless of the independent multiplexed requests generated in the data buffer.

This means that for data center customers, by selecting MRDIMMs when ordering new servers, or replacing RDIMMs with MRDIMMs in rack servers, they can experience new performance without having to change any code.

Xeon 6 + MRDIMM =

MRDIMMs alone are not enough, the industry also needs a CPU that can bring out the performance of MRDIMMs. The Intel Xeon 6-core processor (codenamed Granite Rapids) launched this year is the first processor in the industry to support MRDIMM technology.

Recent media testing compared two identically configured Xeon 6 systems, using MRDIMMs and RDIMMs. The results showed that the system using MRDIMMs completed 33% faster than the system using RDIMMs.

Jaiswal说，“MRDIMM带来的带宽提升，非常适用于小语言模型、传统深度学习和推荐系统这类 AI 工作负载，它可以在至强平台上轻松运行，并提供显著的性能提升。”

Some leading storage vendors have already launched MRDIMMs, and more will release their products in due course. Some research institutions are also actively adopting Xeon 6-core processors that support MRDIMM technology.

Jaiswal said, "It is the strong ecosystem of OEMs and memory providers that gives Intel the opportunity to gain first-mover advantage in this technology."

Notes:

Performance varies based on use, configuration, and other factors.