What chips do the TOP 500 supercomputers use?

Number of new Top500 systems from 2018 to 2023

Submitters' interest in the Top500 list has waned over the years, but there have been some interesting changes to this list. We've seen huge changes in deployed CPUs, accelerators and even interconnects. One of the most important statistics is that 9 of the top 25 systems appear on this list, which means that the turnover rate is 36% among the top 25 systems, but not the next 475 positions on the list Only 8% of them.

First, we emphasize the absolute volume among the top 500. When we began this analysis in 2018, more than a quarter of the list changed with each new publication. In 2020, the industry noted a reduction in the number of systems due to the pandemic. We now appear to be on a new trend line, with the list slightly below average from 2020.

Number of new Top500 systems from 2018 to 2023

We have 48 new systems on this list, up from 44 in June. These are the 41 and 39 new systems added to the 2022 list. Another notable change in this list is that Lenovo commits Linpack to running fewer web hosting type clusters.

It seems like the Top500 list is dying, maybe the top 25, top 50, or top 100 would make more sense these days. This is partly because China has not submitted domestic systems, but Chinese supplier Lenovo is submitting systems outside China.

In this section, we look at CPU architecture trends simply by looking at which new systems enter the Top500 and what CPUs they use. Let’s start by looking at the supplier breakdown.

New Top500 systems from CPU vendors in November 2023

AMD beats Intel again in system count, but that's only part of the story. AMD has more systems, while Intel deploys more cores.

New Top500 Systems for X86 CPU Cores in November 2023

When we talk about CPU cores per socket, the biggest change is the drop to 32-core systems. Systems on this list generally have more CPUs than on previous lists.

New Top500 system CPU cores per socket in November 2023

We finally see the reveal of 96 core components of the AMD EPYC 9004 "Genoa" series. We also see 56 core parts, typically newer Intel Sapphire Rapids Xeon parts.

In the June 2023 list, the most popular architecture is AMD’s Zen3 or third-generation AMD EPYC. The 4th Generation Intel Xeon Scalable Sapphire Rapids really shines in this list.

New Top500 Systems by CPU Architecture in November 2023

The problem is that these parts are now no longer "Sapphire Rapids HBM", but Xeon Max. Xeon Max alone keeps up with AMD's current-generation Zen 4 "Genoa" architecture.

Here are the actual SKUs in use:

New Top500 Systems in November 2023 (by CPU SKU)

It’s kind of crazy that the 2019 AMD EPYC 7002 series Rome CPUs are the most popular on the November 2023 list.

NVIDIA dominates the HPC accelerator market, so we dedicated its title to that. It's far from dominating this list, but it's close:

Top 500 new system accelerators launched by vendors in November 2023

Here's the big question: Where is AMD? Among the four lists, the AMD MI250X accelerator ranks first in the Top500 and is deployed in multiple smaller systems. Now, AMD is absent.

What's probably more significant is that the NVIDIA A100 performed relatively well. The NVIDIA H100 is selling for over MSRP in many of this year's great deals. The H100 is faster, but the AI ​​performance difference between the A100 and H100 is much larger than the FP64 side.

The new generation of Top500 system accelerators in November 2023

To put it in perspective, any AI installation with 10 or more DGX H100 or HGX H100 systems could make the Top500 list.

NVIDIA DGX H100

Considering NVIDIA's data center revenue, it feels like NVIDIA's customers are building systems that don't care about the Top500. In other words, people deploying the popular new HPC accelerators aren't succumbing to the Top500 replacing Lenovo's Web server clusters.

This is one that many people who often look at this piece will agree with. When it comes to interconnects, Ethernet is by far the most common solution. In 2023, InfiniBand is making a comeback.

New Top500 Systems by Interconnect Type for November 2023

This is another great list for Infiniband, especially after Ethernet has included many of the previous lists.

Below are the interconnect details by generation.

New Top500 Systems for November 2023 (generated by interconnection)

We actually saw four Omni-Path systems, which exceeded our expectations.

New Top500 System Ethernet Speeds and Vendors for November 2023

Lenovo's padding system is less generous than on previous lists. There are five CPU-only Ethernet clusters. Dell's cluster uses PowerEdge R750xa with NVIDIA A100 PCIe GPU and 100GbE.

Seeing NVIDIA's massive GPU sales and knowing that these AI systems have the interconnect capabilities to support their operation to be in the Top500 really highlights the challenges facing the Top500. At SC23, big money comes from advertising AI solutions. AMD didn't even hold an SC23 pre-briefing because they told us they were going all-in on AI. Many of the talks at SC23 were about how mixed-precision computing can help achieve scientific progress rather than using double precision. One might say that the goal of the Top500 is to track the largest scientific computing systems. However, it’s hard to argue that all these untracked AI systems aren’t making scientific progress. We have a list now, and a lot of the large artificial intelligence systems are not submitting, the systems deployed in China are not submitting, but the web hosting clusters are submitting results. Maybe it's just the gap between Frontier and the two new exascale systems on this list. Still, it feels like the Top500 is measuring something that few currently consider beneficial. Perhaps over time this will change again.

*Disclaimer: This article is original by the author. The content of the article is the personal opinion of the author. The reprinting by Semiconductor Industry Watch is only to convey a different point of view. It does not mean that Semiconductor Industry Watch agrees or supports the view. If you have any objections, please contact Semiconductor Industry Watch.

Today is the 3596th issue shared by "Semiconductor Industry Observation" with you. Welcome to pay attention.