NVIDIA adds new power to Hopper, the world's leading AI computing platform

World's top server manufacturer and cloud service provider to launch HGX H200 system and cloud instances

Denver - SC23 - November 13 , 2023 , Pacific Time - NVIDIA today announced the launch of NVIDIA HGX™ H200, adding new power to Hopper, the world's leading AI computing platform . The NVIDIA HGX H200 platform is based on the NVIDIA Hopper™ architecture and is equipped with the NVIDIA H200 Tensor Core GPU and leading graphics memory configurations to handle massive data for generative AI and high-performance computing workloads.

NVIDIA H200 is the first GPU to feature HBM3e, whose faster operation and larger memory capacity will further accelerate generative AI and large language models while advancing scientific computing for HPC workloads. With HBM3e, NVIDIA H200 offers 141GB of video memory with transfer speeds of 4.8 TB/sec. Compared with the NVIDIA A100 of the previous generation architecture, its capacity has almost doubled and its bandwidth has increased by 2.4 times.

The world's leading server manufacturer and cloud service provider expects to begin shipping H200-powered systems in the second quarter of 2024.

Ian Buck, vice president of hyperscale and high-performance computing at NVIDIA, said: “Creating intelligence through generative AI and HPC applications requires the use of large, fast GPU memory to process massive amounts of data quickly and efficiently. With NVIDIA H200, industry-leading The end-to-end AI supercomputing platform will become faster and some of the world's most important challenges can be solved."

Continuous innovation brings continuous performance leaps​

The NVIDIA Hopper architecture delivers an unprecedented performance jump compared to previous-generation architectures, and H100's ongoing software upgrades, including the recently released powerful open source libraries such as NVIDIA TensorRT™-LLM, continue to raise its performance standards.

The launch of H200 will bring further performance leaps. For example, the inference speed on Llama 2, a 70 billion parameter LLM, is nearly doubled compared to H100. With continued software upgrades in the future, H200 is expected to achieve greater performance advantages and improvements.

NVIDIA H200 specifications

NVIDIA H200 will be available with four-way and eight-way NVIDIA HGX H200 server motherboard options, which are compatible with the hardware and software of the HGX H100 system. It can also be used on the NVIDIA GH200 Grace Hopper™ super chip using HBM3e released in August this year. These configurations enable the H200 to be deployed in all types of data centers, including on-premises, cloud, hybrid cloud and edge.

Powered by NVIDIA NVLink™ and NVSwitch™ high-speed interconnect technology, HGX H200 delivers the highest performance for a variety of application workloads, including LLM training and inference for very large models with more than 175 billion parameters.

The eight-way HGX H200 can provide more than 32 PetaFLOPS of FP8 deep learning computing power and 1.1TB of aggregate high-bandwidth memory capacity, bringing strong performance to generative AI and HPC applications.

When the H200 is paired with NVIDIA Grace™ CPUs featuring ultra-fast NVLink-C2C interconnect technology, the result is the GH200 Grace Hopper superchip with HBM3e - a compute module designed for large-scale HPC and AI applications.

Accelerate AI with NVIDIA full- stack software

The NVIDIA accelerated computing platform is powered by powerful software tools that enable developers and enterprises to build and accelerate production-ready applications from AI to HPC, including the NVIDIA AI Enterprise software suite for workloads such as speech, recommendation systems and very large-scale inference.

Availability​

NVIDIA H200 will be available through system manufacturers and cloud service providers globally beginning in the second quarter of 2024.