Impact of transients on AI accelerator card power supply

summary

Graphics processing units (GPUs), tensor processing units (TPUs), and other types of application-specific integrated circuits (ASICs) enable high-performance computing by providing parallel processing capabilities to meet the needs of accelerating artificial intelligence (AI) training and inference workloads.

AI requires a lot of computing power, especially when learning and reasoning. This demand continues to push the boundaries of power distribution networks to unprecedented new levels. These high-density workloads become more complex, and higher transient demands drive every part of the power distribution network to operate efficiently. The stringent power consumption requirements of AI accelerator cards also have an impact on system performance. This article will discuss the power distribution network requirements for AI accelerator cards, analyze the impact of transients, and introduce ADI's multiphase power supply solutions for these needs.

Introduction

AI technology has completely changed the computing architecture to replicate neural networks that mimic the human brain. AI may seem to be widespread, but in fact, the technology that drives AI is still evolving. Processor accelerator ICs specifically used for AI computing include GPUs, field programmable gate arrays (FPGAs), TPUs, and other types of ASICs. This article refers to them collectively as xPUs.

As AI technology deployment advances rapidly, data centers will continue to purchase AI accelerator cards in bulk. According to a Gartner report, AI chip revenue totaled more than $34 billion in 2021 and is expected to grow to $86 billion by 2026. ¹ xPU uses a large-scale parallel computing scheme, which has achieved a huge leap in AI performance compared to ordinary CPUs. With a large number of small cores, xPU is well suited for AI workloads, which helps with neural network training and AI inference. However, xPUs usually consume relatively large power for AI calculations and moving data. In short, xPUs are very power-hungry ICs. Their strict power consumption requirements pose new challenges to AI accelerator cards, which also affects system performance. This article will analyze the power supply network requirements of AI accelerator cards and introduce the multi-phase power supply solution proposed by Analog Devices to meet these stringent requirements.

Power supply challenges brought by AI

AI involves many aspects, but energy efficiency is not one of them. AI requires extremely high computing power when it is working, especially when processing AI workloads such as deep learning and inference. At the system level, AI accelerators play a key role in providing near-instant results, which is what makes it valuable. All xPUs have multiple high-end cores that are made up of billions of transistors and consume hundreds of amps of current. The core voltage (V _CORE ) of these xPUs has been reduced to a level below 1.0 V. Figure 1 shows a general block diagram of an AI accelerator card. This article will focus on the multi-phase controller and corresponding power stage IC proposed for such systems.

Figure 1. General AI accelerator card block diagram

The peak current density required by AI accelerator cards is too much for any motherboard to handle. The highly dynamic nature of the workload and the extremely high current transients result in very high di/dt and spike voltage transients lasting several microseconds, which are very destructive and can cause damage to the xPU. The average workload of AI lasts for a long time, and the decoupling capacitors will not always be able to provide the energy to meet the immediate needs. The next section of this article will introduce the multi-phase point-of-load (PoL) solution proposed by Analog Devices, which will eliminate the transients of typical AI accelerators and avoid stressing the entire power distribution network. But first, let's discuss the power design challenges brought by AI.

AI brings new power supply design challenges

Currently, AI power demands far exceed the capabilities of traditional power delivery networks. The requirements for the xPU voltage regulator (VR) are very different from standard PoL regulators. The industry has found that some applications require more than 1000 A of current to be supplied to the xPU at a voltage of less than 1 V. It is important that the power supply is very stable and produces very little noise while eliminating all voltage transients that may cause false triggering inside the xPU. To cope with the staggering current demands, the design of a high-performance AI accelerator VR PoL must meet certain key requirements.

Voltage spike and transient management

One of the key requirements for AI accelerator cards is that the architecture of VR should provide excellent transient voltage management. Providing kilowatts of power to any system is always the primary challenge. The output voltage (including tolerance, ripple, and load transient sags and peaks) must always be higher than the xPU minimum voltage to avoid system hangs, and must also always be lower than the xPU maximum voltage to avoid damaging the xPU. The transient power spikes of the accelerator card may require 2 times or even more of the maximum thermal power target.

What is important here is that the PoL loop bandwidth must be flexible enough to handle the various faster transients encountered. The higher the bandwidth, the faster the loop response and the smaller the voltage deviation. One of the more straightforward ways to achieve a fast transient power rail is to choose a regulator with fast transient performance. The ADI AI V _CORE series of ICs feature very low frequency output noise, fast transient response, and high efficiency. In addition to this, the ADI AI power chipset also supports load lines, which helps power designers effectively manage transients and spikes caused by AI workloads.

I ² R Losses and Thermal Management in Long Power Path Routes

As AI xPU processors continue to increase in current, the density of PoL power delivery solutions has become a critical factor. It is now extremely difficult to reliably deliver power to each part of the xPU without worrying about the dissipated heat affecting the reliability of the chip and causing thermal runaway. In other words, thermal management is one of the major challenges in designing such high-power power supplies. The traditional power delivery method places the regulator on one side of the xPU to transfer power laterally to the processor. Even the smallest resistance of these traces can cause unacceptable voltage (I ² R) drops. The voltage drop across the resistance of the PCB power plane increases proportionally with the increase in xPU current. This means that a few centimeters of PCB power traces between the VR and the BGA pins will generate a lot of losses. Such losses in the PCB copper power plane have become a dominant factor in computing the efficiency and performance of the regulator design. Compared with the traditional 3-chip (discrete) power delivery solution that requires a large number of high-current traces, the use of a single-chip power stage IC with integrated current and temperature circuit blocks can greatly reduce the number of traces on the PCB.

ADI value proposition : MAX16602 + MAX20790 + coupled inductor

The accuracy of AI regulators has become more stringent. Efficiency and size are top priorities. Performance and power consumption are also under scrutiny. As mentioned in the previous section, solving AI accelerator card VR design problems has become a difficult task. Designers are well aware that large steps in the required current cannot be generated without effectively handling unwanted transient effects. Addressing these transient effects also requires some type of high-precision dynamic voltage positioning or load line scheme. Analog Devices has invested heavily in the AI ​​market and provides a complete solution for 48 V and 12 V systems. This section introduces the ADI AI multiphase power chipset, namely the MAX16602 multiphase controller and the MAX20790 power stage, as well as our patented coupled inductor (CL) technology to help solve these AI PoL design challenges. Figure 2 shows the simplified block diagram connection of the MAX16602, MAX20790, and CL for the 8-phase MAX16602CL8_EV design. This relatively simple design achieves a high current delivery capability of approximately 88 A _PK per phase. Internal compensation and advanced control algorithms, along with integrated current sensing circuitry in the power stage and coupled inductors, make this a small size solution with excellent efficiency.

Figure 2. An 8- phase VR design using ADI ’s highly integrated power chipset helps achieve high-density design while reducing external connections.

Single-chip intelligent power stage IC with higher integration density

The MAX20790 is a feature-rich smart power-stage IC designed to work with the MAX16602 (and several other ADI controllers in this product family) to implement high-density multiphase regulators. This monolithic integration virtually eliminates the parasitic resistance and inductance between the FETs and drivers that are common in discrete designs, allowing high switching speeds with significantly lower power losses than traditional solutions. If a switch node (V _X ) fault is detected, the power stage is immediately shut down and the fault ID is communicated to the controller. The smart power-stage IC also has an on-chip current sensor. This current sensing circuit block is clearly superior to the method using the inductor DC resistance. It is well known that DCR sensing is inaccurate and requires temperature compensation to make the current measurement reliable.