Microsoft's first custom AI chip, Maia 100, more specifications revealed

Publisher:SereneGardenerLatest update time:2024-08-30 Source: IT之家 Reading articles on mobile phones Scan QR code
Read articles on your mobile phone anytime, anywhere

On August 30, Microsoft attended the Hot Chip 2024 conference and shared the specifications of the Maia 100 chip. Maia 100 is one of the largest processors manufactured on TSMC's 5nm node and is designed specifically for large-scale AI workloads deployed in Azure.

The Maia 100 chip specifications are as follows:

  • Chip size: 820 mm2

  • Package: TSMC N5 process using COWOS-S sandwich technology

  • HBM BW/Cap;1.8TB/s @ 64GB HBM2E

  • Peak dense Tensor POPS: 6bit: 3,9bit: 1.5, BF16: 0.8

  • L1/L2:500MB

  • Backend network BW: 600GB/s (12X400gbe)

  • Host BW (PCIe): 32GB/s PCIe Gen5X8

  • Design TDP: 700W

  • Provision TDP:500W

The Microsoft Maia 100 system is vertically integrated to optimize cost and performance. It also uses a custom server board with a specially designed chassis and software stack to improve performance.

Maia 100 Architecture

  • The high-speed Tensor unit provides high-speed processing for training and inference while supporting multiple data types. The unit adopts a 16xRx16 structure.

  • The vector processor is a loosely coupled superscalar engine with a custom instruction set architecture (ISA) that supports multiple data types including FP32 and BF16.

  • The direct memory access (DMA) engine supports different tensor sharding schemes.

  • Hardware semaphores support asynchronous programming of Maia systems.

  • To improve data utilization and power efficiency, large L1 and L2 scratch pads are managed by software.

Maia 100 uses Ethernet-based interconnect technology and custom protocols like RoCE to enable ultra-high bandwidth computing. It supports up to 4800 Gbps of all-gather and scatter-reduced bandwidth, and 1200 Gbps of all-to-all bandwidth.

On the software side, the Maia software development kit (SDK) allows anyone to quickly port their PyTorch and Triton models to Maia. The Maia SDK provides developers with several components that enable them to easily deploy models to the Azure OpenAI service.


Reference address:Microsoft's first custom AI chip, Maia 100, more specifications revealed

Previous article:IBM launches Telum II processor and Spyre AI accelerator: 8 cores, 5.5GHz, 360MB cache
Next article:Intel engineering resources are tilted towards Intel 18A, Arrow Lake mainly uses external processes

Latest Embedded Articles
Change More Related Popular Components

EEWorld
subscription
account

EEWorld
service
account

Automotive
development
circle

About Us Customer Service Contact Information Datasheet Sitemap LatestNews


Room 1530, 15th Floor, Building B, No.18 Zhongguancun Street, Haidian District, Beijing, Postal Code: 100190 China Telephone: 008610 8235 0740

Copyright © 2005-2024 EEWORLD.com.cn, Inc. All rights reserved 京ICP证060456号 京ICP备10001474号-1 电信业务审批[2006]字第258号函 京公网安备 11010802033920号