Why is storage system performance critical for AI workloads?

Follow Micron for updates

数据是各种现代企业的生命线，而数据存储、访问与管理策略对企业的生产力、盈利能力以及竞争力会产生显著影响。随着人工智能（AI）的兴起，各行各业都在经历变革，企业不得不重新思考如何利用数据来加速创新和增长。 然而，AI训练和推理对数据管理和存储提出了独特的挑战，因为它们需要处理庞大的数据，同时要求高性能、可扩展性和高可用性。

The performance of storage systems varies and is affected by many factors. In this blog post, we will explore several factors that affect the performance of storage systems in the field of AI, and focus on how the choice of underlying storage media will affect these performance factors.

AI workloads are both data-intensive and compute-intensive, meaning they require processing large amounts of data at high speeds and with low latency. Storage plays a key role in enabling AI workloads to efficiently and effectively access, ingest, process, and store data. Several key attributes of typical AI workloads that impact storage requirements include:

Data diversity : AI workloads need to access data from multiple sources in structured, unstructured, and semi-structured formats, and these data are located in different locations (such as local, cloud, or edge devices). Storage solutions need to ensure fast and reliable data access and transmission between different environments and platforms.

Data velocity : AI workloads require data to be processed in real time or near real time. Storage solutions need to ensure high throughput, low latency, and stable and consistent performance during data ingestion, processing, and analysis.

Data volume : As AI models become more complex and accurate, and GPU cluster computing power continues to grow, storage solutions need to provide flexible and scalable capacity and performance.

Data reliability and availability : AI workloads must ensure data integrity, security, and very high availability, especially when connected to large GPU clusters that cannot tolerate interruptions in data access, so the requirements are higher.

Storage system performance is not a single indicator, but a combination of multiple factors, which depends on the characteristics and requirements of data, applications and data center infrastructure. These include the following important factors:

Throughput : The rate at which data is transferred from the storage system to the network or host, and from the network or host to the storage system. Improving throughput improves system performance by increasing bandwidth and reducing congestion and bottlenecks in the data flow. Throughput is often affected by network bandwidth or the speed of the storage media.

Latency : The time it takes a storage system to respond to a read or write request. Low latency improves performance by reducing GPU idle time and increasing the system's responsiveness to user input. Mechanical devices such as HDDs inherently have much higher latency than solid-state devices (SSDs).

Scalability : The ability of a storage system to adapt to the volume, velocity, and diversity of data. High scalability is key to ensuring that storage systems can grow and evolve with business needs and goals. The critical challenge in increasing the amount of data that a system can store and manage is to maintain performance expansion without hitting bottlenecks or storage device limitations.

Resilience : The ability of a storage system to maintain data integrity and availability in the face of failures, errors, or disasters. Higher reliability can improve performance by reducing the frequency and impact of data corruption, loss, and recovery.

In data center applications, hard disk drives (HDDs) and solid-state drives (SSDs) are the two main persistent storage devices. HDDs are mechanical devices that store data on spinning disk platters coated with a layer of magnetic material, while SSDs store data on solid-state flash memory chips. For decades, HDDs have been the dominant storage device. HDDs have a low cost per bit and long-term power-off durability, but are inferior to SSDs in terms of speed and reliability. SSDs have the characteristics of high throughput, low latency, high reliability, and denser packaging options.

As technology continues to advance and computing demands increase, the mechanical properties of HDDs do prevent them from matching SDDs in performance. System designs can improve the effective performance of HDD-based storage systems in several ways, such as mixing hot and cold data (letting hot data borrow performance from cold data), sharing data in parallel across multiple HDD disks (increasing throughput without reducing latency), reserving redundant capacity in HDDs (essentially pre-provisioning for IO rather than adding capacity), and adding an SSD cache layer for requests or operations with abnormal latency. From a cost-effectiveness perspective, the capabilities of these system-level solutions can only be scaled to a limited extent. These solutions need to scale to meet the performance requirements of the actual application. For many current AI workloads, HDD-based systems lack performance scalability and power efficiency.

SSD-based mass storage systems can provide a more elegant and scalable solution, and they are rapidly gaining ground as the storage medium for high-performance AI data lakes in many large GPU-centric data centers. From a drive-level perspective, SSDs cost more (based on cost per bit) than HDDs. From a system-wide perspective, systems built with SSDs cost less to operate than HDDs if the following improvements are taken into account:

Higher throughput

Latency reduced by more than 100 times

Fewer servers and racks per petabyte

Higher reliability and longer service life

Higher energy efficiency at a given performance level

SSD capacities are expected to exceed 120TB in the next few years. As capacities increase and the price gap between SSDs and HDDs narrows, these SSDs will be attractive alternatives for other workloads that require above-average performance or very low latency over large data sets, such as video editing and medical imaging diagnostics.

Currie Munce

Senior technical consultant and strategic expert of Micron Storage Division