MLPerf Storage benchmark updated for modern AI

Memory & Storage

MLPerf Storage V3.0 Adds S3, KV Cache and Vector Tests

MLPerf Storage V3.0 adds an S3 object storage layer and inference workload tests for vector databases and KV caches. Nineteen suppliers submitted results. Everpure's FlashBlade//EXA led large-model checkpointing at 877.52 GiB/s writes.

By
Nathan Brooks
Filed
Channel
Memory & Storage
Read
3 min read

By Nathan Brooks

Nineteen suppliers submitted results to MLCommons' MLPerf Storage V3.0 benchmark, released this month with an S3 object storage layer and two inference-oriented workload tests for vector databases and KV caches. The submission count is itself a fair marker of where the AI storage market now sits: the field has consolidated around specialist vendors, while most of the legacy enterprise incumbents sit this round out.

The V3.0 suite replaces the older Nvidia A100 and H100 simulation that defined V2.0. Training now runs Unet3D and RetinaNet workloads targeting Nvidia B200 or AMD MI355 accelerators. Checkpointing scales to 8 billion, 70 billion, 405 billion and 1,250 billion parameter Llama 3-style models.

Two new inference tests round out the suite. The vector database workload queries a one-million-vector Milvus database at 1,536 dimensions. The KV cache test measures throughput for conversational inference workloads running against the same Llama 3.1 family used in checkpointing.

Brian Belgodere, co-chair of the MLPerf Storage working group, said the additions "round out the test collection, covering a larger range of AI inference workloads that drive storage needs." The new structure, he added, gives engineers a "much clearer idea of how to engineer and provision AI systems to minimize storage performance bottlenecks."

Roughly one-sixth of submissions in this round used the S3 access layer, which V3.0 supports for training, checkpointing and certain VDB tests. Curtis Anderson, the group's other co-chair, framed the move as preparation for the next storage wave. "As the scale of AI contexts reaches into the trillions, we expect object-based storage systems to emerge as a viable – and possibly preferred – alternative to filesystem-based storage," he said.

Who is missing from the table?

DDN, Dell, Huawei, IBM, NetApp, VAST Data and WEKA did not submit results. Traditional enterprise storage incumbents are largely absent from a list otherwise crowded with newer, AI-focused vendors such as Everpure, Samsung and Solidigm.

Only NewFW, Samsung, Suzhou Zishan and TTA submitted VDB results. The thin field reflects how selectively suppliers pick their workloads in a benchmark that does not require an all-in submission.

What does the leaderboard show?

Everpure claimed the top spot in 405 billion and 1.25 trillion parameter checkpointing and in KV cache performance with its FlashBlade//EXA system. The platform delivered 877.52 GiB/s of write bandwidth, completing the 1.25T parameter model in 17.74 seconds, and 588.28 GiB/s of read bandwidth over 28.99 seconds. As a rough comparison, the write figure works out to about 220 gigabytes per second of sustained throughput, more than most consumer SSDs deliver in an entire day.

The result used 30 data nodes against 1,024 simulated accelerators. Performance scaled predictably, rising from 327.65 GiB/s at 10 data nodes to 877.52 GiB/s at 30. The scaling curve suggests the architecture has headroom beyond the headline configuration.

How is power efficiency reported?

MLCommons invites power-normalised results alongside the headline numbers. On-premises checkpointing write submissions posted a median of 14 GB/s per watt with a maximum of 201 GB/s per watt.

UNet3D read submissions reached a 34 GB/s per watt median and a 277 GB/s per watt maximum. The wide spread matters as buyers weigh rack-scale efficiency against raw throughput.

Why is the spreadsheet so large?

The results file runs to roughly 145 rows and 55 columns, or about 8,000 cells. KV cache alone consumes 12 columns covering Llama 3.1-8b and Llama 3.1-70b models in storage-only and storage-plus-memory configurations.

There is no single number that summarises the leaderboard, which is the point. The benchmark tries to mirror the storage reality behind modern AI training and inference, where throughput, latency, capacity and power all pull in different directions.

V3.0 lands as context windows approach the trillion-token range and object stores begin to encroach on the file systems that long dominated AI training. Expect the next submission round to widen the S3 column and lengthen the spreadsheet further.

Original: mlcommons.org

Share this article:

More from Nathan Brooks

Nathan Brooks

Show full bio

Senior reporter covering people, launches and day-to-day practice in Semiconductors, chip manufacturing, hardware, AI compute at Chip Dispatch.

357 articles

Related articles

« Previous articleNext article »