How we test enterprise SSDs

Memory & Storage

Tom's Hardware Builds a Bench That Shows SSDs on Their Worst Day

Tom's Hardware returns to enterprise SSD testing with a Xeon 658X bench, SNIA-derived steady-state gates ten times tighter than the standard, and AI checkpoint workloads up to 912GiB.

By
Nathan Brooks
Filed
Channel
Memory & Storage
Read
4 min read

By Nathan Brooks

A single enterprise SSD review at Tom's Hardware now consumes most of a day on a 7.68TB PCIe 5.0 drive, because the drive is written end to end many times over before a single headline number appears. The outlet has returned to enterprise and data center SSD testing after a hiatus of several years, and the methodology it published is stricter than most.

The motive is commercial. Consumer SSDs and hard drives lack the performance and endurance for extensive AI workloads, and demand for enterprise drives has grown accordingly. Form factors once rare — U.2, U.3, E3.S and E1.S — now appear in high-end workstations and home labs. These drives offer higher capacity, better performance and greater reliability than consumer parts, and they represent a major investment for buyers.

What hardware drives the bench?

The test workstation runs on a Xeon 658X, a 24-core, 48-thread Granite Rapids chip with a 4.9 GHz boost clock. Clock speed sets how many IOPS one core can push; 24 cores let the suite spread a drive's load across eight fio jobs and test several drives at once. Asus, Corsair, G.Skill, Icy Dock and Noctua supplied components.

The supporting cast is precise:

  • Asus Pro WS W890E-Sage SE motherboard, with an MCIO x8 split into x4/x4 for two U.2 or U.3 drives, and seven PCIe 5.0 slots feeding the Xeon's 128 PCIe 5.0 lanes.
  • G.Skill G5 DDR5-6400 64GB R-DIMM kit, run at the JEDEC DDR5-5600 baseline because memory frequency does not affect these tests.
  • Corsair MP700 Pro XT 2TB as the OS drive, at up to 14,900 MB/s read and 14,500 MB/s write, fed by an 80 Plus Platinum Corsair HX1500i at 1,500W.
  • Icy Dock ExpressSlot Elite adapters for one and two E1.S SSDs with tool-less mounting and 50mm fans, plus a ToughArmor bay holding eight E1.S drives for future RAID work. E3.S drives run on a JMT ADT-R27C adapter with a Noctua industrial fan.

An RTX Pro graphics card for real-world AI workloads is still on the wish list.

How does the suite reach steady state?

Every drive starts with a secure erase via nvme format, then preconditioning with SPRandom, SanDisk's open-source contribution to fio. SPRandom takes capacity and over-provisioning as inputs and, per SanDisk, reaches a steady-state layout in one pass. A 7.68TB drive assumed to sit on 8TiB of NAND works out to about 14.5% OP.

The suite does not take that on faith. A steady-state gate runs 4K random writes at QD256 until write speed stops moving, applying the SNIA rule at limits ten times tighter: five six-minute round averages within 2% of one another, and a trend line that cannot move more than 1%. A drive that fails to settle within three hours is marked as not reaching steady state. A backstop repeats the 4K write hours later and flags any run where results differ by more than 3%.

The whole bench runs on a custom Debian 13 image with the kernel pinned to Linux 7.2.6, because the stock 6.12 kernel predates support for the board's Intel E610 10GbE controller. The team picked Debian over Ubuntu for fewer background services, no Snap and no unattended upgrades. Every test runs in fio 3.42 with io_uring and direct I/O against the raw block device, so the bench's 64GB of RAM never caches a byte. Vdbench was rejected; it has had no release since 2018.

What gets measured, and what does not?

The suite covers four-corner throughput, random and mixed sweeps from QD1 to QD4096, block sizes from 512 bytes to 1MB, tail latency to P99.999, saturation curves, one-hour sustained writes, write-to-read recovery, idle recovery, simulated database I/O, and AI checkpoints modeled on 8B- and 70B-class models — 105GiB over eight writers and 912GiB over 64.

The bench also polices itself against measuring the host instead of the drive. Waking an idle Xeon core takes up to 170 microseconds, so cores are held awake, which cut QD1 read latency by about a sixth. Fresh random seeds fixed a bug where fio replayed identical offsets and inflated 4K random writes to more than twice the sustained figure. Interrupt delivery caps the bench near 2.8 million 4K random read IOPS, a limit the suite discloses and works around with polled-I/O runs kept in separate charts.

Out of scope by design: dual-port failover, power-loss validation and SR-IOV, which belong in a qualification lab. The promise is repeatability — every result stored with the parameters that produced it, so the numbers can be traced, and vendor disputes settled with evidence rather than assertion.

Source: Tom's Hardware

Share this article:

More from Nathan Brooks

Nathan Brooks

Show full bio

Senior reporter covering people, launches and day-to-day practice in Semiconductors, chip manufacturing, hardware, AI compute at Chip Dispatch.

1,022 articles

Related articles

« Previous article