VDURA CEO says bandwidth, not capacity, should size AI SSD tiers

By Grace Kim

A PCIe Gen5 x4 NVMe SSD provides roughly 14 GB/s of read bandwidth whether it holds 7.68 TB or 61.44 TB, according to VDURA CEO Ken Claffey.

In his soon-to-be-published 6,000-plus-word blog, “Long Live the Short King. Now Look One Tier Down,” Claffey extends SemiAnalysis’s “Long Live the Short King: Why 4-hi HBM Wins” argument to durable NVMe storage used in KV-cache tiering. His central claim is that write bandwidth matters more than capacity for those SSDs.

The SemiAnalysis article appeared the previous month and addressed stack heights for next-generation accelerators. Claffy treats it as an understated guide to sizing a high-performance storage tier. He calls it the strongest sizing argument he has read this year, although the SemiAnalysis article never makes that broader claim.

What does the HBM comparison establish?

Claffey says HBM4 and HBM4E stacks range from 4-hi to 12-hi. Their connection pin count remains fixed at 2,048, so he argues that bandwidth does not rise with stack height. He reads the result as proof that capacity beyond 4-hi cannot use the available I/O bandwidth.

“Stack height buys capacity and nothing else.”

He calls capacity beyond the usable bandwidth “stranded capacity.” In his flash analogy, bandwidth is a property of the device, constrained by whichever limit binds first among the interface, the controller, and the media itself under load. Capacity sits behind that limit and scales independently. The premium tier in a hierarchy therefore earns its price through bandwidth, not through additional storage.

Claffy uses Kioxia’s CD9P PCIe Gen5 TLC SSD to test the argument. In his comparison, a 61.44-TB drive holds eight times the capacity of a 7.68-TB model. It has identical write bandwidth and nine percent less read bandwidth. The larger device does not gain speed because only the NAND behind it grew. Once bandwidth becomes the constraint, additional terabytes cost premium-media prices for capacity a cheaper medium could provide.

The mirror image applies to SSDs, Claffey says. Stranded HBM capacity is storage paid for but unable to produce tokens. Stranded SSD performance is speed paid for but unable to carry useful work. The financial stakes are material: he says SSD terabytes cost 14 times as much as disk-drive terabytes.

How should buyers size the flash tier?

Claffey recommends sizing for bandwidth in both directions before counting capacity. Buyers should count devices rather than terabytes, select the smallest drive that reaches the required device count, inspect the capacity that comes with it, and place the remainder in an external disk tier.

He provides a formula:

N = R ÷ (d × E)

Here, N is the number of drives, R is required bandwidth, d is bandwidth delivered by one SSD, and E is the fraction of d that reaches the client. Total capacity then equals N multiplied by drive capacity, while flash spending equals N multiplied by drive price. Delivered cost per gigabyte per second equals drive price divided by (d × E).

Accelerators set R. Drive makers set d, and Claffey says every buyer draws from the same short list of suppliers. Architecture sets E, which affects every other term. Halving E doubles the drive count, installed capacity and power draw. Choosing a larger drive instead raises nominal capacity without increasing d. That improves one line of a purchase quote while leaving bandwidth economics unchanged or worse.

What must the disk tier absorb?

The performance drop from flash to disk is not a small multiple, Claffey says. It spans orders of magnitude.

A capacity tier optimized only for dollars per terabyte can become a bottleneck because it cannot accept data as quickly as the flash tier sends it. The strategy therefore depends on a tier built for the workload, not merely on owning some flash.

He warns against archive-style configurations with too few, oversized devices because such a storage purchase cannot be revisited later once the accelerator workload is under way.

“The flash tier is a buffer, and a buffer only works if it empties as fast as it fills.”

System designers must therefore build the downstream tier for throughput rather than dollars per terabyte alone. Claffy presents VDURA’s parallel file system, with built-in S3 support, as a response to that requirement. He says it can ingest data from the SSD tier and feed it fast enough to keep accelerator activity high, while scaling performance as capacity increases.

Claffy calls the “short king” thesis correct but incomplete. It stops at the accelerator package, while his proposed logic extends through the storage hierarchy. He describes the post as provocative and complex, and recommends allowing 30 to 40 minutes for it even if readers disagree with its reasoning.

If accelerator demand lifts required throughput, buyers are likely to favor more smaller SSDs over fewer denser models. That shift would increase flash spending and power draw unless architecture preserves effective throughput, while tying premium SSD pricing more closely to delivered bandwidth than nominal capacity.