DapuStor Maps KV Cache Offload to High-Capacity NVMe SSDs

By Grace Kim

PCIe Gen5 NVMe SSDs have reached capacities of 245 TB, according to DapuStor’s five-page KV cache offload explainer.

The document describes how evicted KV cache datasets can move into external NVMe storage. It examines transfer mechanisms in vLLM and LMCache.

Why does AI inference need an external storage tier?

AI inference is memory-limited, DapuStor says. The first constraint is the high-bandwidth memory attached to a GPU. The next comes from a tiered cache scheme. It runs through x86 host DRAM, then internal and external NVMe SSDs.

Offloading large KV cache datasets to a more cost-effective external storage tier is emerging as a practical approach for scaling inference clusters, the company says.

The document frames this move as a solution to the recomputation delay problem. It presents NVMe SSDs as a cost-efficient storage layer for AI infrastructure.

The explanation focuses on transfer efficiency and bandwidth utilization. It offers no measured latency, power or total-cost results.

How do vLLM and LMCache make the offload effective?

The explainer discusses the mechanisms used by vLLM and LMCache to make SSD-based KV cache more effective. Its supporting material includes text, diagrams and two Python code examples.

DapuStor states: “Both vLLM and LMCache support packing KV Cache data from multiple tokens for transfer, improving memory efficiency and I/O efficiency.”

Packing creates larger transfer chunks, and external storage can arrange those chunks into sequential, large-block I/O patterns that favor flash-friendly access and better bandwidth utilization.

Improved bandwidth utilization therefore remains an explainer claim, not a published benchmark. The source provides no throughput numbers, latency measurements or power results for either mechanism.

Where does the 245 TB capacity figure fit?

DapuStor cites PCIe Gen5 NVMe SSD capacities of up to 245 TB. The number points to the storage density available for external KV cache tiers. It does not identify a 245 TB DapuStor SSD or announce a new product family.

The cited capacity opens a path toward petabyte-level external storage, although the explainer supplies no device count, bandwidth target or price for such a pool.

The accompanying commentary expresses hope for PCIe Gen5 SSD capacity beyond 245 TB next year. That is an aspiration, not a confirmed shipment schedule or supplier roadmap.

What does the publication establish?

DapuStor’s five-page explainer offers a design account of KV cache offload, not a product launch. The material announces no order volume, revenue figure or fab investment.

The publisher has also prepared a generalized version of DapuStor’s paper. That version extends the approach to all NVMe SSDs and is available as a download alongside the article.

The publication advances an architectural reference and illustrative code, but it does not establish a shipping system at petabyte scale or disclose a deployment timeline.

The commercial question now turns on whether high-capacity PCIe Gen5 SSDs can deliver the required bandwidth at an acceptable price, allowing operators to scale inference clusters without extending recomputation delays.