2.8-Trillion-Parameter Kimi K3 Runs on CPUs, Thanks to Dead Optane

A Reddit homelabber has run Moonshot AI's 2.8-trillion-parameter Kimi K3 model entirely on CPU inference — no GPU in the loop — using discontinued Intel Optane PMem 200 memory sticks, a 40-core 3rd-gen Xeon Scalable processor, and a pair of 12W blower fans to stop the modules from cooking the DRAM next door.

The result crawls at 0.85 tokens per second with 1-bit quantization, and drops to 0.75 tokens per second at higher 3-bit quantization. It's slow. It's loud. It works.

The numbers make the point plainly. Kimi K3 is not a home model by any stretch — Moonshot AI itself recommends data center inference engines to run it, and even the smallest community quantization demands at least 610GB of RAM. Secondhand Optane is one of the few realistic paths to that capacity outside a server hall, especially now that DRAM prices have skyrocketed. The builder says the Optane memory cost "a fraction" of the equivalent DRAM.

Why Optane, and why this generation?

Intel launched the Optane Persistent Memory 200 Series in 2020 alongside its 3rd-gen Xeon Scalable CPUs, in 128GB, 256GB, and 512GB capacities. On two-socket Xeon platforms it runs at a higher effective MT/s rate than the first-generation PMem 100, and Intel's figures say it delivers an average of up to 32% more memory bandwidth.

The project is a follow-up to a separate first-gen Optane build earlier this year, and the two builds differ in more than silicon. Optane can run in either Memory Mode, where it acts as system memory with DRAM as cache, or App Direct Mode, where it appears as a separate persistent region the software maps directly. The first build used Memory Mode; the new one uses App Direct, which can give "more control over where the model's tensors live," according to the first build's creator, Reddit user u/APFrisco, who had planned to try the approach.

How slow is 0.85 tokens per second?

Slow enough that a 50,000-token task would take roughly 13 hours at about 1 token per second, as one commenter pointed out. But another commenter countered that overnight batch jobs don't much care about speed — a fair point for a hobby rig.

Both figures compare poorly with the first build's roughly 4 tokens per second. That earlier project had advantages beyond Optane generation: an RTX 3060 handled part of the model alongside Memory Mode, and it ran the smaller, earlier 1-trillion-parameter K2.5 model with a 2-bit quantization.

What about the heat?

Here's where the build gets spicy. The Optane DIMMs get "extremely hot" under sustained inference, according to the builder, hot enough to endanger neighboring DRAM modules. The advice is blunt: "treat active DIMM cooling as mandatory," with the dual 12W blower fans aimed straight at the sticks to keep temperatures under control.

The heat is consistent with Intel's own specifications. The manufacturer rates the modules for forced airflow of 2.4 to 2.7 meters per second — the kind of airflow you find in a data center, not pointed at a homelab's memory slots. Intel rates each module at 12W or 15W regardless of capacity; a four-stick build puts up to 60W of heat right beside the system memory. Throttling kicks in at 83 degrees Celsius, with a thermal trip shutdown at 85 degrees. Typical DDR4 server memory is rated for 85 degrees, extending up to 95 degrees in extended ranges.

What's next for the build?

The current performance numbers are pre-tuning — thread counts and NUMA affinity haven't been optimized yet, so there's headroom. One commenter suggested Memory Mode with DRAM caching for the most-used experts. The builder plans to add GPUs and load parts of the model into DRAM as a faster tier alongside the Optane, promising "more benchmarks to come." For now, the project stands as proof that end-of-life Optane, at scrap prices, can brute-force a data-center-class model onto a desktop — provided you're willing to wait thirteen hours and cool your RAM like a server rack.