AI & ComputeLOT 261119 · 5 min read
Three used Nvidia V100s hit 100 tokens per second on Qwen 3.6
Three used Nvidia V100s deliver 100.8 tokens/s on Qwen 3.6 35B in a $1,042 homelab, sustaining 72.7 tokens/s at 128K and outrunning a $5,399 Mac Studio at long context.
Key takeaways
- A $1,042 three-V100 system generated 100.8 tokens/s on Qwen 3.6 35B at 4K context.
- The V100 build delivered 72.7 tokens/s at 128K, above the builder’s 70-token real-time threshold.
- The V100 setup used about 0.02 kWh for a 128K 35B run, counting GPU energy only.
- The tested $5,399 M5 Max Mac Studio generated 61.3 tokens/s at 128K in an 8-bit test.
- Hosted tests produced about 315 tokens/s on two H100s and 455 on one B200 at 128K.

Three decommissioned Nvidia Tesla V100-16GB accelerators produced 100.8 tokens per second on Qwen 3.6 35B at a 4K context in a build with $1,042 of listed hardware, according to a published builder benchmark. The same three-card system held 72.7 tokens per second at 128K context, clearing the benchmark’s 70-token real-time interaction threshold and beating a $5,399 M5 Max Mac Studio at the longest context.
What hardware produced the result?
The post, titled Scaling ‘E-Waste’: LLMs on Multiple Decommissioned GPUs, compared nine builds containing one to three Nvidia K80, P100 or V100-16GB cards on one Supermicro dual-Xeon platform. The fixed system used a Supermicro X10DRG-Q board, two Intel Xeon E5-2687W 12-core CPUs and a Corsair HX1500i power supply.
Here, e-waste means decommissioned enterprise GPUs: a popular low-cost option, but less capable than costly modern accelerators. The test followed a July round that covered 15 such GPUs; unlike that earlier round, every build kept the same board and CPUs.
The builder ran llama.cpp across more than 130 language-model tests, including Qwen 2.5, Gemma 4 and Qwen 3.6. The suite covered Q4, Q8 and F16 precision, context lengths from 4K to 128K, and GPU energy consumption for every run.
The builder said the rig was for “daily driving Qwen 3.6 35B,” while adding that scaling decommissioned GPUs “will only get you so far.” The benchmark establishes a repeatable multi-GPU platform, not a claim that retired accelerators can replace current data-center hardware.
What did the build actually cost?
The listed prices do not necessarily match current eBay quotations. Every total includes a $211 board, $147.70 Corsair HX1500i supply and one $28.20 Xeon CPU, but leaves out RAM, storage, the case and cooling.
- Three K80s: GPUs $270, or $90 each; total $657, rising to $685 with the second CPU.
- Three P100s: GPUs $403.50, or $134.50 each; total $790, rising to $819 with the second CPU.
- Three V100-16GBs: GPUs $655.16, with $436.77 noted per pair; total $1,042, rising to $1,070 with the second CPU.
The K80 and P100 remain inexpensive because they run hot and ship without cooling. The builder made a 3D-printed blower duct for each card and developed a standalone cooler, showing that low purchase prices exclude one of the build’s largest practical costs.
Why does speed change with context?
Each score came from 10 runs of 512 generated tokens, with the builder prefilling contexts to 4K, 32K, 64K or 128K. Qwen 3.6 35B is a mixture-of-experts model that activates about 3 billion of its 35 billion parameters per token; Qwen 3.6 27B is dense.
At Q8, or 8-bit quantization, the builder reports that Qwen 3.6 35B fails on one or two V100s, which explains the three-card primary configuration. At Q4, or 4-bit quantization, two V100s can run the model.
- 3× V100, Q8: 100.8 tokens/s at 4K; 89.4 at 32K; 83.5 at 64K; 72.7 at 128K.
- 2× V100, Q4: 105.4; 96.2; 89.5; 77.1 tokens/s across those contexts.
- 3× P100, Q8: 55.2; 50.9; 47.5; 40.2 tokens/s.
- Qwen 3.6 27B, dense, on the V100 platform: 24.3; 21.6; 18.7; 14.2 tokens/s.
The 35B mixture-of-experts model ran four to five times faster than the dense 27B model, depending on context length. Other dense models of about 30B parameters also trailed: Gemma 4 31B managed 20.9 tokens/s and Qwen 2.5 32B managed 21.0 at 4K.
The mixture-of-experts Gemma 4 26B cleared 70 tokens/s with two V100s at Q8. The P100s reached that threshold only with Qwen 2.5 3B. The builder’s practical conclusion was that adding cards mainly adds memory for larger models, not speed.
Sparse activation, rather than GPU count alone, drives the largest difference between the dense and mixture-of-experts results.
How do the V100s compare with newer systems?
For the M5 Max Mac Studio, the benchmark lists an 8-bit run at 105.5 tokens/s at 4K, 91.1 at 32K, 78.5 at 64K and 61.3 at 128K. The Mac starts at $2,499, while the tested 40-core GPU, 128GB unified-memory and 1TB SSD configuration costs $5,399.
The Mac was slightly faster at 4K and 32K. The V100 system pulled ahead at 64K and 128K, with a 19% advantage at 128K. A separate oMLX result gave the Mac 43.9 tokens/s on an approximately 195K-token prompt, but the machines used different runtimes, limiting direct comparison.
An Nvidia DGX Spark, priced at $4,700 MSRP, produced 45.8 tokens/s in Kleybrink’s dgx-spark-bench with Ollama in thinking mode and short prompts. That was less than half the V100s’ 4K rate.
Hosted benchmarks showed about 315 tokens/s on two H100s and 455 on one B200 at 128K, making the B200 roughly six times the three-V100 box’s result. Parasail, an inference provider, sponsored the comparison using dedicated H100 and B200 deployments, with full 16-bit Qwen 3.6 35B against the V100s’ Q8 version. The builder said modern GPUs “destroy our decommissioned equivalents.”
Newer systems won the raw comparison, while the used V100s offered a different price and energy trade-off. Runtime differences make the percentages directional rather than laboratory-equivalent.
What do energy use and planned tests show?
The three V100s used about 0.02 kWh for a 128K run of the 35B model. The P100 setup used twice as much, while two or three K80s consumed roughly 0.35 to 0.45 kWh. These figures cover GPU energy, not the complete system’s wall consumption.
The V100-16GB cards posted better results than the P100s while consuming less energy. That advantage narrows the appeal of used accelerators once buyers include power delivery and cooling.
The builder wants to map the results to software-factory use, where models receive prompts continuously around the clock and immediate speed matters less. A standalone cooler is nearly ready for release, and the builder plans to recruit beta testers through a mailing list, without giving a date.
On Reddit, the builder also said he intends to compare inference backends and test speculative decoding. The next comparison will show whether this low-cost V100 approach can sustain around-the-clock inference while retaining its energy advantage over P100 and K80 clusters.
Tom's Hardware (Source)
More from Tom Whitfield
Show full bio
Staff writer covering companies, deals and strategy in Semiconductors, chip manufacturing, hardware, AI compute at Chip Dispatch.
1,109 articles