AI & ComputeLOT 261122 · 3 min read
367K-Parameter AI Transformer Runs on a Floppy for 486DX and Pentium DOS PCs
WollyMonjac's Tiny 2.1 squeezes a 367K-parameter, 4-layer transformer in 380KB of 8-bit weights onto a floppy, running in DOS with 540KB of RAM and PC-speaker voice output.
Key takeaways
- Tiny 2.1 is a 367,000-parameter transformer with 4 layers, 96 wide, that fits on one floppy disk
- The model needs about 540KB of free conventional memory and runs on 486DX and Pentium PCs under DOS
- 8-bit weights total 380KB; training covered roughly 3,500 facts in multiple wordings
- The MIT-licensed distribution includes a training folder for building custom question-answering models
- The program speaks responses through the PC speaker and was written in Turbo C 2.0

A developer has packed a working question-answering transformer onto a single floppy disk, and it runs on 486DX and Pentium machines under DOS with roughly 540KB of free conventional memory. The model, Tiny 2.1 by WollyMonjac, carries 367,000 parameters — a figure that sits several orders of magnitude below modern large language models, yet it shares their transformer architecture and runs on hardware designed more than three decades ago.
The numbers behind the project explain why it fits. A formatted floppy of the 486 era holds just under 1,475,000 bytes, and Tiny 2.1 uses that headroom deliberately:
- 367,000 parameters, arranged in 4 layers, 96 wide, reading and writing letter by letter
- 8-bit weights with one scale per row, totaling 380KB
- Trained on about 3,500 facts, each phrased in many different wordings
The developer is candid about what the model is and is not. The GitHub repository describes it simply as "a tiny question-answering transformer written in Turbo C 2.0." The repo also notes the codebase was "Written by Fable 5.2 and trained by Opus 5.5" — a nod to the modern AI tools that produced this deliberately retro artifact.
What does the user actually get?
Tiny 2.1 arrives with a clean text interface that goes beyond a bare prompt. The UI includes a conversation history pane, a neural monitor panel, and an Ask Tiny prompt window, plus the F-key shortcuts that DOS power users expect.
The most retro-futuristic touch is the voice: the program can speak its responses through the PC speaker, giving answers a robotic timbre that no modern sound card emulation quite reproduces.
How does it handle wrong answers?
Like its far larger siblings, the model does not always tell the truth. Monjac admits that outside its roughly 3,500-fact training set, "it makes things up confidently." On hardware this old, hallucination feels less like a defect and more like part of the charm.
The distribution, released under an MIT license, ships with a training folder. A determined user could therefore retrain the tiny transformer on a favorite topic and build a custom question-answering system that still fits on one disk.
Where does this fit in the retro-computing trend?
Tiny 2.1 belongs to a small but growing class of projects that push new software onto old silicon, reversing the usual direction of emulation. Recent examples include a ChatGPT client for DOS, Minecraft running on a Game Boy, generative image creation on the Commodore 64, and Doom playable on an Atari ST.
Running old software on new hardware is easy. Running new ideas on machines built for DOS is harder, and that constraint is precisely what makes projects like this one interesting.
Whether Tiny 2.1 attracts retrainers or remains a curiosity, it demonstrates that the transformer concept compresses far further than current datacenter-scale deployments suggest — 380KB of 8-bit weights is enough to answer questions on a 486, one letter at a time.
Tom's Hardware (Source)
More from Tom Whitfield
Show full bio
Staff writer covering companies, deals and strategy in Semiconductors, chip manufacturing, hardware, AI compute at Chip Dispatch.
1,200 articles