Jailbroken PS4 runs 9-billion-parameter AI model at up to 43 tokens per second

AI & Compute

Jailbroken PS4 runs 9B-parameter AI model offline at up to 43 tokens/s

A jailbroken PS4 runs Alibaba's Qwen3.5-9B offline at 31-39 tokens per second, thanks to hand-built OpenCL kernels that talk straight to the 2013-era GCN GPU. The 5.66GB app fits 4-bit weights and voice models into 8GB of shared memory.

By
Tom Whitfield
Filed
Channel
AI & Compute
Read
3 min read

A jailbroken PlayStation 4 can run a 9-billion-parameter language model entirely offline at 31 to 39 tokens per second in text chat, and up to 43 tokens per second for code. The project, called Garlic LLM, is the work of developer Leonardo Alegría and is available free as a single 5.66GB download on Hugging Face. It needs no PC and no internet connection — only a console running GoldHEN, the homebrew enabler for exploited PS4 systems.

How does it reach the GPU?

The technical interest lies in what the software does not use. On the PS5, the closest comparable project, PS5LM, runs llama.cpp as a native app and reaches the AMD GPU through PS5_Vulkan, a port of the open-source RADV driver. The PS4 has no such driver layer for homebrew. Garlic LLM uses neither llama.cpp nor a Vulkan port. Instead, it builds the low-level PM4 command buffers itself — the commands a graphics driver would normally generate — and issues them directly to the GPU.

The model's compute operations, such as matrix multiplies, are kernels written in OpenCL C and compiled with LLVM for AMD's Graphics Core Next (GCN) architecture, the design behind the console's 2013-era GPU. The CPU side, written in plain C, handles the tokenizer, the interface, text rendering and audio. The whole build uses the open-source OpenOrbis PS4 toolchain. The takeaway: the app replaces the driver, not just the software stack on top of it.

How does 9 billion parameters fit in 8GB?

The PS4 has 8GB of memory shared between CPU and GPU. At 8 bits per weight, a 9-billion-parameter model would need at least 9GB on its own. Alegría's answer is mixed quantization. Most weights run at 4 bits, while the two largest matrices in each layer use 3-bit weights, about 3.5 bits with scales. That combination brings the model down to roughly 4.5GiB, versus 4.9GiB if it were all 4-bit.

The attention, or KV, cache is also 4-bit, which allows a 16K-token context against the model's native 256K. Two Chatterbox voice models add 0.29GiB and 0.09GiB, and the Parakeet dictation model adds another 0.33GiB, for a total of 5.23GiB, or 5.62GB — a close match to the 5.66GB package size. Installation through the PS4's Package Installer needs about 6GB free, or 12GB if the package is copied to internal disk first.

Where does the speed come from?

Alibaba's Qwen3.5-9B ships with a multi-token prediction (MTP) head, and Garlic LLM uses it to draft up to two tokens ahead while the full model verifies. The vLLM instructions on Alibaba's model card also specify two speculative tokens, so the scheme matches the model's training. The project credits this trick for its chat speed on 2013 hardware. The default temperature is 0.7, in line with Alibaba's recommendation for general use, though the demo video uses 0 for predictable output.

Voice conversation works too. Spoken replies come in seven languages through the Chatterbox speech model at about 22 tokens per second, and a held controller button enables dictation via Nvidia's Parakeet TDT 0.6B v3, which supports 25 European languages. At the mid-range 35 tokens per second, a 300-token answer arrives in under nine seconds.

For context, the PS5 ran the roughly three-times-larger Qwen3.8-27B at about 20 tokens per second, and smaller models — Gemma 4 E4B and Llama 3.2 3B — at 61 and 124 respectively. Alegría tested only on a base/Slim PS4 with firmware 11.52; the PS4 Pro remains untested, with reports requested on GitHub. The PS5 is unsupported, and while the app itself contains no exploit, GoldHEN is installed at the user's own risk. The project shows what twelve-year-old console silicon can still do when someone writes the driver's job by hand — and the PS4 Pro numbers, once reported, will show whether the upgraded GPU widens the margin.

Source: Tom's Hardware

Share this article:

More from Tom Whitfield

Tom Whitfield

Show full bio

Staff writer covering companies, deals and strategy in Semiconductors, chip manufacturing, hardware, AI compute at Chip Dispatch.

1,002 articles

Related articles

« Previous articleNext article »