AI & ComputeLOT 261122 · 3 min read

Inkling 2 Mini runs its middle layers three times to reason on a laptop

Thinking Machines' Inkling 2 Mini packs 26.4B parameters into 20GB, loops its middle layers three times per token, and matches Qwen3.8 27B at roughly a sixth of the cost.

By Tom Whitfield650 words

Key takeaways

  1. Inkling 2 Mini has 26.4B total parameters, 3.4B active per token, and ships in a ~20GB footprint under Apache 2.0.
  2. The looped design runs two middle six-layer blocks three times per token: 48 layer applications from 24 stored layers.
  3. A non-looped equivalent cost 7.5% less to train but needed 140% more inference compute to match the looped model's loss on long agentic traces.
  4. Artificial Analysis scores it level with Qwen3.8 27B at roughly six times lower cost and four times the speed.
  5. Released Nov. 20, the model runs on 32GB Macs and Nvidia's DGX Spark with a 1-million-token context window.
Thinking Machines releases Inkling 2 Mini, an open looped model that runs on a laptop
Thinking Machines releases Inkling 2 Mini, an open looped model that runs on a laptop — Faraaz Zuberi / unsplash

Thinking Machines Lab released Inkling 2 Mini on Nov. 20, an open-weights model with 26.4 billion total parameters and 3.4 billion active per token that runs on a laptop with 32GB of memory and occupies about 20GB on disk. The lab reports 48.2% on DeepSWE v1.1 and 46.5% on Vals Index v2.2, which it says beats every open-weights model under 100 billion total parameters in its comparison. The weights are on Hugging Face under Apache 2.0.

The model is a looped transformer. Instead of passing each token through the network once, it runs the middle of the network three times, which the lab says lets the model reason more internally and reach answers with fewer generated tokens.

How does the looping work?

Inkling 2 Mini stores 24 layers split into four blocks of six. The first and last blocks run once. The two middle blocks share weights across three passes, so each token receives 48 layer applications from 24 stored layers.

The lab says it looped the middle because those layers carry more abstract reasoning, while the outer layers translate between embeddings and internal representations. In its tests, looping the middle 12 layers beat looping the last 12.

The architecture is a mixture-of-experts design. Counting the repeated passes, each token triggers 5.4 billion weight activations. Training began from a larger shape: the model was pruned from 384 experts to 128 during training, then trained on 1 trillion tokens distilled from a larger internal Inkling model to recover performance.

The economics of the design show up in the lab's scaling numbers. A non-looped model on the same ladder cost 7.5% less to train but needed 140% more inference compute to match the looped version's loss on long agentic traces. In plain terms, the looped shape trades a small amount of training cost for a large cut in inference cost.

What hardware does it need?

Most weights ship in Nvidia's 4-bit NVFP4 format, with the last four layers in 8-bit MXFP8, for the roughly 20GB footprint. The model card recommends 32GB or more of memory on MacBooks, Mac minis and Mac Studios, and lists Nvidia's DGX Spark as supported hardware.

The model handles text and images with a context window of up to 1 million tokens. It is also available on the company's Tinker platform for inference and fine-tuning, and the Hugging Face release includes a research version that can run a variable number of loops.

How does it perform?

On third-party measurement, Artificial Analysis gives the model 20 points on its Intelligence Index, level with Qwen3.8 27B. The Decoder reports that parity comes at roughly six times lower cost and four times the speed. Stanford researchers given early access measured 0.703 on an intelligence-per-watt test on a DGX Spark, against 0.430 for Qwen3.6-35B-A3B.

The lab also publishes weak spots. Looping does not add stored knowledge, so factual recall tracks other models of its size. The model trails Qwen3.8-27B on multi-turn chat, scoring 50.0% on MultiChallenge, and on following many constraints.

What does looping mean for monitoring?

Looped models raise a question for chain-of-thought monitoring: they do more computation before writing anything down, which could hide reasoning from tools that watch a model's output. Thinking Machines says its tests with the variable-loop model found more no-reasoning capability, with the no-CoT time horizon rising from 16 to 37 seconds between one and three passes, but no meaningful loss of monitorability at this scale.

The release follows the lab's larger Inkling 2 Small earlier this month. If the looped design keeps delivering inference savings at larger scales, it gives laptop-class hardware a path to results that previously needed much bigger models.

mlq.ai (Source)

Filed under

Share this article:

More from Tom Whitfield

Tom Whitfield

Show full bio

Staff writer covering companies, deals and strategy in Semiconductors, chip manufacturing, hardware, AI compute at Chip Dispatch.

1,200 articles

Read also

« Previous article