AMD has acquired AI inference startup Taalas for an undisclosed sum, buying a technology that burns model weights directly into ROM circuits rather than loading them into memory at runtime. The deal, confirmed this week, gives AMD an in-house answer to the decode bottleneck that limits GPU inference latency — the same bottleneck that drove Nvidia's $20 billion acquisition of Grok's team in December.

The problem AMD is buying into is arithmetic, not opinion. GPU clusters handle the prefill stage of inference well, but the decode stage, where a model generates its response, runs with less determinism than low-latency workloads demand. At GTC 2026 in March, Nvidia chief executive Jensen Huang showed that a Grace-Hopper node delivered roughly 100 tokens per second per user before throughput flattened. An NVL72 rack with Grace CPUs and Blackwell B300 GPUs offered about 3.5 times the tokens per second per megawatt at that level; a future Vera-Rubin NVL72 could double that again.

At a Premium tier of 400 TPS per user, Huang's charts showed GPU-only systems approaching zero useful throughput per megawatt. Only by offloading decode onto a massively parallel, SRAM-heavy matrix engine — Grok's LP30 hardware, in Nvidia's demonstration — did an Ultra tier above 1,000 TPS per user open up. The Premium tier on a Vera-Rubin plus Grok combination reached 35 times the performance of a Grace-Blackwell NVL72 alone. The charts were, in effect, the argument for the Grok deal and now for AMD's own moves. GPUs alone, the numbers say, do not carry agentic workloads.

Why AMD needed two deals, not one

AMD's GPUs share the same decode limitation. Its first response was a partnership with Cerebras for disaggregated inference, pairing what is presumed to be the fourth-generation WSE-4 waferscale engine, expected later this year, with AMD's Helios clusters built on Verano Epyc CPUs and Altair MI455X GPUs.

The catch is that AMD does not control Cerebras. The company went public earlier this year and now carries a market capitalization of $50.9 billion on just under $200 million in quarterly revenue. Buying it outright could cost AMD around $60 billion — roughly 300 times annual revenue in a market that typically pays a tenth of that premium. The alternatives were thin: Softbank owns Graphcore, Nvidia took Groq's team, and SambaNova partnered with Intel. Partner it must, for now.

Taalas closes the ownership gap. The company was founded by engineers from Tenstorrent, the AI accelerator and RISC-V specialist, and emerged from stealth in February. Its approach is unorthodox: hard-code the model and its weights into ROM circuits, attach giant SRAM blocks serving as an on-chip KV cache, and ship a chip that runs one model with extreme efficiency. The current HC1 generation stores an 8-billion-parameter model; the planned HC2 targets 20 billion. Interlink a few tens of chips and, in theory, a trillion-parameter model fits.

What changes when the model lives in metal?

Each model requires its own variant of the HC1 chiplet. The SRAM and surrounding components stay the same; two metal layers encoding the weights change. Taalas argues the economics favor this rigidity, claiming it costs 100 times more to train a new generative AI model than to customize an HC chip and buy it in volumes reaching a few hundred thousand units. In initial benchmarks, the company reports much lower latencies and lower cost per token than Nvidia's Blackwell B200 GPUs.

The trade is plain: flexibility for speed. A revised model means a revised chip. Whether customers will accept that constraint is the wager AMD has quietly made.

AMD is saying little beyond a single statement: "AMD plans to integrate the technology into its accelerator roadmap and develop system-level solutions with AMD Instinct GPUs." The phrase "accelerator roadmap" places Taalas silicon beside, not behind, the Instinct line. Expect disaggregated inference with Cerebras in the near term and model-specific silicon further out — assuming customers decide that owning the model means owning the metal too.