Anthropic warns of self-replicating AI worm threat that doesn't need data centers to self-improve and spread

AI & Compute

Anthropic Demonstrates AI Worm Concepts on $55 TV Boxes

Anthropic's Frontier Red Team showed AI models can set up inference on cheap devices and orchestrate network replication, but the practical gap remains wide.

By
Tom Whitfield
Filed
Channel
AI & Compute
Read
5 min read

By Tom Whitfield

Anthropic's Frontier Red Team has demonstrated two building blocks of a potential AI-powered worm: a model that can set up inference on nine classes of hardware, and a large model that can orchestrate replication across eight networked nodes. The October 12 report, titled "AI models may be capable of self-replicating without access to a datacenter," examines a hypothetical class of malware the company calls a weight-based replicator. Instead of shipping conventional executable code, it would propagate AI model weights together with the software needed to run inference on each compromised device.

The key commercial and security question is not whether this works today. It does not. The question is when the gap between model capability and target hardware closes.

What did the experiments actually show?

The first experiment tested whether small models can configure inference on unfamiliar hardware. Qwen3.5-9B established an inference server on eight of nine tested hardware types, including NVIDIA Jetson Nano devices, Apple Silicon Macs, Xeon servers, a Google TPU, AMD's MI300X, and a PC with an RTX 4090. The team also ran Gemma4-26B-A4B across 15 off-brand Android TV boxes costing roughly $55 each, achieving a combined 0.39 tokens per second.

The second experiment tested orchestration. Using eight Google Cloud nodes running Android 15 emulators, a seeded agent spread between machines through an unauthenticated Android Debug Bridge port, transferring files and standing up an inference server on each target. GLM-5.2, a 744-billion-parameter model larger than 1TB, completed the replication sequence across all eight nodes in four of five trials, and reached seven nodes in the fifth.

The division of labor matters here. GLM-5.2 did not replicate itself onto the emulated nodes, which could not run it. It ran on a separate server and installed the much smaller Llama 3.1 8B on each target. No other tested open-weight model completed the sequence on its own, and the smaller models could not meaningfully ask a larger model for help despite having that capability.

The takeaway is that Anthropic proved a large, externally hosted model can direct a simplified replication sequence, and that small models can set up inference, but it did not prove a small model can independently spread across consumer hardware.

Why does the threat not apply today?

The research exposes a central constraint. Models reliable enough to complete the replication sequence are too large for cheap target devices, and models that fit within those memory budgets cannot reliably finish the job. The report warns that this gap could narrow.

How narrow, and how soon? Using historical gains in general capability measured through the Epoch Capabilities Index, Anthropic projects that a model in the 30-billion-parameter class could reach the required capability level by the end of 2026. That projection is a roadmap, not a demonstrated result, and it does not address efficiency.

Efficiency is the harder problem. Based on measured throughput on the $55 boxes, which carried a relatively generous 4GB of RAM, Anthropic estimates that establishing an inference environment would take roughly 28 to 51 hours for Qwen3.5-9B and 71 to 150 hours for Gemma4-26B-A4B. A conventional worm executes its payload in seconds and can reach tens of thousands of targets in minutes.

The takeaway is that setup time alone makes this class of replicator uncompetitive with ordinary malware on today's hardware.

What would a botnet-sized fleet deliver?

Anthropic ran the arithmetic on scale. A fleet comparable to the Kimwolf botnet, which infected more than two million devices at its peak, could host roughly 133,000 instances of Gemma4-26B-A4B if every 15 devices collectively carried one model. At the measured throughput, benefiting considerably from LAN latency, that fleet would supply about 40 Hopper H100-equivalents of inference capacity.

The comparison takes liberties with the data, and Anthropic acknowledges as much. Two million devices would need to stay infected and operable for months or years, and infecting them at the fastest observed 28-hour stand-up speed would consume around 6,300 agent-years. The result is less inference throughput than a few blades of previous-generation GPUs.

Model weights add a detection problem on top of the efficiency problem. Distributing multi-gigabyte payloads, or maintaining communication between machines running a sharded model, generates network traffic that is easy to spot. Anthropic notes that compromised hosts could pull weights from external repositories rather than from an infected peer, which changes the traffic pattern but not the fundamental bandwidth cost.

The takeaway is that even in the most favorable scenario the researchers sketched, the economics of weight-based replication remain poor.

What is the practical security concern?

The report identifies one near-term risk that does not depend on a full replicator: AI agents can make malware more adaptable to unfamiliar environments and reduce target-specific engineering. A hybrid design, using conventional code for routine operations and AI-driven reasoning only when conditions turn unexpected, could capture much of that benefit without running inference continuously on every infected device.

Even that angle is not entirely new. Security researchers have already documented AI agents using frontier intelligence to write code that compromises secured systems.

The experiments provide evidence that autonomous software configuration and simplified network replication are increasingly feasible. Turning those capabilities into a materially more dangerous class of malware still requires gains in model reliability, hardware compatibility, deployment efficiency, and the ability to defeat real-world security controls, which Anthropic did not test. If the capability curve holds and a 30-billion-parameter model crosses the threshold by late 2026, device makers and endpoint security vendors will face a threat model that today exists only in the lab.

Source: Tom's Hardware

Share this article:

More from Tom Whitfield

Tom Whitfield

Show full bio

Staff writer covering companies, deals and strategy in Semiconductors, chip manufacturing, hardware, AI compute at Chip Dispatch.

455 articles

Related articles

« Previous articleNext article »