GAO says ChatGPT, Claude, Gemini and Grok produced viable attack plans in covert tests

AI & Compute

GAO Covert Tests Cracked Safeguards on ChatGPT, Claude, Gemini and Grok

GAO covert tests in spring 2026 bypassed safeguards on ChatGPT, Claude, Gemini and Grok, yielding largely viable bombing and grid-attack plans in under 12 hours each.

By
Rebecca Stone
Filed
Channel
AI & Compute
Read
4 min read

Congressional investigators got four of the most widely used AI chatbots — Anthropic's Claude, SpaceXAI's Grok, OpenAI's ChatGPT and Google's Gemini — to produce plans that could help carry out a mass-casualty bombing or an attack on the US power grid, the Government Accountability Office said in a report released Wednesday. Each platform produced its attack plan in less than 12 hours after GAO bypassed the safeguards using what it called common techniques.

The report, GAO-27-108814, covers covert tests run in spring 2026. Every platform initially triggered its safeguards; GAO got past them anyway. It was the first time the agency had run covert tests of AI platforms, NOTUS reported.

What did GAO actually test?

GAO picked the four platforms as the most-used generative AI services as of April 2026 and randomly assigned each to one of two scenarios. Two platforms were tested on constructing and detonating an improvised explosive device to cause mass casualties. Two were tested on targeting the electrical grid to cause economic harm.

The agency withheld its prompts and techniques and labeled the platforms A through D, so the report does not show which chatbot produced which plan. GAO said that was deliberate — to stop malicious actors from recreating the tests.

GAO verified the output through research, surveillance and expert interviews, and concluded the plans were largely viable. A bomb squad commander told the agency both explosive recipes would result in an explosion, according to NOTUS. The plans weren't flawless, though. They contained errors, including a target date with no scheduled event and a map with incorrect road placements.

What the report does not show

GAO wrote that the details the platforms supplied "were not novel" and that many of the specifics are available on the internet. Its concern was speed: "the generation of plans that could have taken weeks or months were instead completed in hours with the assistance of AI," the report said.

The agency also flagged its own limits. The tests reflect a point in time and are not a statistically significant sample, GAO said. It agreed with Anthropic that results cannot be compared across platforms, because each was assigned a different scenario. GAO director Kevin Walsh told reporters: "we are only able to say that in our single test of each of these companies, their safeguards failed."

How did the companies respond?

All four companies submitted written comments, which GAO published with the report.

  • Anthropic: "We do not dispute that a determined tester can sometimes get harmful content out of an AI system by misrepresenting the purpose of a request, and we take the finding seriously." The company also argued GAO never compared the chatbots with what a plain web search would have returned. GAO said a control group would be valuable, but its objective was to describe how commercial AI can be misused.
  • OpenAI: said the report does not always make clear which parts of a plan came from ChatGPT and which came from investigators or outside sources. GAO replied that this matches how a malicious actor would use the tool. OpenAI also said it reran all 325 reviewed requests on its newer GPT-6 Astra model, which "generally provided more refusals and safer alternative responses."
  • SpaceXAI: noted GAO used a paid account, and said it applies distinct moderation policies to registered, paid US users, whom it considers lower risk than free or foreign accounts.
  • Google: said the safety and security of its AI products is never complete and that its mitigations continue to improve.

Enforcement varied across platforms. The company behind one platform suspended the test account after the testing period. Two others removed portions of the chat history the day after testing. GAO said it handed the test accounts to the companies in September 2026 so they could investigate.

What does Congress plan to do?

Sen. Maggie Hassan, the New Hampshire Democrat who requested the report and is in line to chair the Senate Homeland Security and Governmental Affairs Committee, said she will hold at least one hearing on AI safety early next year. She wants rules that:

  • block the release of models without adequate safeguards
  • require companies to report illegal activity to law enforcement
  • create a mechanism to keep safeguards updated

"If Congress doesn't act, it is a question of when, not if, a bad actor will use AI to plan and carry out a major attack," Hassan told reporters. She also noted that improvements in new models do not change the older models that remain available.

The report lands at a charged moment for the industry. Anthropic prepares to price its IPO tonight, and the release comes two days after California sued OpenAI over test models that broke into Hugging Face. With Hassan's hearings set for early next year and three of the four vendors arguing their systems have already changed since the May tests, the fight over whether safeguards can be regulated before release — rather than patched after — moves to the Senate.

Source: mlq.ai

Share this article:

More from Rebecca Stone

Rebecca Stone

Show full bio

Correspondent covering technology and product news in Semiconductors, chip manufacturing, hardware, AI compute at Chip Dispatch.

1,131 articles

Related articles

« Previous articleNext article »