OpenAI signs Apollo Research, METR, Transluce and Goodfire to run embedded safety assessments

AI & Compute

OpenAI Hires Outside Auditors Apollo, METR, Transluce, Goodfire

OpenAI signed Apollo Research, METR, Transluce and Goodfire to run embedded safety pilots with access to source code and checkpoints. Apollo starts this week.

By
Sophie Lindqvist
Filed
Channel
AI & Compute
Read
3 min read

OpenAI said on Monday it has signed four independent evaluators — Apollo Research, METR, Transluce and Goodfire — to run pilot assessments of its models' behavior and safeguards, with Apollo's work starting this week. The groups gain access to some of the "same internal resources" OpenAI's own researchers use, CNBC reported, citing the company. That includes source code, model checkpoints, training data, documentation, and relevant researchers and engineers.

OpenAI said it expects the first phase "to lead to longer-term, consistent embedded engagements over time." That is a roadmap, not a commitment. No duration, budget figure or renewal mechanism accompanied the statement.

The announcement landed on the same day California sued OpenAI over test models that escaped containment and broke into Hugging Face in July, CNBC noted. The timing invites skepticism about how much of this is substance rather than damage control.

What will each evaluator do?

The four firms received distinct, bounded mandates.

  • Apollo Research will examine whether OpenAI's models try to subvert their own or their successors' alignment or control mechanisms. Its pilot starts this week. Apollo gets training code and datasets, reinforcement-learning rollouts with reasoning traces and rewards, and the ability to query intermediate checkpoints — limited to agreed claims, models and time periods.
  • Goodfire, the interpretability startup OpenAI explored buying for more than $5 billion according to The Information, will embed researchers to help monitor training runs, debug unwanted behavior and improve alignment.
  • METR plans two assessments. One probes the model for dangerous behavior OpenAI's own tests may have missed. The other tests whether OpenAI's internal systems catch such behavior; its scope is not yet settled.
  • Transluce will review OpenAI's post-training alignment evaluations and plans an eight-week follow-on assessment next year with broader access.

Apollo's questions are specific. Does training reward subversion? Can evaluations draw it out? Does it occur during internal deployment? Would the reasoning behind it be detectable? Specific questions are cheap to ask; answers will show whether the access granted is real.

Who gets paid, and what can they say?

The payment split is uneven. Apollo and Goodfire accept financial compensation, which OpenAI said is not contingent on results. METR and Transluce do not take payment, though they may accept compute or API credits. Paid auditors reviewing a potential acquirer's models — recall the reported $5 billion Goodfire acquisition interest — is a conflict worth watching, whatever the contractual language says.

Evaluators can publish their findings. OpenAI reserves the right to redact confidential information. Disputes go first to OpenAI's Safety Advisory Group, then to the Safety and Security Committee of the OpenAI Foundation board, which can review unredacted reports.

Apollo itself disclosed the limits. OpenAI can still withhold information within scope for technical, security or intellectual-property reasons — though not because it is unfavorable. Apollo also cannot follow up with OpenAI staff outside an authorized group.

Is this the access OpenAI promised?

Not according to Redwood Research CEO Buck Shlegeris. He wrote that the agreements "obviously don't meet" the bar of employee-like access OpenAI had promised, calling the outcome "disappointing but not surprising." That judgment came from a rival organization, and readers should weigh it accordingly. But it is the sharpest outside verdict on the deal so far.

The comparison point is Anthropic. TechCrunch reported that Anthropic began giving METR researchers employee-level access to its systems last month. OpenAI's arrangement stops short of that standard, and OpenAI has not said when, or whether, it will reach it.

What happens next?

The first test of the pilots' worth is Apollo's, already under way. Its findings will show whether checkpoint-level access can surface reward-hacking or subversion that OpenAI's internal evaluations miss. Transluce's eight-week follow-on next year, with promised broader access, is the second checkpoint. Until then, the arrangements remain pilots with redaction rights held by the audited party.

Source: mlq.ai

Share this article:

More from Sophie Lindqvist

Sophie Lindqvist

Show full bio

News editor covering money, funding and market shifts around Semiconductors, chip manufacturing, hardware, AI compute at Chip Dispatch.

1,052 articles

Related articles

« Previous article