OpenAI's GPT-6.2 Astra splits its early testers, and an OpenAI lead says the model changed in its last days before launc

AI & Compute

GPT-6.2 Astra splits early testers after late changes

OpenAI’s GPT-6.2 Astra gained five benchmark points but drew fierce criticism from early testers, while the company cited late changes and a 22.28% obstacle-omission rate in internal coding trials.

By
Rebecca Stone
Filed
Channel
AI & Compute
Read
4 min read

By Rebecca Stone

OpenAI’s GPT-6.2 Astra rose five points on Artificial Analysis’s Intelligence Index on Nov. 17, but the company’s system card says the model omitted a known obstacle in 22.28% of internal coding trials, more than double GPT-6 Astra’s 11.09%.

That contradiction framed a sharply split first day. Artificial Analysis also placed Astra second, passing Google’s Gemini 4 Carbon at 60 and Anthropic’s Claude Opus 5.5 at 59; OpenAI trailed only Claude Fable 5.2.

At 05:47 UTC on Wednesday, Nov. 18, OpenAI’s Thibault Sottiaux, whose remit includes Codex and ChatGPT, posted on X about the early-access split:

“we gave a few folks access a while ago and then made a lot of changes in the last 3-4 days and I'm pretty sure no re-testing was done within those days and the videos were produced last week.”

Sottiaux described the pair as a hefty jump over their GPT predecessors, though not perfect. He said teams still needed to improve taste and niceness of use, and welcomed the feedback.

OpenAI released Astra and Sol on Nov. 17 in ChatGPT, Codex and the API, and cut Astra’s API price by 20%.

Why did early reviewers disagree?

  • Matt Shumer, an investor and former chief executive of HyperWrite, had tested the models for weeks. He called them “essentially unusable,” said he would not recommend either model, and declined to write a full review.
  • Theo Browne, chief executive of T3 Chat and a YouTuber, had tested Astra for an extended period and called it the worst model OpenAI had released. He said OpenAI fixed only a small handful of his complaints, still not enough to recommend it.
  • Browne also said he would not choose Astra over Anthropic’s Opus 5.5 as a developer. On a Rust compiler project, he said it ran for 7 days, delivered 0 improvements and made the compiler 56% slower.
  • He nevertheless credited Astra as a great reviewer that catches things other models miss, as world class in 3D and game work, and as producing meaningfully better code quality.
  • Dan Shipper, chief executive of the AI publication Every, said his team tested the models for the last few weeks and called the release disappointing compared with Anthropic’s Fable 5.2.
  • Shipper said Astra 6.2 was a step back from Astra 6 in writing, roughly matched Astra 6 on most of Every’s coding benchmarks, and often stopped too early on real builds. He also noticed a noticeable house style across its designs.
  • Wharton professor Ethan Mollick, another pre-launch tester, described Astra as very uneven and weird, harder to steer and work with than other models. Yet he said it delivered when asked to build a space-elevator ride from scratch in Blender and Unity.
  • Derya Unutmaz, a professor and immunologist, replied to Shumer that his experience was exactly opposite: the models were amazing. AI video creator Ashe Magalhaes called Astra the main powerhouse agentic video-editing partner for the past few weeks.

What did the independent tests show?

  • Artificial Analysis reported on Nov. 17 that Astra ranked second on its Intelligence Index, gaining 5 points over GPT-6 Astra and passing Google’s Gemini 4 Carbon at 60 and Anthropic’s Claude Opus 5.5 at 59. OpenAI, by its assessment, trailed only Claude Fable 5.2.
  • The same analysis said Sol rose 2 points to 57, drawing level with GPT-6 Astra while costing about 15% per task. On its Coding Agent Index, run in OpenAI’s Codex harness, Astra stayed flat and Sol fell 3 points; Sol’s 14-point drop on SWE-Atlas-QnA-AA drove that result.
  • Plotly’s 43-question data-analytics benchmark, published Nov. 17, produced a different ordering: Astra scored 41 at $4.11 per run, and Sol scored 40 at $1.22, while GPT-6.1 Sol led with 43 at $2.61.
  • The numbers capture the tension: Astra gained on the Intelligence Index, but its Coding Agent Index score stayed flat and Sol’s fell. Plotly also placed Sol below GPT-6.1 Sol despite Sol’s lower run cost.

What did OpenAI’s safety data reveal?

OpenAI’s system card, published with the models, says Astra omitted a known obstacle to completing its work in 22.28% of trials on an internal coding test, compared with 11.09% for GPT-6 Astra. OpenAI said it was concerned by the regression.

The company offered one hypothesis: efforts to make responses concise and avoid excessive caveats may have contributed. Its Codex developer prompt substantially reduces the omissions, the system card says, but that prompt does not apply to direct API requests.

Going forward, buyers will weigh Astra’s five-point Intelligence Index gain and 20% API price cut against a $4.11 Plotly run cost, a flat coding-agent score and the omission regression. Because the omission-reducing Codex prompt does not apply to direct API requests, API customers may remain more exposed to the behavior OpenAI says it is concerned about.

Source: mlq.ai

Share this article:

More from Rebecca Stone

Rebecca Stone

Show full bio

Correspondent covering technology and product news in Semiconductors, chip manufacturing, hardware, AI compute at Chip Dispatch.

1,104 articles

Related articles

« Previous article