[UPDATE] GPT-6's Headline AGI Score Gets a Second Look

When OpenAI released GPT-6 Astra on 4 September, it led with a number that sounded like a finish line: roughly 98.6 percent on ARC-AGI-3, a reasoning test built specifically to be hard for machines and easy for people. Two days on, the number that mattered most in that launch is the one being argued over.

The ARC Prize Foundation, which runs the benchmark, reran Astra on its own provider-neutral harness and got about 62.7 percent. The near-perfect figure, it says, only reproduces under a different setup, a provider-adapter configuration rather than the standard evaluation environment. That is a large gap, and it is the kind of gap that decides headlines.

It would be easy to read this as a gotcha. It is more interesting than that. ARC itself calls the 62.7 percent result a real generational jump over anything that came before it, and notes that Astra reaches it with an efficiency no prior model has shown. The disagreement is not about whether GPT-6 is a serious advance. It is about which number gets to stand in for the whole model, and whether that number should be the one measured on the tester's terms or the maker's.

The rest of the scoreboard is split too

Look past ARC and the picture stays mixed. Artificial Analysis, in an index dated 3 September, put Astra at its highest effort setting around 61, level with one rival and behind both Anthropic's Fable 5.1 and Opus 5. Epoch AI, which blends more than fifty tests into a single score, put Astra clearly in first place. Same model, three defensible rankings, depending on what you choose to measure and how hard you let it think.

None of this is unusual for a frontier release. What is unusual is how much weight OpenAI placed on one clean figure, and how quickly the people who own that figure pushed back. For readers trying to make sense of the original launch, the lesson is a familiar one on this beat: a benchmark score is a claim, not a fact, until someone independent has run it the same way twice.

A rollout that did not help

The launch itself was rocky in more ordinary ways. Paying subscribers found themselves locked out while enterprise customers got Astra first, and Sam Altman described the rollout as messy and promised wider access without giving a date. OpenAI's own researchers also acknowledged that their ability to monitor the model's reasoning is, in their word, fragile, a candid admission that sits awkwardly next to talk of arriving at general intelligence.

That tension is the real story. We covered the question of whether AGI has actually arrived when the model shipped, and the benchmark dispute lands in the same place. GPT-6 Astra looks like the strongest model OpenAI has built. Whether it is the smartest model anyone has built depends on the test, and right now the tests do not agree. That is worth sitting with before the next launch arrives with its own perfect number.