Impossible Research — with collaborators at UC Berkeley and Carnegie Mellon — has published a result that reads, at first glance, as the end of a benchmark: ~99% on ARC-AGI-3, the test that has proved exceptionally difficult for frontier models.

The number is the authors' own: self-reported, on the public set, and never run on the held-back set at all. What makes the write-up worth reading is that it spends more effort narrowing its own claim than advertising it.

What ARC-AGI-3 actually asks

ARC-AGI-3 does not show a model puzzles. It drops an agent into a game and refuses to explain it. At each step the agent gets a 64×64 grid of 16 colour indices and a set of legal actions. As the Schema team put it, the environment "supplies no object list, rule sheet, stated goal, or shaped reward."

The agent has to work out what the pixels represent, what its actions do, and what winning even means — while acting on a model of the game that is still provisional.

The metric is Relative Human Action Efficiency (RHAE), which compares an agent's per-level action count against a first-exposure human baseline and aggregates across environments. 100% means completing every level of every environment at or above human-baseline action efficiency. It measures how economically you learn, rather than only whether you finish.

The number that has not moved much

Here is the verified record, from the Schema team's own summary of the field: on the semi-private set, frontier-model performance "rose from 0.51% at launch in March to 7.78% with GPT‑5.6 Sol at max reasoning in July."

Four months of frontier progress, on the evaluation set held back from developers, took the field from roughly half a per cent to under eight.

The same configuration scored 13.33% on the public set — the one anyone can download and iterate against. The authors treat that pair as "one official calibration point across evaluation sets", and caution that it "does not justify numerically extrapolating a near-ceiling public score." Two numbers from one model cannot tell you why the sets diverge. They are enough to establish that a public-set score and a held-back score are not interchangeable.

What the harness did — and the comparison the authors refuse to make

Schema is a wrapper, not a model. It does not, in its authors' words, "change the underlying model weights. Instead, it changes the process around them."

The approach is explicitly modelled on physics: have the model write each game's mechanism as an executable program, test that program against what actually happens, and plan inside it. When a prediction fails, the agent can revise either its rules or its representation of what the objects even are — the two are held in a single editable program, because state-grounding and mechanism-discovery "cannot be resolved independently."

With that wrapper, on the public set: 95.35% using GPT‑5.6 Sol, and 98.98% using Claude Opus 4.8 and Fable 5.

The obvious move is to set 98.98% against the official leaderboard row of 13.33% and call it an eighty-point leap. The authors explicitly decline to. The leaderboard figure, they write, "is not a matched harness comparison," and the resulting gap "is therefore contextual, not a controlled estimate of the gain produced by Schema." It compares a two-configuration fallback pairing against a single best variant, measured under different conditions.

What they offer instead is a controlled comparison that holds the models fixed and changes only the wrapper. Running the same Opus 4.8 and Fable 5 pairing under Claude Code as a general-purpose baseline produces 42.83%; under Schema, "the same pairing reaches 98.98%, an increase of 56.15%." As they put it, that comparison "isolates a difference in process rather than underlying model capability."

Same weights, same evaluation set, same tasks, one variable. That is the load-bearing result, and it is the one least likely to reach a headline.

Three caveats the authors state themselves

The Schema page does not bury its qualifications — it draws them into the chart.

The results are unverified. "Both Schema results are self-reported and have not been verified by ARC Prize." The figure distinguishes filled markers for ARC Prize-verified evaluations from hollow ones for self-reported public-set runs, which places their own results in the hollow category.

They are on the public set, and only there. "What 98.98% public maps to on Semi‑private is unknown until it is measured. The available artifacts document public-set runs only, so this draft makes no frozen-harness or held-out-performance claim." Schema has no semi-private score. The verified 7.78% belongs to GPT‑5.6 Sol, not to Schema's honest counterpart.

The headline number uses a fallback rule. Both scores "come from a fixed fallback rule: Opus 4.8 and Sol xhigh run first; games scoring below 80 are rerun with Fable 5 and Sol max, respectively, and the higher per-game score is retained." A conditional best-of-two per game — and the page says so.

The authors also place their own result in context rather than above it, noting that other teams' self-reported results had "climbed as high as 58.12% mean per-game RHAE", and that "our self-reported 98.98% continues this trajectory rather than breaking from it."

Why it matters

The reflex reading — AI hits 99% on the test that was supposed to stump it — is wrong in a specific and instructive way. On the held-back tasks, the best verified frontier score anyone has is 7.78%, and Schema has not been measured there.

What did move is the scaffolding. If swapping the harness alone lifts the same model pairing from 43% to 99% on the same tasks, without touching a weight, then some meaningful share of measured capability is architecture around the model rather than the model itself. That makes "which model is best" a harder question than a leaderboard row suggests, and it makes cost-per-task — which the ARC Prize leaderboard plots against score — the axis worth reading.

The next thing to watch is narrow and checkable: whether these numbers survive ARC Prize verification, and what they look like on the semi-private set. The authors have not claimed they will.

Background: we have covered what an AI benchmark actually measures, and, when ARC-AGI-2 arrived, how a new benchmark reset the score to zero.