Opus 4.7 on ARC-AGI-3: scores and setup

ARC-AGI-3 evaluates an agent in unfamiliar, interactive environments. The system must explore, infer how an environment works, pursue a goal, and adapt its actions from feedback rather than answer a static question once.

That makes it useful evidence about interactive reasoning, but it also makes the evaluation setup part of the thing being measured.

The primary method is Relative Human Action Efficiency (RHAE). Completion matters, and efficient completion matters: an evaluated system is compared with first-attempt human action baselines, then level and game results are aggregated onto a 0-100 scale.

A higher score is better, but a score is only interpretable alongside the exact environment set, scoring version, action policy, and budget used to produce it.

An ARC-AGI-3 result is a system result. It reflects the model plus the agent loop, prompts, state representation, tool access, action budget, retries, and scoring implementation. Calling it a bare model score hides variables that may materially change performance.

The useful comparison question is therefore: what did the evaluation hold constant, and what did it allow each system to vary?

Springprompt will display a result only with a complete evaluation contract and a reviewed source snapshot. Results produced under materially different harnesses, tools, budgets, sampling policies, or scoring protocols will be shown as distinct setups rather than merged into one model number.

This filtered table contains only the published Opus 4.7 configurations from the complete official comparison. It does not hide other model families from the benchmark page or merge results from a different evaluation track.