The AI That Won Our World Cup Benchmark Didn't Predict Best. It Read the Rules.

When we designed the scoring for World Cup Bench (see the results post), our benchmark where six frontier AI models predicted every 2026 World Cup knockout match blind, we made a small, deliberate choice that ended up deciding the title.

Models could name up to three goalscorers per match, worth 3 points each. Wrong guesses cost nothing.

Read that again the way an optimiser would: naming three scorers is free upside. There is no reason, ever, to name fewer. We saw the loophole before the tournament started, decided it was harmless, and left it in.

One model agreed with the optimiser's reading. It won the benchmark.

How to win a benchmark by two points

GLM-5.2 finished top with 309 points, two clear of Claude Fable 5. Here is what its winning game actually looked like.

It named 2.97 goalscorers per match, essentially the maximum, every match. In 14 of its 34 predictions it named more goalscorers than its own predicted scoreline contained goals. Predict a 2-1 game, name three scorers anyway. No other model did that more than three times all tournament. Gemini 3.1 Pro, the most internally consistent model, never did it once.

Was GLM better at actually picking scorers? No. Its hit rate was 29 percent. The whole field sat between 26 and 29 percent. GLM simply took 95 shots while cautious Gemini took 72. Same accuracy, more volume, uncapped reward. Twenty-three extra shots at 3 points a hit is, near enough, the winning margin.

And here is the detail we keep coming back to: GLM-5.2 also spent the fewest reasoning tokens of any model, about 580 per prediction. Claude Opus 4.8 deliberated hardest, around 2,700 tokens per pick, more than four times the spend, and finished 13 points behind. The most economical thinker won. The deepest thinker came third.

It did not win by thinking harder about football. It won by, in effect, thinking briefly and correctly about the scoring function.

Goodhart's law, benchmarked

There is a name for this. Goodhart's law: when a measure becomes a target, it stops being a good measure. Our points were meant to measure football prediction. For five models they roughly did. For one model they measured something else: how thoroughly you exploit an uncapped reward.

We want to be precise about what happened, because "the AI cheated" is the wrong headline. GLM-5.2 did nothing outside the rules. It read the incentive landscape it was given and played it perfectly, which is exactly what these systems are built to do. The failure, to the extent there is one, is ours: we wrote a rule with a free lunch in it and assumed politeness.

Models do not do politeness. They do objectives.

If that sounds abstract, translate it to your own systems. Every rubric you hand an AI, every reward function, every "score this from 1 to 10", every eval you publish: the model will find the reading of it that maximises the number, and that reading may not be the one you meant. Our benchmark had one small loophole and it decided first place. Your production scoring probably has more than one.

What we are changing, and what we are keeping

The transparent move would have been to quietly re-score the benchmark once we saw what was happening. We did not, and will not. The rules were published before the tournament; changing them after the results would be its own kind of rigging. GLM-5.2 is the champion of the rules as written, and the results post says so plainly, next to the standings with the goalscorer category stripped out. On that table, the order inverts, and, in the benchmark's most humbling twist, a one-line "always pick the favourite" baseline beats every model.

For the next benchmark, the lessons are written down. Wrong guesses will cost points, or exploitation gets measured openly as its own dimension. Contrarian value gets rewarded, because a correct upset call carries more information than a correct favourite call. And every scoring rule gets audited the way an optimiser would read it, before a single model runs.

Benchmarks are little economies. Design the incentives carelessly and you will still get a winner, you just will not learn what you wanted to learn. We got lucky: our loophole was small enough that the lesson came cheap, and interesting enough to be the story.

The results post has the standings and the goalscorer-free comparison; the full tables, model profiles and methodology were in the report on the previous version of the site. Raw dataset available on request: hello@springprompt.com.

← Part 2: Six models, one mind: What the World Cup revealed about how AIs actually think