For the last three weeks we ran an experiment we called World Cup Bench: six frontier AI models, each predicting every knockout match of the 2026 FIFA World Cup, completely blind.
Claude Opus 4.8 and Claude Fable 5, GPT-5.5, Gemini 3.1 Pro, GLM-5.2 and Qwen 3.7 Max. Every model got an identical scouting report, frozen to the eve of the tournament: Elo ratings, recent form, head-to-head records, full squads. No internet access. Every training cut-off predates the tournament. Knockout picks were saved before kickoff.
Then we graded all 32 matches against reality: who went through, the exact score, the goalscorers, penalties, red cards.
The headline result was close and fun. GLM-5.2 won the benchmark with 309 points, two ahead of Claude Fable 5. Spain won the actual World Cup, which all six models called before the final.
The two results underneath the headline are the ones worth your time.
The AIs beat the human expert
For a human yardstick we graded the knockout predictions that Chris Sutton published round-by-round on BBC Sport, against the same results.
Through the quarter-finals, the human and the machines were indistinguishable. Sutton went 13/16, 6/8 and 4/4 on who advances. The AI consensus went 13/16, 6/8 and 4/4. Identical, round after round.
Then came the semi-finals, and the human did the most human thing possible: he trusted reputation. He backed France and England. Both lost. He picked France to win a final that France never reached. The models backed the favourites, Spain and Argentina, and banked all three matches.
Final score on winners called: AI consensus 26/31, the human expert 23/31.
To Sutton's enormous credit, he owns the single best call of the tournament. He predicted Norway would beat Brazil, 2-2, through to penalties. That exact upset, with that exact mechanism, fooled all six AI models. Twice, in fact: Norway also knocked out Ivory Coast in a match every model got wrong.
So the machines are the more reliable pundit. The human still produced the one call worth framing.
And then they all lost to a one-line rule
Before the tournament we added the most boring competitor imaginable to the leaderboard: a baseline that always backs the higher-ranked team. One line of code. No reasoning tokens, no squad analysis, no anything.
The baseline cannot name goalscorers, so the fair comparison strips the goalscorer category and scores everyone on the core calls only: who advances, the scoreline, extra time and penalties, red cards.
On those terms, the one-line rule beat every single AI model.
The baseline scored 239 core points. The best AI, Claude Fable 5, scored 235. The benchmark champion, GLM-5.2, scored 225. Six frontier reasoning models, hundreds of thousands of reasoning tokens, and not one of them out-predicted "pick whoever the rankings say is better".
The models' entire winning margin over the baseline lived in one category: naming goalscorers, which the baseline structurally cannot play. That value is real. Knowing that Lamine Yamal is likely to score for Spain is genuine, useful knowledge a ranking table does not contain. But it is depth of knowledge, not judgement. On the judgement calls, the rankings already knew everything the models knew.
Why this happened
The models were a herd. All six picked the same winner in 31 of 34 fixtures they predicted. Gemini 3.1 Pro agreed with the ranking favourite in every single match it predicted. Across roughly 200 picks, there were nine contrarian calls in total, and only three of them landed.
Their confidence barely moved either: a uniform 70 to 73 percent on almost every match, whether it was a mismatch or a coin flip.
We even ran a second, experimental version of the benchmark where, from the round of 16 onward, each model was shown every earlier result plus its own previous picks and scores. Rolling memory instead of a frozen snapshot. Given all of that evidence, across 96 later-round predictions, the number of times a model changed which team it tipped was zero. The full breakdown of that experiment is in the learning section of the report.
Football's ranking systems compress decades of results into one number. It turns out that for the binary question, who wins, that number already contains nearly everything a frontier model can extract from a scouting report. The models add value where breadth of specific knowledge matters. They do not, yet, add judgement.
What we took from it
If you are using AI to forecast anything, three habits from this benchmark will serve you well.
Always score a dumb baseline first. It cost us one line of code and it reframed the entire result. Without it, our headline would have been "AI predicts the World Cup" and it would have been misleading.
Expect the herd. Six different flagship models, given identical inputs, produced near-identical predictions. Model diversity is not opinion diversity. If your use case needs someone to spot the upset, current models will not be that someone.
Put the AI where the detail is. The models genuinely beat the baseline at squad-level knowledge, hitting on about 27 percent of named goalscorers. That is where the value lives: broad, specific, checkable knowledge. Not one-bit verdicts.
The full report has the complete tables, every model's match-by-match picks, the human pundit comparison, the learning experiment, and the methodology with all its disclosed quirks. The tournament is over; the numbers are final.
Spring Prompt builds benchmarks that test AI models on real work, with baselines that keep the answers honest. If you want the raw World Cup Bench dataset, email us.
