Benchmarks / Open data
Open benchmark data
The public tasks of our own benchmarks and every model's results on them, to download and reuse. For what each benchmark tests and how it is scored, see our benchmarks, explained. The results come from the release the site is serving, 2026-11-10-cad1a1ab581e, built 10 November 2026.
- Credit
- Spring Prompt (springprompt.com)
- Everything
- results.csv · results.json
Downloads
| Benchmark | File | Rows |
|---|---|---|
| BulletBench | bulletbench/results.csv Results, CSV | 460 |
| bulletbench/results.json Results, JSON | 460 | |
| CatalogBench | catalogbench/results.csv Results, CSV | 961 |
| catalogbench/results.json Results, JSON | 961 | |
| catalogbench/tasks.json Public tasks, JSON | 56 | |
| catalogbench/tasks.csv Public tasks, CSV | 56 | |
| DeckBench | deckbench/results.csv Results, CSV | 519 |
| deckbench/results.json Results, JSON | 519 | |
| deckbench/tasks.json Public tasks, JSON | 6 | |
| deckbench/tasks.csv Public tasks, CSV | 6 | |
| ROASBench | roasbench/results.csv Results, CSV | 279 |
| roasbench/results.json Results, JSON | 279 | |
| SurveyBench | surveybench/results.csv Results, CSV | 386 |
| surveybench/results.json Results, JSON | 386 | |
| surveybench/tasks.json Public tasks, JSON | 9 | |
| surveybench/tasks.csv Public tasks, CSV | 9 |
A results row is one measure for one model configuration: the value, its 95% range where there is one, the date, and the sample. Costs are at each run's prices. Each task file holds the full public task as the models saw it, with what it was scored against. Task CSVs carry each task as JSON in one column.
Spring Prompt Score
The Spring Prompt Score for each of the 28 models it ranks, with its percentile on each benchmark, how many benchmarks it covers and the release: score.csv · score.json. Each of our benchmarks becomes a percentile from 0 to 100 among the models with one of our commerce benchmarks (CatalogBench or ROASBench), each model at its best setting, on the headline measure the overall leaderboard uses. The score is the plain average of the percentiles a model has; it needs at least 3. No third-party results are used. Values are to one decimal; the site shows whole numbers.
Using it
You may share and adapt the data for any purpose, including commercially, if you credit Spring Prompt (springprompt.com) and link to this page. Please say which release you used: results change as new runs arrive.
The public tasks are public so you can see exactly what was tested. If you train on them, results on them stop meaning much; our private tasks are held back for that reason.
What is not included
- Third-party benchmark results. Theirs to publish, under their own terms; see each benchmark page for the source and licence.
- Private and development tasks. Held back so the benchmarks stay honest; they never appear on the site.
- Model outputs. Shown on each benchmark page; not in the download yet.
- CatalogBench product images. Generated for the benchmark; not in the download yet.
- Google product categories. CatalogBench's category answers come from Google's product taxonomy, whose terms we have not checked for republishing.
- ROASBench and BulletBench tasks. Their tasks are a market simulator and chess games against Stockfish, run by code rather than stored as task files; their results are included.
- Where's Waldo study images. Third-party images; the study is not a catalogue benchmark.
Score badges
Every model page has a badge for each of our benchmarks it was ranked on, with the code to embed it: /badge/<benchmark>/<model>.svg. It shows the model's rank and score from the current release and links to the benchmark page. A model with a Spring Prompt Score also has /badge/score/<model>.svg.