Models / o4-mini

OpenAI

o4-mini

o4-mini is strongest at agents and tool use; it is not on sale through OpenRouter, so it is here for reference.

Results 16 Nov 2026 · 45 results · 6 sources

Price per million tokens
Not available through OpenRouter
Speed
Not measured
Intelligence Index
16.7
Artificial Analysis, at high reasoning
Spring Prompt overall
Not ranked yet
needs our benchmarks and two groups of results
Developer
OpenAI, based in the US
Weights not published
Released
16 Apr 2025

Where it stands among the models you could choose

Available models grouped by price, intelligence, speed and developer location. Retired models are excluded.

AmongIntelligence IndexSpring Prompt overallPrice, blendedSpeed
Models available today 106th of 186Claude Opus 5.5 – – –
Models from developers based in the US 59th of 93Claude Opus 5.5 – – –

The name under each rank is the leader of that group; green is the top quarter of the group and red the bottom quarter, by rank. Price is shown as a percentile: the share of the group that costs less, so lower is cheaper. "Sets this group" marks the measure the group is defined by. Spring Prompt overall is our score out of 100 across every source (how it works). Price is the developer's list price (or the typical OpenRouter host where we haven't read one) for three input tokens to one output; intelligence and speed are from Artificial Analysis (data sourced from Artificial Analysis); "similar intelligence" means within 4 points on its Intelligence Index. Developer locations are where each company is based, not where a model is served. Groups under 3 models are not ranked.

How good is it, and for what?

Each use case ranks the available models its benchmarks measured. The bar shows its percentile in that field, best to the right. Open a row for the results behind it.

Product listingsTurning a sparse product feed and photos into listings that can go live Not measured
  • CatalogBench: has not measured this model.
Decks from an analysisTurning a finished analysis into a deck you could present as it is Not measured
  • DeckBench: has not measured this model.
User surveysPlanning a user survey and reading its results without being misled Not measured
  • SurveyBench: has not measured this model.
Marketing planningPlanning a year of ad spend without overspending Not measured
  • ROASBench: has not measured this model.
Agents and tool useMulti-step tasks with tools: support desks, coding agents, function calls 50th of 160 available
Benchmarko4-miniRank among availableBest available
Berkeley Function Calling Leaderboard (BFCL) V4BFCL: overall accuracy · reported 53.2% 12th of 37 Claude Opus 4.5 77.5%
  • tau2-bench: has not measured this model.
  • OpenHands Index: has not measured this model.
  • Microsoft STATE-Bench: has not measured this model.
  • Artificial Analysis: has not measured this model.
  • Vending-Bench 2: has not measured this model.
Professional workReal tasks from banking, consulting and law, business documents and freelance projects Not measured
  • APEX-Agents: has not measured this model.
  • GDP.pdf: has not measured this model.
  • Remote Labor Index: has not measured this model.
Reasoning and knowledgeHard questions across science, maths and general knowledge 111th of 188 available
Benchmarko4-miniRank among availableBest available
Artificial AnalysisArtificial Analysis: Artificial Analysis Intelligence Index · reported 16.7 106th of 186 Claude Opus 5.5 57.6
Artificial AnalysisArtificial Analysis: Humanity's Last Exam · reported 16.5% 111th of 187 Claude Opus 5.5 61.4%
Artificial AnalysisArtificial Analysis: GPQA Diamond · reported 78.4% 100th of 169 GPT-6 Astra 96.3%
WritingWhat people prefer in blind comparisons, and judged writing quality 95th of 170 available
Benchmarko4-miniRank among availableBest available
Arena (formerly LMArena)Arena Text: overall · reported 1,391 104th of 154 Gemini 4 Argon 1,525
UGI LeaderboardUGI: writing score · reported 46.9 63rd of 125 Gemini 3.8 Flash 78.6
Sticking to the factsSummarising without inventing things, and factual answers 132nd of 140 available
Benchmarko4-miniRank among availableBest available
Vectara Hallucination LeaderboardVectara: hallucination rate · reported 18.6% 67th of 72 GPT-5.4 nano 3.1%
Arena (formerly LMArena)Arena Text factuality: overall · reported 1,368 107th of 110 Gemini 4 Argon 1,511
SimpleQA Verified (Epoch AI)SimpleQA Verified: correct answers · reported 19.6% 60th of 69 GPT-6 Astra 75.6%
Speed under pressureGood decisions against a real clock (fast chess) Not measured
  • BulletBench: has not measured this model.

Nearest alternatives

Against o4-mini, Intelligence Index 16.7. "Similar intelligence" means within 4 points or better.

Against the alternatives

The models you would most likely weigh it against: the leaders of the groups above.

Scroll sideways to see every alternative.

o4-miniClaude Opus 5.5
most intelligent available today
Claude Fable 5.2
leads overall
GPT-6.1 Sol
OpenAI's best other model
GPT-6.1 Terra
near the top overall
Price per million tokens – $8.00$16.00$4.00$2.00
Tokens a second – 96–53–
Intelligence Index 16.7 57.6–51.8–
Spring Prompt overall – 82908080
Rank among available models, by use case
Product listings – 13th5th2nd4th
Decks from an analysis – 7th4th5th8th
User surveys – 4th1st3rd8th
Marketing planning – 9th1st–2nd
Agents and tool use 50th of 160 3rd–6th7th
Professional work – 3rd–9th–
Reasoning and knowledge 111th of 188 1st–6th22nd
Writing 95th of 170 5th–16th–
Sticking to the facts 132nd of 140 3rd–18th–
Speed under pressure – –8th3rd17th

Shaded figures are better than o4-mini's. A dash means no result.

Where it has been measured

Compared with the models available today (it is not on sale itself), in the bottom quarter for sticking to the facts.

Artificial Analysis lists it but has not published a speed for it yet

Every result

45 published results from 6 sources, each in the source's own units, with the configuration that produced it.

Arena (formerly LMArena) · reported by Arena (formerly LMArena) · 7 results
MeasureValueRankConfigurationDated
Arena Text factuality: overallArena rating, higher is better 1,368
1,358–1,378
171 of 182 o4-mini 2 Oct 2026
Arena Text: business, management and financeArena rating, higher is better 1,379
1,372–1,387
182 of 406 o4-mini 2 Oct 2026
Arena Text: creative writingArena rating, higher is better 1,337
1,329–1,345
191 of 411 o4-mini 2 Oct 2026
Arena Text: expert promptsArena rating, higher is better 1,408
1,395–1,420
165 of 363 o4-mini 2 Oct 2026
Arena Text: instruction followingArena rating, higher is better 1,369
1,362–1,375
185 of 413 o4-mini 2 Oct 2026
Arena Text: overallArena rating, higher is better 1,391
1,387–1,395
172 of 413 o4-mini 2 Oct 2026
Arena Text: writing, literature and languageArena rating, higher is better 1,354
1,348–1,361
190 of 412 o4-mini 2 Oct 2026

Not shown: Not an error rate: claims that cannot be checked on the web are skipped, and preference still carries most of the weight.

Artificial Analysis · reported by Artificial Analysis · 7 results
MeasureValueRankConfigurationDated
Artificial Analysis: Artificial Analysis Intelligence Indexindex score, higher is better 16.7 238 of 413 o4-mini (high reasoning) 20 Oct 2026
Artificial Analysis: GPQA Diamond% of questions, higher is better 78.4% 188 of 360 o4-mini (high reasoning) 20 Oct 2026
Artificial Analysis: Humanity's Last Exam% of questions, higher is better 16.5% 222 of 419 o4-mini (high reasoning) 20 Oct 2026
Artificial Analysis: IFBench% of instructions, higher is better 68.7% 72 of 288 o4-mini (high reasoning) 20 Oct 2026
Artificial Analysis: Long-context reasoning (AA-LCR)% of questions, higher is better 61.0% 251 of 407 o4-mini (high reasoning) 20 Oct 2026
Artificial Analysis: Terminal-Bench Hard% of tasks, higher is better 15.2% 173 of 282 o4-mini (high reasoning) 20 Oct 2026
Artificial Analysis: τ²-bench telecom% of tasks, higher is better 55.6% 154 of 286 o4-mini (high reasoning) 20 Oct 2026

Not shown: Not business work, and a blend: read the parts for any one task.

Berkeley Function Calling Leaderboard (BFCL) V4 · reported by Berkeley Function Calling Leaderboard (BFCL) V4 · 16 results
MeasureValueRankConfigurationDated
BFCL: irrelevance detection% correct, higher is better 83.9% 40 of 109 o4-mini (bfcl fc) 16 Dec 2025
BFCL: irrelevance detection% correct, higher is better 87.2% 23 of 109 o4-mini (bfcl prompt) 16 Dec 2025
BFCL: memory% correct, higher is better 34.2% 21 of 109 o4-mini (bfcl fc) 16 Dec 2025
BFCL: memory% correct, higher is better 35.3% 20 of 109 o4-mini (bfcl prompt) 16 Dec 2025
BFCL: multi-turn tasks% correct, higher is better 41.8% 26 of 109 o4-mini (bfcl fc) 16 Dec 2025
BFCL: multi-turn tasks% correct, higher is better 16.6% 54 of 109 o4-mini (bfcl prompt) 16 Dec 2025
BFCL: overall accuracy% correct, higher is better 53.2% 21 of 109 o4-mini (bfcl fc) 16 Dec 2025
BFCL: overall accuracy% correct, higher is better 50.3% 28 of 109 o4-mini (bfcl prompt) 16 Dec 2025
BFCL: relevance detection% correct, higher is better 81.3% 41 of 109 o4-mini (bfcl fc) 16 Dec 2025
BFCL: relevance detection% correct, higher is better 81.3% 41 of 109 o4-mini (bfcl prompt) 16 Dec 2025
BFCL: single-turn calls (curated)% correct, higher is better 37.7% 103 of 109 o4-mini (bfcl fc) 16 Dec 2025
BFCL: single-turn calls (curated)% correct, higher is better 81.3% 73 of 109 o4-mini (bfcl prompt) 16 Dec 2025
BFCL: single-turn calls (user-contributed)% correct, higher is better 66.1% 77 of 109 o4-mini (bfcl fc) 16 Dec 2025
BFCL: single-turn calls (user-contributed)% correct, higher is better 70.8% 54 of 109 o4-mini (bfcl prompt) 16 Dec 2025
BFCL: web search% correct, higher is better 75.5% 10 of 109 o4-mini (bfcl fc) 16 Dec 2025
BFCL: web search% correct, higher is better 71.5% 15 of 109 o4-mini (bfcl prompt) 16 Dec 2025

Not shown: Not general refusal or safety behaviour.

SimpleQA Verified (Epoch AI) · reported by SimpleQA Verified (Epoch AI) · 2 results
MeasureValueRankConfigurationDated
SimpleQA Verified: correct answers% of questions, higher is better 18.8% 74 of 85 o4-mini (high reasoning) 27 Aug 2026
SimpleQA Verified: correct answers% of questions, higher is better 19.6% 72 of 85 o4-mini (low reasoning) 27 Aug 2026

Not shown: Not answers grounded in your documents; tests what the model remembers.

UGI Leaderboard · reported by UGI Leaderboard · 9 results
MeasureValueRankConfigurationDated
UGI: requested-length error% off the requested word count, lower is better 19.0% 171 of 379 o4-mini (high reasoning) 4 Oct 2025
UGI: requested-length error% off the requested word count, lower is better 36.0% 290 of 379 o4-mini (low reasoning) 4 Oct 2025
UGI: requested-length error% off the requested word count, lower is better 27.0% 247 of 379 o4-mini (medium reasoning) 4 Oct 2025
UGI: style adherencescore from 0 to 1, higher is better 0.31 317 of 379 o4-mini (high reasoning) 4 Oct 2025
UGI: style adherencescore from 0 to 1, higher is better 0.33 250 of 379 o4-mini (low reasoning) 4 Oct 2025
UGI: style adherencescore from 0 to 1, higher is better 0.31 311 of 379 o4-mini (medium reasoning) 4 Oct 2025
UGI: writing scorescore out of 100, higher is better 43.2 198 of 379 o4-mini (high reasoning) 4 Oct 2025
UGI: writing scorescore out of 100, higher is better 46.9 180 of 379 o4-mini (low reasoning) 4 Oct 2025
UGI: writing scorescore out of 100, higher is better 45.4 184 of 379 o4-mini (medium reasoning) 4 Oct 2025

Not shown: Not other format limits such as character counts or bullet counts.

Vectara Hallucination Leaderboard · reported by Vectara Hallucination Leaderboard · 4 results
MeasureValueRankConfigurationDated
Vectara: answer rate% of documents, higher is better 99.2% 62 of 108 o4-mini (high reasoning) 22 Sep 2026
Vectara: answer rate% of documents, higher is better 98.7% 74 of 108 o4-mini (low reasoning) 22 Sep 2026
Vectara: hallucination rate% of summaries, lower is better 18.6% 98 of 108 o4-mini (high reasoning) 22 Sep 2026
Vectara: hallucination rate% of summaries, lower is better 18.6% 98 of 108 o4-mini (low reasoning) 22 Sep 2026

Not shown: Not a quality score: a low rate usually means content filters were triggered, and hallucination rates are measured on answered documents only.

Test your own prompt.

Benchmarks aren't your task. Run your prompt on the models you could use, with every answer checked and costed.