Models / Claude Sonnet 4.5
Claude Sonnet 4.5
Claude Sonnet 4.5 is best for writing; pick it when quality matters more than cost. The cheapest model of similar intelligence is Gemma 4 26B A4B, at $0.14 per million tokens against $6.00.
Superseded. Claude Sonnet 5.5 is Anthropic's newer model in this line, at $4.00 blended: see Claude Sonnet 5.5 or compare the two.
Retiring 30 Nov 2026. Anthropic retires claude-sonnet-4-5-20250929; the official replacement is claude-sonnet-5-5. How to re-test before you switch.
Results 16 Nov 2026 · 77 results · 10 sources
Test Claude Sonnet 4.5 on your prompt → It runs beside the models you could use instead, with every answer checked and costed.
- Price per million tokens
- $3.00 in · $15.00 out
Typical host on OpenRouter, 10 Nov 2026 · compare prices - Speed
- Not measured
- Intelligence Index
- 20.7
Artificial Analysis, at thinking reasoning; 19.3–20.7 across 2 settings - Spring Prompt overall
- Not ranked yet
needs our benchmarks and two groups of results - Developer
- Anthropic, based in the US
Weights not published - Released
- 29 Sep 2025
1,000,000 tokens of context
Where it stands among the models you could choose
Available models grouped by price, intelligence, speed and developer location. Retired models are excluded.
| Among | Intelligence Index | Spring Prompt overall | Price, blended | Speed |
|---|---|---|---|---|
| Models available today | 96th of 185Claude Opus 5.5 | – | $6.00 blended, 90th percentilecheapest: Mistral Nemo | – |
| Models priced $3–10 per million tokens | 29th of 38Claude Opus 5.5 | – | sets this group | – |
| Models with similar intelligence | sets this group | – | $6.00 blended, 93rd percentilecheapest: Gemma 4 26B A4B | – |
| Models from developers based in the US | 53rd of 92Claude Opus 5.5 | – | $6.00 blended, 81st percentilecheapest: Gemma 3 4B | – |
The name under each rank is the leader of that group; green is the top quarter of the group and red the bottom quarter, by rank. Price is shown as a percentile: the share of the group that costs less, so lower is cheaper. "Sets this group" marks the measure the group is defined by. Spring Prompt overall is our score out of 100 across every source (how it works). Price is the developer's list price (or the typical OpenRouter host where we haven't read one) for three input tokens to one output; intelligence and speed are from Artificial Analysis (data sourced from Artificial Analysis); "similar intelligence" means within 4 points on its Intelligence Index. Developer locations are where each company is based, not where a model is served. Groups under 3 models are not ranked.
How good is it, and for what?
Each use case ranks the available models its benchmarks measured, then those priced $3–10 per million tokens. The bar shows its percentile in that field, best to the right. Open a row for the results behind it.
Product listingsTurning a sparse product feed and photos into listings that can go live
- CatalogBench: has not measured this model.
Decks from an analysisTurning a finished analysis into a deck you could present as it is
- DeckBench: has not measured this model.
User surveysPlanning a user survey and reading its results without being misled
- SurveyBench: has not measured this model.
Marketing planningPlanning a year of ad spend without overspending
- ROASBench: has not measured this model.
Agents and tool useMulti-step tasks with tools: support desks, coding agents, function calls
| Benchmark | Claude Sonnet 4.5 | Rank among available | Best available |
|---|---|---|---|
| tau2-benchtau2 v1.0.1, retail: task success (pass^1) · reported | 72.4% | 6th of 6 | Qwen3.5-397B-A17B 84.4% |
| tau2-benchtau2 v1.0.1, airline: task success (pass^1) · reported | 72.0% | 6th of 6 | Claude Opus 4.5 84.0% |
| tau2-benchtau2 v1.0.1, telecom: task success (pass^1) · reported | 84.9% | 6th of 6 | Qwen3.5-397B-A17B 97.8% |
| Berkeley Function Calling Leaderboard (BFCL) V4BFCL: overall accuracy · reported | 73.2% | 2nd of 36 | Claude Opus 4.5 77.5% |
| OpenHands IndexOpenHands Index: average · reported | 53.0% | 15th of 31 | Claude Fable 5 81.0% |
| Artificial AnalysisArtificial Analysis: Terminal-Bench 4.0 · reported | 0.0% | 67th of 107 | Kimi K3.1 Pro 67.2% |
| Artificial AnalysisArtificial Analysis: Terminal-Bench 2.1 · reported | 55.8% | 51st of 107 | Claude Fable 5.1 91.4% |
| Artificial AnalysisArtificial Analysis: τ-bench banking · reported | 24.5% | 38th of 104 | Grok 4.6 50.7% |
| Vending-Bench 2Vending-Bench 2: money after a year · reported | $3,839 | 32nd of 53 | GPT-6 Astra $15,515 |
- Microsoft STATE-Bench: has not measured this model.
Professional workReal tasks from banking, consulting and law, business documents and freelance projects
| Benchmark | Claude Sonnet 4.5 | Rank among available | Best available |
|---|---|---|---|
| Remote Labor IndexRemote Labor Index: projects done to client standard · reported | 2.1% | 10th of 12 | GPT-6 Astra 20.8% |
- APEX-Agents: has not measured this model.
- GDP.pdf: has not measured this model.
Reasoning and knowledgeHard questions across science, maths and general knowledge
| Benchmark | Claude Sonnet 4.5 | Rank among available | Best available |
|---|---|---|---|
| Artificial AnalysisArtificial Analysis: Artificial Analysis Intelligence Index · reported | 20.7 | 96th of 185 | Claude Opus 5.5 57.6 |
| Artificial AnalysisArtificial Analysis: Humanity's Last Exam · reported | 17.8% | 108th of 186 | Claude Opus 5.5 61.4% |
| Artificial AnalysisArtificial Analysis: GPQA Diamond · reported | 83.4% | 81st of 168 | GPT-6 Astra 96.3% |
WritingWhat people prefer in blind comparisons, and judged writing quality
| Benchmark | Claude Sonnet 4.5 | Rank among available | Best available |
|---|---|---|---|
| Arena (formerly LMArena)Arena Text: overall · reported | 1,457 | 43rd of 153 | Gemini 4 Argon 1,525 |
| UGI LeaderboardUGI: writing score · reported | 66.9 | 19th of 124 | Gemini 3.8 Flash 78.6 |
Sticking to the factsSummarising without inventing things, and factual answers
| Benchmark | Claude Sonnet 4.5 | Rank among available | Best available |
|---|---|---|---|
| Vectara Hallucination LeaderboardVectara: hallucination rate · reported | 12.0% | 54th of 71 | GPT-5.4 nano 3.1% |
| Arena (formerly LMArena)Arena Text factuality: overall · reported | 1,449 | 50th of 109 | Gemini 4 Argon 1,511 |
| SimpleQA Verified (Epoch AI)SimpleQA Verified: correct answers · reported | 30.7% | 54th of 68 | GPT-6 Astra 75.6% |
Speed under pressureGood decisions against a real clock (fast chess)
- BulletBench: has not measured this model.
Nearest alternatives
- Cheaper at similar intelligenceGemma 4 26B A4B$0.14 blended · Intelligence Index 16.7Compare with Claude Sonnet 4.5 →
- Anthropic's newer model in this lineClaude Sonnet 5.5$4.00 blended · Intelligence Index 56.0Compare with Claude Sonnet 4.5 →
- Open weights at similar intelligenceGemma 4 26B A4B$0.14 blended · Intelligence Index 16.7Compare with Claude Sonnet 4.5 →
Against Claude Sonnet 4.5 at $6.00 blended, Intelligence Index 20.7. "Similar intelligence" means within 4 points or better.
Where to run it
7 hosts on OpenRouter at the standard service tier, with a median of $6.00 blended per million tokens. Cheapest first.
| Host | Quantisation | Input | Output | Blended |
|---|---|---|---|---|
| Amazon Bedrock | not stated | $3.00 | $15.00 | $6.00 |
| Anthropic | not stated | $3.00 | $15.00 | $6.00 |
| Azure | not stated | $3.00 | $15.00 | $6.00 |
| Claude Platform on AWS | not stated | $3.00 | $15.00 | $6.00 |
| not stated | $3.00 | $15.00 | $6.00 | |
| Amazon Bedrock | not stated | $3.30 | $16.50 | $6.60 |
| not stated | $3.30 | $16.50 | $6.60 |
Prices per million tokens from OpenRouter's public endpoint list, checked 10 Nov 2026.
Against the alternatives
The models you would most likely weigh it against: the leaders of the groups above.
Scroll sideways to see every alternative.
| Claude Sonnet 4.5 | Claude Opus 5.5 most intelligent at $3–10 per million tokens | Gemma 4 26B A4B cheapest with similar intelligence | Claude Fable 5.2 leads overall | GPT-6.1 Sol near the top overall | |
|---|---|---|---|---|---|
| Price per million tokens | $6.00 | $8.00 | $0.14 | $16.00 | $4.00 |
| Tokens a second | – | 96 | 82 | – | 53 |
| Intelligence Index | 20.7 | 57.6 | 16.7 | – | 51.8 |
| Spring Prompt overall | – | 82 | – | 90 | 80 |
| Rank among available models, by use case | |||||
| Product listings | – | 13th | – | 5th | 2nd |
| Decks from an analysis | – | 7th | – | 4th | 5th |
| User surveys | – | 4th | – | 1st | 3rd |
| Marketing planning | – | 9th | – | 1st | – |
| Agents and tool use | 103rd of 159 | 3rd | 105th | – | 6th |
| Professional work | 40th of 51 | 3rd | – | – | 9th |
| Reasoning and knowledge | 104th of 187 | 1st | 110th | – | 6th |
| Writing | 35th of 169 | 5th | 82nd | – | 16th |
| Sticking to the facts | 94th of 139 | 3rd | 43rd | – | 18th |
| Speed under pressure | – | – | – | 8th | 3rd |
| Side by side → | Side by side → | Side by side → | |||
Shaded figures are better than Claude Sonnet 4.5's. A dash means no result.
Research and examples
Where it has been measured
29th most intelligent of the 38 models priced $3–10 per million tokens. Among models available today, in the top quarter (by percentile) for writing; in the bottom quarter for professional work.
Artificial Analysis lists it but has not published a speed for it yet
- Arena (formerly LMArena)16 results
- Artificial Analysis19 results
- Berkeley Function Calling Leaderboard (BFCL) V416 results
- OpenHands Index6 results
- Remote Labor Index1 result
- SimpleQA Verified (Epoch AI)2 results
- tau2-bench8 results
- UGI Leaderboard6 results
- Vectara Hallucination Leaderboard2 results
- Vending-Bench 21 result
- BulletBenchNot measured
- CatalogBenchNot measured
- DeckBenchNot measured
- ROASBenchNot measured
- SurveyBenchNot measured
- APEX-AgentsNot measured
- GDP.pdfNot measured
- Microsoft STATE-BenchNot measured
Every result
77 published results from 10 sources, each in the source's own units, with the configuration that produced it.
Arena (formerly LMArena) · reported by Arena (formerly LMArena) · 16 results
| Measure | Value | Rank | Configuration | Dated |
|---|---|---|---|---|
| Arena Document: overallArena rating, higher is better | 1,449 1,443–1,455 |
33 of 44 | Claude Sonnet 4.5 | 13 Sep 2026 |
| Arena Search: overallArena rating, higher is better | 1,175 1,170–1,180 |
23 of 34 | Claude Sonnet 4.5 (arena search) | 24 Aug 2026 |
| Arena Text factuality: overallArena rating, higher is better | 1,449 1,446–1,451 |
70 of 182 | Claude Sonnet 4.5 | 2 Oct 2026 |
| Arena Text factuality: overallArena rating, higher is better | 1,447 1,444–1,449 |
76 of 182 | Claude Sonnet 4.5 (high-32k reasoning) | 2 Oct 2026 |
| Arena Text: business, management and financeArena rating, higher is better | 1,461 1,455–1,466 |
59 of 406 | Claude Sonnet 4.5 | 2 Oct 2026 |
| Arena Text: business, management and financeArena rating, higher is better | 1,459 1,453–1,464 |
65 of 406 | Claude Sonnet 4.5 (high-32k reasoning) | 2 Oct 2026 |
| Arena Text: creative writingArena rating, higher is better | 1,454 1,448–1,460 |
33 of 411 | Claude Sonnet 4.5 | 2 Oct 2026 |
| Arena Text: creative writingArena rating, higher is better | 1,450 1,444–1,456 |
37 of 411 | Claude Sonnet 4.5 (high-32k reasoning) | 2 Oct 2026 |
| Arena Text: expert promptsArena rating, higher is better | 1,483 1,475–1,491 |
67 of 363 | Claude Sonnet 4.5 | 2 Oct 2026 |
| Arena Text: expert promptsArena rating, higher is better | 1,498 1,490–1,506 |
49 of 363 | Claude Sonnet 4.5 (high-32k reasoning) | 2 Oct 2026 |
| Arena Text: instruction followingArena rating, higher is better | 1,462 1,457–1,466 |
46 of 413 | Claude Sonnet 4.5 | 2 Oct 2026 |
| Arena Text: instruction followingArena rating, higher is better | 1,464 1,459–1,468 |
45 of 413 | Claude Sonnet 4.5 (high-32k reasoning) | 2 Oct 2026 |
| Arena Text: overallArena rating, higher is better | 1,455 1,452–1,458 |
70 of 413 | Claude Sonnet 4.5 | 2 Oct 2026 |
| Arena Text: overallArena rating, higher is better | 1,457 1,454–1,459 |
65 of 413 | Claude Sonnet 4.5 (high-32k reasoning) | 2 Oct 2026 |
| Arena Text: writing, literature and languageArena rating, higher is better | 1,454 1,449–1,459 |
46 of 412 | Claude Sonnet 4.5 | 2 Oct 2026 |
| Arena Text: writing, literature and languageArena rating, higher is better | 1,452 1,447–1,457 |
51 of 412 | Claude Sonnet 4.5 (high-32k reasoning) | 2 Oct 2026 |
Not shown: Not accuracy or correctness: it ranks which answer voters preferred, with answer length and formatting controlled for.
Artificial Analysis · reported by Artificial Analysis · 19 results
| Measure | Value | Rank | Configuration | Dated |
|---|---|---|---|---|
| Artificial Analysis: Artificial Analysis Coding Indexindex score, higher is better | 52.1 | 95 of 187 | Claude Sonnet 4.5 (thinking reasoning) | 20 Oct 2026 |
| Artificial Analysis: Artificial Analysis Intelligence Indexindex score, higher is better | 19.3 | 217 of 413 | Claude Sonnet 4.5 (no reasoning) | 20 Oct 2026 |
| Artificial Analysis: Artificial Analysis Intelligence Indexindex score, higher is better | 20.7 | 203 of 413 | Claude Sonnet 4.5 (thinking reasoning) | 20 Oct 2026 |
| Artificial Analysis: GPQA Diamond% of questions, higher is better | 72.7% | 233 of 360 | Claude Sonnet 4.5 (no reasoning) | 20 Oct 2026 |
| Artificial Analysis: GPQA Diamond% of questions, higher is better | 83.4% | 147 of 360 | Claude Sonnet 4.5 (thinking reasoning) | 20 Oct 2026 |
| Artificial Analysis: Humanity's Last Exam% of questions, higher is better | 7.2% | 310 of 419 | Claude Sonnet 4.5 (no reasoning) | 20 Oct 2026 |
| Artificial Analysis: Humanity's Last Exam% of questions, higher is better | 17.8% | 217 of 419 | Claude Sonnet 4.5 (thinking reasoning) | 20 Oct 2026 |
| Artificial Analysis: IFBench% of instructions, higher is better | 42.7% | 189 of 288 | Claude Sonnet 4.5 (no reasoning) | 20 Oct 2026 |
| Artificial Analysis: IFBench% of instructions, higher is better | 57.3% | 113 of 288 | Claude Sonnet 4.5 (thinking reasoning) | 20 Oct 2026 |
| Artificial Analysis: Long-context reasoning (AA-LCR)% of questions, higher is better | 54.0% | 266 of 407 | Claude Sonnet 4.5 (no reasoning) | 20 Oct 2026 |
| Artificial Analysis: Long-context reasoning (AA-LCR)% of questions, higher is better | 72.3% | 178 of 407 | Claude Sonnet 4.5 (thinking reasoning) | 20 Oct 2026 |
| Artificial Analysis: SciCode% of problems, higher is better | 45.7% | 143 of 205 | Claude Sonnet 4.5 (thinking reasoning) | 20 Oct 2026 |
| Artificial Analysis: Terminal-Bench 2.1% of tasks, higher is better | 55.8% | 96 of 188 | Claude Sonnet 4.5 (thinking reasoning) | 20 Oct 2026 |
| Artificial Analysis: Terminal-Bench 4.0% of tasks, higher is better | 0.0% | 152 of 204 | Claude Sonnet 4.5 (thinking reasoning) | 20 Oct 2026 |
| Artificial Analysis: Terminal-Bench Hard% of tasks, higher is better | 28.8% | 112 of 282 | Claude Sonnet 4.5 (no reasoning) | 20 Oct 2026 |
| Artificial Analysis: Terminal-Bench Hard% of tasks, higher is better | 35.6% | 71 of 282 | Claude Sonnet 4.5 (thinking reasoning) | 20 Oct 2026 |
| Artificial Analysis: τ-bench banking% of tasks, higher is better | 24.5% | 77 of 175 | Claude Sonnet 4.5 (thinking reasoning) | 20 Oct 2026 |
| Artificial Analysis: τ²-bench telecom% of tasks, higher is better | 70.5% | 127 of 286 | Claude Sonnet 4.5 (no reasoning) | 20 Oct 2026 |
| Artificial Analysis: τ²-bench telecom% of tasks, higher is better | 78.1% | 107 of 286 | Claude Sonnet 4.5 (thinking reasoning) | 20 Oct 2026 |
Not shown: Not your codebase or tools.
Berkeley Function Calling Leaderboard (BFCL) V4 · reported by Berkeley Function Calling Leaderboard (BFCL) V4 · 16 results
| Measure | Value | Rank | Configuration | Dated |
|---|---|---|---|---|
| BFCL: irrelevance detection% correct, higher is better | 86.6% | 26 of 109 | Claude Sonnet 4.5 (bfcl fc) | 16 Dec 2025 |
| BFCL: irrelevance detection% correct, higher is better | 95.0% | 5 of 109 | Claude Sonnet 4.5 (bfcl prompt) | 16 Dec 2025 |
| BFCL: memory% correct, higher is better | 65.0% | 2 of 109 | Claude Sonnet 4.5 (bfcl fc) | 16 Dec 2025 |
| BFCL: memory% correct, higher is better | 5.4% | 89 of 109 | Claude Sonnet 4.5 (bfcl prompt) | 16 Dec 2025 |
| BFCL: multi-turn tasks% correct, higher is better | 61.4% | 9 of 109 | Claude Sonnet 4.5 (bfcl fc) | 16 Dec 2025 |
| BFCL: multi-turn tasks% correct, higher is better | 1.6% | 97 of 109 | Claude Sonnet 4.5 (bfcl prompt) | 16 Dec 2025 |
| BFCL: overall accuracy% correct, higher is better | 73.2% | 2 of 109 | Claude Sonnet 4.5 (bfcl fc) | 16 Dec 2025 |
| BFCL: overall accuracy% correct, higher is better | 24.9% | 89 of 109 | Claude Sonnet 4.5 (bfcl prompt) | 16 Dec 2025 |
| BFCL: relevance detection% correct, higher is better | 68.8% | 80 of 109 | Claude Sonnet 4.5 (bfcl fc) | 16 Dec 2025 |
| BFCL: relevance detection% correct, higher is better | 37.5% | 103 of 109 | Claude Sonnet 4.5 (bfcl prompt) | 16 Dec 2025 |
| BFCL: single-turn calls (curated)% correct, higher is better | 88.7% | 13 of 109 | Claude Sonnet 4.5 (bfcl fc) | 16 Dec 2025 |
| BFCL: single-turn calls (curated)% correct, higher is better | 59.8% | 97 of 109 | Claude Sonnet 4.5 (bfcl prompt) | 16 Dec 2025 |
| BFCL: single-turn calls (user-contributed)% correct, higher is better | 81.1% | 7 of 109 | Claude Sonnet 4.5 (bfcl fc) | 16 Dec 2025 |
| BFCL: single-turn calls (user-contributed)% correct, higher is better | 46.6% | 102 of 109 | Claude Sonnet 4.5 (bfcl prompt) | 16 Dec 2025 |
| BFCL: web search% correct, higher is better | 81.0% | 6 of 109 | Claude Sonnet 4.5 (bfcl fc) | 16 Dec 2025 |
| BFCL: web search% correct, higher is better | 16.0% | 47 of 109 | Claude Sonnet 4.5 (bfcl prompt) | 16 Dec 2025 |
Not shown: Not general refusal or safety behaviour.
OpenHands Index · reported by OpenHands Index · 6 results
| Measure | Value | Rank | Configuration | Dated |
|---|---|---|---|---|
| OpenHands Index: average% resolved, higher is better | 53.0% | 15 of 34 | Claude Sonnet 4.5 (openhands) | 30 Jun 2026 |
| OpenHands Index: front end (SWE-Bench Multimodal)% resolved, higher is better | 36.8% | 11 of 34 | Claude Sonnet 4.5 (openhands) | 30 Jun 2026 |
| OpenHands Index: greenfield (Commit0)% resolved, higher is better | 12.5% | 25 of 34 | Claude Sonnet 4.5 (openhands) | 30 Jun 2026 |
| OpenHands Index: information gathering (GAIA)% resolved, higher is better | 72.7% | 10 of 34 | Claude Sonnet 4.5 (openhands) | 30 Jun 2026 |
| OpenHands Index: issue resolution (SWE-Bench)% resolved, higher is better | 74.2% | 17 of 34 | Claude Sonnet 4.5 (openhands) | 30 Jun 2026 |
| OpenHands Index: testing (SWT-Bench)% resolved, higher is better | 68.8% | 16 of 34 | Claude Sonnet 4.5 (openhands) | 30 Jun 2026 |
Not shown: Not the model alone: the score is for the model inside the OpenHands agent, at the SDK version shown.
Remote Labor Index · reported by Remote Labor Index · 1 result
| Measure | Value | Rank | Configuration | Dated |
|---|---|---|---|---|
| Remote Labor Index: projects done to client standard% of projects, higher is better | 2.1% | 10 of 14 | Claude Sonnet 4.5 | 1 Oct 2026 |
Not shown: Not speed or cost; a small set of projects, so a few points either way are noise.
SimpleQA Verified (Epoch AI) · reported by SimpleQA Verified (Epoch AI) · 2 results
| Measure | Value | Rank | Configuration | Dated |
|---|---|---|---|---|
| SimpleQA Verified: correct answers% of questions, higher is better | 23.7% | 69 of 85 | Claude Sonnet 4.5 | 10 Aug 2026 |
| SimpleQA Verified: correct answers% of questions, higher is better | 30.7% | 64 of 85 | Claude Sonnet 4.5 (thinking-59k reasoning) | 27 Aug 2026 |
Not shown: Not answers grounded in your documents; tests what the model remembers.
tau2-bench · reported by tau2-bench · 8 results
| Measure | Value | Rank | Configuration | Dated |
|---|---|---|---|---|
| tau2 v1.0.1, airline: consistency (pass^4)% of tasks, higher is better | 48.0% | 7 of 8 | Claude Sonnet 4.5 (thinking reasoning, tau2) | 24 Feb 2026 |
| tau2 v1.0.1, airline: task success (pass^1)% of tasks, higher is better | 72.0% | 7 of 8 | Claude Sonnet 4.5 (thinking reasoning, tau2) | 24 Feb 2026 |
| tau2 v1.0.1, banking knowledge, terminal retrieval: consistency (pass^4)% of tasks, higher is better | 10.3% | 1 of 3 | Claude Sonnet 4.5 (thinking reasoning, tau2) | 24 Feb 2026 |
| tau2 v1.0.1, banking knowledge, terminal retrieval: task success (pass^1)% of tasks, higher is better | 25.3% | 2 of 3 | Claude Sonnet 4.5 (thinking reasoning, tau2) | 24 Feb 2026 |
| tau2 v1.0.1, retail: consistency (pass^4)% of tasks, higher is better | 39.5% | 8 of 8 | Claude Sonnet 4.5 (thinking reasoning, tau2) | 24 Feb 2026 |
| tau2 v1.0.1, retail: task success (pass^1)% of tasks, higher is better | 72.4% | 8 of 8 | Claude Sonnet 4.5 (thinking reasoning, tau2) | 24 Feb 2026 |
| tau2 v1.0.1, telecom: consistency (pass^4)% of tasks, higher is better | 64.0% | 6 of 8 | Claude Sonnet 4.5 (thinking reasoning, tau2) | 24 Feb 2026 |
| tau2 v1.0.1, telecom: task success (pass^1)% of tasks, higher is better | 84.9% | 7 of 8 | Claude Sonnet 4.5 (thinking reasoning, tau2) | 24 Feb 2026 |
Not shown: Not your policies or customers: the tasks and policies are tau2's own, and the customer is simulated by gpt-5.2.
UGI Leaderboard · reported by UGI Leaderboard · 6 results
| Measure | Value | Rank | Configuration | Dated |
|---|---|---|---|---|
| UGI: requested-length error% off the requested word count, lower is better | 5.0% | 41 of 379 | Claude Sonnet 4.5 (no reasoning) | 30 Sep 2025 |
| UGI: requested-length error% off the requested word count, lower is better | 8.0% | 63 of 379 | Claude Sonnet 4.5 (thinking reasoning) | 30 Sep 2025 |
| UGI: style adherencescore from 0 to 1, higher is better | 0.38 | 75 of 379 | Claude Sonnet 4.5 (no reasoning) | 30 Sep 2025 |
| UGI: style adherencescore from 0 to 1, higher is better | 0.41 | 16 of 379 | Claude Sonnet 4.5 (thinking reasoning) | 30 Sep 2025 |
| UGI: writing scorescore out of 100, higher is better | 66.9 | 50 of 379 | Claude Sonnet 4.5 (no reasoning) | 30 Sep 2025 |
| UGI: writing scorescore out of 100, higher is better | 64.2 | 75 of 379 | Claude Sonnet 4.5 (thinking reasoning) | 30 Sep 2025 |
Not shown: Not other format limits such as character counts or bullet counts.
Vectara Hallucination Leaderboard · reported by Vectara Hallucination Leaderboard · 2 results
| Measure | Value | Rank | Configuration | Dated |
|---|---|---|---|---|
| Vectara: answer rate% of documents, higher is better | 95.6% | 90 of 108 | Claude Sonnet 4.5 | 22 Sep 2026 |
| Vectara: hallucination rate% of summaries, lower is better | 12.0% | 78 of 108 | Claude Sonnet 4.5 | 22 Sep 2026 |
Not shown: Not a quality score: a low rate usually means content filters were triggered, and hallucination rates are measured on answered documents only.
Vending-Bench 2 · reported by Vending-Bench 2 · 1 result
| Measure | Value | Rank | Configuration | Dated |
|---|---|---|---|---|
| Vending-Bench 2: money after a yearUS dollars, higher is better | $3,839 | 38 of 63 | Claude Sonnet 4.5 | 1 Oct 2026 |
Not shown: Not a real business; one simulated market with set rules.
Will Claude Sonnet 4.5 work for your product?
Benchmarks aren't your task. Paste the prompt your product uses and run it beside the models you could use instead, with every answer checked and costed.
New here? Your email starts a free trial. No credit card needed.