Benchmarks / Microsoft STATE-Bench
Share of customer support tasks completed correctly, averaged over repeated runs.
Data older than 90 days: last updated 1 Jul 2026
5 results · % of tasks, higher is better. Choose a model to highlight it.Clear highlight
| # | Model | Customer support task success (pass@1) % of tasks, higher is better | Price $ per million tokens, in / out |
|---|---|---|---|
| 1 | GPT-5.4 (high reasoning)OpenAI |
57.6%
|
$2.50 / $15 |
| 2 | Claude Opus 4.7 (high reasoning)Anthropic |
51.0%
|
$5 / $25 |
| 3 | GPT-5.4 (default reasoning)OpenAI |
47.2%
|
$2.50 / $15 |
| 4 | DeepSeek-V4-Pro (0423)DeepSeek |
45.6%
|
$1.42 / $2.83 |
| 5 | Kimi K2.6Moonshot AI |
45.1%
|
$0.95 / $4 |
Swipe the table sideways for more columns.
Ranks follow the score as shown, so equal numbers share a rank. Results as published by Microsoft STATE-Bench; we do not re-run them.
Customer support task success (pass@1), plotted at each model's release date, higher is better. Each model counts once, at its best setting. Scores are today's, not launch day's. Hover or tap a mark for its model.
Every model at its release date. The line moves each time a new release beats every earlier one, and each new best is named where there is room.
Each lab's best so far, stepping whenever one of its releases beats its own earlier models.
OpenAIAnthropicDeepSeekMoonshot AI
Share of customer support tasks completed correctly, averaged over repeated runs.
Not the model alone: it runs inside STATE-Bench's agent loop against simulated users, and versions of the benchmark are not comparable.
We rank a result only when we can tie it to a specific model you can use. These are left out:
STATE-Bench by Microsoft and STATE-Bench contributors, MIT License.