Benchmarks / tau2-bench

Reported by tau2-bench

tau2-bench

Share of telecom account and technical support tasks completed within policy in all four trials.

Data older than 90 days: last updated 3 Aug 2026

Results dated
24 Feb 2026 to 26 May 2026
Results
8 configurations of 7 models
Unit
% of tasks
Licence
MIT License

tau2-bench in brief

What it measures. tau2-bench is a third-party benchmark, shown here with credit. Measured here: Share of telecom account and technical support tasks completed within policy in all four trials.

Latest results. , 7 models ranked.

Top 3 on Consistency (pass^4) (% of tasks, higher is better, each model at its best setting):

  1. Qwen3.5-397B-A17B (reasoning on) (Alibaba): 92.1%
  2. Claude Opus 4.5 (high reasoning) (Anthropic): 78.1%
  3. Gemini 3 Pro Preview (high reasoning) (Google): 74.6%

How to cite. tau2-bench, via Spring Prompt, results dated 26 May 2026. https://springprompt.com/benchmarks/tau2-bench These results belong to tau2-bench, not to us. Licence: MIT License. Credit tau2-bench first.

Consistency (pass^4): GLM-5

7 results · % of tasks, higher is better. Choose a model to highlight it.Clear highlight

  1. 1 Qwen3.5-397B-A17B (reasoning on)Alibaba 92.1%
  2. 2 Claude Opus 4.5 (high reasoning)Anthropic 78.1%
  3. 3 Gemini 3 Pro Preview (high reasoning)Google 74.6%
  4. 4 GPT-5.2 (high reasoning)OpenAI 71.9%
  5. 5 Gemini 3 Flash Preview (high reasoning)Google 70.2%
  6. 6 Claude Sonnet 4.5 (reasoning on)Anthropic 64.0%
  7. 7 GLM-5 (reasoning on)Z.ai 62.3%

Full results

tau2 v1.0.1, telecom: consistency (pass^4), % of tasks, higher is better
#ModelConsistency (pass^4)
% of tasks, higher is better
Price
$ per million tokens, in / out
1 Qwen3.5-397B-A17B (reasoning on)Alibaba
92.1%
$0.55 / $3.50
2 Claude Opus 4.5 (high reasoning)Anthropic
78.1%
$5 / $25
3 Gemini 3 Pro Preview (high reasoning)Google
74.6%
–
4 GPT-5.2 (high reasoning)OpenAI · best of 2 settings
71.9%
$1.75 / $14
5 Gemini 3 Flash Preview (high reasoning)Google
70.2%
$0.50 / $3
6 Claude Sonnet 4.5 (reasoning on)Anthropic
64.0%
$3 / $15
7 GLM-5 (reasoning on)Z.ai
62.3%
$1 / $3.20

Swipe the table sideways for more columns.

Source: tau2-bench, via Spring Prompt, springprompt.com/benchmarks/tau2-bench. Test your own prompt on these models →

Ranks follow the score as shown, so equal numbers share a rank. Each model is shown at its best setting; show every setting. Results as published by tau2-bench; we do not re-run them.

Progress over time

Consistency (pass^4), plotted at each model's release date, higher is better. Each model counts once, at its best setting. Scores are today's, not launch day's. Hover or tap a mark for its model.

Best score so far

Every model at its release date. The line moves each time a new release beats every earlier one, and each new best is named where there is room.

60%70%80%90%100% Oct 20252026AprJulOct Claude Sonnet 4.5Gemini 3 Pro PreviewClaude Opus 4.5Qwen3.5-397B-A17B

Best score by lab

Each lab's best so far, stepping whenever one of its releases beats its own earlier models.

AlibabaAnthropicGoogleOpenAIZ.ai

60%70%80%90%100% Oct 20252026AprJulOct AlibabaAnthropicGoogleOpenAIZ.ai

What it measures

Share of telecom account and technical support tasks completed within policy in all four trials.

What it does not measure

Not your policies or customers: the tasks and policies are tau2's own, and the customer is simulated by gpt-5.2.

2 results from tau2-bench not ranked here · show why

We rank a result only when we can tie it to a specific model you can use. These are left out:

  • A custom agent harness, not the standard one: 2

tau2-bench by Sierra Research and contributors, MIT License.