Benchmarks / tau2-bench

Reported by tau2-bench

tau2-bench

Share of retail orders, returns and exchanges tasks completed within policy, averaged over trials.

Data older than 90 days: last updated 3 Aug 2026

Results dated
24 Feb 2026 to 26 May 2026
Results
8 configurations of 7 models
Unit
% of tasks
Licence
MIT License

tau2-bench in brief

What it measures. tau2-bench is a third-party benchmark, shown here with credit. Measured here: Share of retail orders, returns and exchanges tasks completed within policy, averaged over trials.

Latest results. , 7 models ranked.

Top 3 on Task success (pass^1) (% of tasks, higher is better, each model at its best setting):

  1. Qwen3.5-397B-A17B (reasoning on) (Alibaba): 84.4%
  2. GPT-5.2 (high reasoning) (OpenAI): 81.6%
  3. Claude Opus 4.5 (high reasoning) (Anthropic): 79.6%

How to cite. tau2-bench, via Spring Prompt, results dated 26 May 2026. https://springprompt.com/benchmarks/tau2-bench These results belong to tau2-bench, not to us. Licence: MIT License. Credit tau2-bench first.

Task success (pass^1): Gemini 3 Flash Preview

7 results · % of tasks, higher is better. Choose a model to highlight it.Clear highlight

  1. 1 Qwen3.5-397B-A17B (reasoning on)Alibaba 84.4%
  2. 2 GPT-5.2 (high reasoning)OpenAI 81.6%
  3. 3 Claude Opus 4.5 (high reasoning)Anthropic 79.6%
  4. 4 Gemini 3 Flash Preview (high reasoning)Google 76.8%
  5. 5 Gemini 3 Pro Preview (high reasoning)Google 75.9%
  6. 6 GLM-5 (reasoning on)Z.ai 73.7%
  7. 7 Claude Sonnet 4.5 (reasoning on)Anthropic 72.4%

Full results

tau2 v1.0.1, retail: task success (pass^1), % of tasks, higher is better
#ModelTask success (pass^1)
% of tasks, higher is better
Price
$ per million tokens, in / out
1 Qwen3.5-397B-A17B (reasoning on)Alibaba
84.4%
$0.55 / $3.50
2 GPT-5.2 (high reasoning)OpenAI · best of 2 settings
81.6%
$1.75 / $14
3 Claude Opus 4.5 (high reasoning)Anthropic
79.6%
$5 / $25
4 Gemini 3 Flash Preview (high reasoning)Google
76.8%
$0.50 / $3
5 Gemini 3 Pro Preview (high reasoning)Google
75.9%
–
6 GLM-5 (reasoning on)Z.ai
73.7%
$1 / $3.20
7 Claude Sonnet 4.5 (reasoning on)Anthropic
72.4%
$3 / $15

Swipe the table sideways for more columns.

Source: tau2-bench, via Spring Prompt, springprompt.com/benchmarks/tau2-bench. Test your own prompt on these models →

Ranks follow the score as shown, so equal numbers share a rank. Each model is shown at its best setting; show every setting. Results as published by tau2-bench; we do not re-run them.

Progress over time

Task success (pass^1), plotted at each model's release date, higher is better. Each model counts once, at its best setting. Scores are today's, not launch day's. Hover or tap a mark for its model.

Best score so far

Every model at its release date. The line moves each time a new release beats every earlier one, and each new best is named where there is room.

70%75%80%85%90% Oct 20252026AprJulOct Claude Sonnet 4.5Gemini 3 Pro PreviewClaude Opus 4.5GPT-5.2Qwen3.5-397B-A17B

Best score by lab

Each lab's best so far, stepping whenever one of its releases beats its own earlier models.

AlibabaOpenAIAnthropicGoogleZ.ai

70%75%80%85%90% Oct 20252026AprJulOct AlibabaOpenAIAnthropicGoogleZ.ai

What it measures

Share of retail orders, returns and exchanges tasks completed within policy, averaged over trials.

What it does not measure

Not your policies or customers: the tasks and policies are tau2's own, and the customer is simulated by gpt-5.2.

2 results from tau2-bench not ranked here · show why

We rank a result only when we can tie it to a specific model you can use. These are left out:

  • A custom agent harness, not the standard one: 2

tau2-bench by Sierra Research and contributors, MIT License.