Benchmarks / tau2-bench

Reported by tau2-bench

tau2-bench

Share of banking questions answered from a policy knowledge base tasks completed within policy in all four trials.

Data older than 90 days: last updated 3 Aug 2026

Results dated
26 May 2026
Models
1
Unit
% of tasks
Licence
MIT License

tau2-bench in brief

What it measures. tau2-bench is a third-party benchmark, shown here with credit. Measured here: Share of banking questions answered from a policy knowledge base tasks completed within policy in all four trials.

Latest results. , 1 model ranked.

Top 1 on Consistency (pass^4) (% of tasks, higher is better):

  1. GPT-5.2 (no reasoning) (OpenAI): 4.1%

How to cite. tau2-bench, via Spring Prompt, results dated 26 May 2026. https://springprompt.com/benchmarks/tau2-bench These results belong to tau2-bench, not to us. Licence: MIT License. Credit tau2-bench first.

Consistency (pass^4):

1 results · % of tasks, higher is better. Choose a model to highlight it.Clear highlight

  1. 1 GPT-5.2 (no reasoning)OpenAI 4.1%

Full results

tau2 v1.0.1, banking knowledge, qwen_embeddings retrieval: consistency (pass^4), % of tasks, higher is better
#ModelConsistency (pass^4)
% of tasks, higher is better
Price
$ per million tokens, in / out
1 GPT-5.2 (no reasoning)OpenAI
4.1%
$1.75 / $14

Swipe the table sideways for more columns.

Source: tau2-bench, via Spring Prompt, springprompt.com/benchmarks/tau2-bench. Test your own prompt on these models →

Ranks follow the score as shown, so equal numbers share a rank. Results as published by tau2-bench; we do not re-run them.

What it measures

Share of banking questions answered from a policy knowledge base tasks completed within policy in all four trials.

What it does not measure

Not your policies or customers: the tasks and policies are tau2's own, and the customer is simulated by gpt-5.2.

2 results from tau2-bench not ranked here · show why

We rank a result only when we can tie it to a specific model you can use. These are left out:

  • A custom agent harness, not the standard one: 2

tau2-bench by Sierra Research and contributors, MIT License.