Benchmarks / Artificial Analysis

Reported by Artificial Analysis

Artificial Analysis

Share of Terminal-Bench 2.1 tasks completed, as run by Artificial Analysis with its own harness and prompts.

Last updated 9 Nov 2026

Results dated
20 Oct 2026
Results
188 configurations of 131 models
Unit
% of tasks
Licence
Artificial Analysis commercial data licence

Artificial Analysis in brief

What it measures. Artificial Analysis is a third-party benchmark, shown here with credit. Measured here: Share of Terminal-Bench 2.1 tasks completed, as run by Artificial Analysis with its own harness and prompts.

Latest results. , 131 models ranked.

Top 3 on Terminal-Bench 2.1 (% of tasks, higher is better, each model at its best setting):

  1. Claude Fable 5.1 (max reasoning) (Anthropic): 91.4%
  2. GPT-6 Astra (high reasoning) (OpenAI): 89.9%
  3. GPT-5.6 Sol (extra-high reasoning) (OpenAI): 89.5%

How to cite. Artificial Analysis, via Spring Prompt, results dated 20 October 2026. https://springprompt.com/benchmarks/artificial-analysis These results belong to Artificial Analysis, not to us. Licence: Artificial Analysis commercial data licence. Credit Artificial Analysis first.

Terminal-Bench 2.1: Qwen3.8-27B

Top 15 of 131 results · % of tasks, higher is better. Choose a model to highlight it.Clear highlight

  1. 1 Claude Fable 5.1 (max reasoning)Anthropic 91.4%
  2. 2 GPT-6 Astra (high reasoning)OpenAI 89.9%
  3. 3 GPT-5.6 Sol (extra-high reasoning)OpenAI 89.5%
  4. 4 Claude Opus 5 (max reasoning)Anthropic 89.1%
  5. 5 Qwen3.8-Max (0902)Alibaba 88.8%
  6. 6 Grok 4.6 (high reasoning)xAI 88.4%
  7. 7 GPT-5.6 Terra (max reasoning)OpenAI 88.0%
  8. 8 Gemini 3.8 Flash (high reasoning)Google 87.6%
  9. 9 Qwen3.8-Flash-NextAlibaba 86.1%
  10. 10 Gemini 3.7 Flash (high reasoning)Google 85.8%
  11. 11 Muse Spark 1.3 (extra-high reasoning)Meta 85.4%
  12. 12 Kimi K3 (max reasoning)Moonshot AI 85.0%
  13. 13 Claude Fable 5 (max reasoning)Anthropic 84.6%
  14. 13 Claude Opus 4.8 (max reasoning)Anthropic 84.6%
  15. 15 GPT-5.5 (extra-high reasoning)OpenAI 84.3%
  16. 25 Qwen3.8-27B (extra-high reasoning)Alibaba 79.8%

Full results

Artificial Analysis: Terminal-Bench 2.1, % of tasks, higher is better
#ModelTerminal-Bench 2.1
% of tasks, higher is better
Price
$ per million tokens, in / out
1 Claude Fable 5.1 (max reasoning)Anthropic · best of 5 settings
91.4%
$10 / $50
2 GPT-6 Astra (high reasoning)OpenAI · best of 5 settings
89.9%
$10 / $50
3 GPT-5.6 Sol (extra-high reasoning)OpenAI · best of 6 settings
89.5%
$4 / $20
4 Claude Opus 5 (max reasoning)Anthropic · best of 5 settings
89.1%
$5 / $25
5 Qwen3.8-Max (0902)Alibaba
88.8%
$2 / $6
6 Grok 4.6 (high reasoning)xAI · best of 4 settings
88.4%
$2 / $6
7 GPT-5.6 Terra (max reasoning)OpenAI · best of 6 settings
88.0%
$2 / $12
8 Gemini 3.8 Flash (high reasoning)Google · best of 3 settings
87.6%
$1.50 / $7.50
9 Qwen3.8-Flash-NextAlibaba
86.1%
–
10 Gemini 3.7 Flash (high reasoning)Google · best of 3 settings
85.8%
$1.50 / $7.50
11 Muse Spark 1.3 (extra-high reasoning)Meta · best of 2 settings
85.4%
$1.25 / $4.25
12 Kimi K3 (max reasoning)Moonshot AI · best of 2 settings
85.0%
$3 / $15
13 Claude Fable 5 (max reasoning)Anthropic
84.6%
$10 / $50
13 Claude Opus 4.8 (max reasoning)Anthropic
84.6%
$5 / $25
15 GPT-5.5 (extra-high reasoning)OpenAI · best of 5 settings
84.3%
$5 / $30
15 GLM-5.3-FlashZ.ai
84.3%
$0.15 / $0.50
17 GLM-5.3 (max reasoning)Z.ai
83.9%
$1.40 / $4.40
18 Claude Opus 4.7 (max reasoning)Anthropic
83.1%
$5 / $25
19 Qwen3.8-2.4T-A95BAlibaba
82.0%
$2 / $6
20 Grok 4.5 (high reasoning)xAI
81.6%
$2 / $6
21 Qwen3.8-Max (0803)Alibaba
81.3%
–
22 GPT-5.6 Luna (max reasoning)OpenAI · best of 6 settings
80.9%
$0.20 / $1.20
23 Claude Sonnet 5 (max reasoning)Anthropic · best of 2 settings
80.5%
$2 / $10
24 Muse Spark 1.2 (extra-high reasoning)Meta
80.2%
$1.25 / $4.25
25 Qwen3.8-27B (extra-high reasoning)Alibaba · best of 4 settings
79.8%
$0.50 / $3
26 DeepSeek-V4-Flash (0731, max reasoning)DeepSeek
78.7%
$0.13 / $0.40
26 DeepSeek-V4-Pro (0813, max reasoning)DeepSeek
78.7%
$1.32 / $3.96
26 Gemini 3.5 Flash (high reasoning)Google
78.7%
$1.50 / $9
29 GPT-5.4 (extra-high reasoning)OpenAI
78.3%
$2.50 / $15
30 Muse Spark 1.1 (extra-high reasoning)Meta
77.9%
$1.25 / $4.25
30 GLM-5.2 (max reasoning)Z.ai · best of 2 settings
77.9%
$1.40 / $4.40
32 Gemini 3.6 Flash (high reasoning)Google
77.5%
$1.50 / $7.50
33 Muse Spark Open (extra-high reasoning)Meta
76.4%
$1.25 / $4.25
34 Qwen3.7-MaxAlibaba
74.5%
$1.48 / $4.42
35 DeepSeek-V4-Flash-Vision-Exp (max reasoning)DeepSeek
74.2%
$0.22 / $0.65
36 Gemini 3.1 Pro PreviewGoogle
73.8%
$2 / $12
37 Claude Sonnet 4.6 (max reasoning)Anthropic
71.2%
$3 / $15
38 Kimi K2.7 CodeMoonshot AI
67.4%
$0.95 / $4
39 Kimi K2.6Moonshot AI
65.9%
$0.95 / $4
40 MiniMax-M3MiniMax
65.2%
$0.30 / $1.20
40 MiMo-V2.5-Pro (reasoning on)Xiaomi
65.2%
$0.43 / $0.87
42 DeepSeek-V4-Pro (0423, high reasoning)DeepSeek · best of 2 settings
64.8%
$1.42 / $2.83
43 Hy3Tencent
64.4%
$0.14 / $0.58
44 MiMo-V2.5Xiaomi
63.7%
$0.40 / $2
45 Muse SparkMeta
62.2%
–
46 DeepSeek-V4-Flash (0423, max reasoning)DeepSeek · best of 2 settings
61.8%
$0.05 / $0.45
46 GLM-5.1Z.ai
61.8%
$1.33 / $4.18
48 Qwen3.6-PlusAlibaba
61.4%
$0.33 / $1.95
49 Qwen3.7-PlusAlibaba
61.0%
$0.32 / $1.28
50 Qwen3.6-27BAlibaba · best of 2 settings
60.7%
$0.30 / $3.20
50 GPT-5.4 nano (extra-high reasoning)OpenAI
60.7%
$0.20 / $1.25
52 GPT-5.4 mini (extra-high reasoning)OpenAI
59.2%
$0.75 / $4.50
53 Solar Pro 4Upstage
57.3%
$0.15 / $0.60
54 Claude Sonnet 4.5 (reasoning on)Anthropic
55.8%
$3 / $15
55 MiniMax-M2.7MiniMax
55.4%
$0.30 / $1.20
56 Inkling SmallThinking Machines
55.1%
$0.45 / $1.20
56 Inkling (extra-high reasoning)Thinking Machines
55.1%
$1 / $4.05
58 Gemini 3.5 Flash-LiteGoogle
53.6%
$0.30 / $2.50
59 GPT-5.1 (high reasoning)OpenAI
52.4%
$1.25 / $10
60 Grok Build 0.1xAI
52.1%
$1 / $2
61 Muse Glimmer (high reasoning)Meta
51.7%
–
62 Qwen3.5-397B-A17BAlibaba
51.3%
$0.55 / $3.50
63 Mistral Medium 3.5Mistral AI
50.6%
$1.50 / $7.50
64 GLM-4.6 (reasoning on)Z.ai
49.4%
$0.50 / $2
65 Qwen3.5-122B-A10BAlibaba · best of 2 settings
47.6%
$0.26 / $2.08
66 DeepSeek-V3.2 (reasoning on)DeepSeek
46.8%
$0.28 / $0.42
67 Kimi K2.5Moonshot AI
45.7%
$0.57 / $2.85
68 GLM-4.7Z.ai
45.3%
$0.60 / $2.20
69 Qwen3.6-35B-A3BAlibaba · best of 2 settings
44.9%
$0.10 / $0.95
69 DeepSeek-V3.1-Terminus (reasoning on)DeepSeek
44.9%
$0.27 / $1
71 Claude Haiku 4.5 (reasoning on)Anthropic
44.2%
$1 / $5
72 Gemma 4 31BGoogle · best of 2 settings
43.4%
$0.15 / $0.40
73 Qwen3.5-35B-A3B (no reasoning)Alibaba
40.8%
$0.15 / $1
74 Grok 4.3 (high reasoning)xAI · best of 2 settings
39.7%
$1.25 / $2.50
75 Gemma 4 26B A4BGoogle
39.0%
$0.09 / $0.30
76 Nemotron 3 Super 120B A12BNVIDIA
38.6%
$0.08 / $0.45
77 Qwen3-Coder-NextAlibaba
38.2%
$0.12 / $0.80
78 Claude Sonnet 4 (reasoning on)Anthropic
36.3%
–
79 GPT-5 (high reasoning)OpenAI
35.2%
$1.25 / $10
80 GPT-5.5 Instant (2026-06-26)OpenAI
34.8%
–
81 Gemini 3.1 Flash-Lite PreviewGoogle
31.1%
$0.25 / $1.50
82 Devstral 2Mistral AI
30.3%
$0.40 / $2
83 Devstral Small 2Mistral AI
29.6%
–
84 Qwen3.5-9BAlibaba · best of 2 settings
29.2%
$0.10 / $0.15
85 Gemini 2.5 ProGoogle
28.5%
$1.25 / $10
86 Gemma 4 12BGoogle
27.3%
–
86 Mercury 2Inception
27.3%
–
88 Mistral Small 3.1 24BMistral AI
26.2%
$0.35 / $0.56
88 gpt-oss-120b (high reasoning)OpenAI · best of 2 settings
26.2%
$0.10 / $0.50
90 Qwen3.5-4B (reasoning on)Alibaba · best of 2 settings
25.8%
–
91 Mistral Small 4Mistral AI
21.0%
$0.15 / $0.60
92 Trinity Large ThinkingArcee AI
20.6%
–
93 DeepSeek-R1DeepSeek
19.1%
–
94 Granite 4.2 8BIBM
18.4%
$0.06 / $0.15
95 DeepSeek-V3DeepSeek
16.9%
$0.26 / $1.03
96 Nova 2 Lite (high reasoning)Amazon
16.1%
$0.30 / $2.50
97 DeepSeek-V3-0324DeepSeek
13.9%
$0.25 / $1
97 Mistral Medium 3.1Mistral AI
13.9%
$0.40 / $2
97 gpt-oss-20b (high reasoning)OpenAI
13.9%
$0.03 / $0.14
100 DiffusionGemma 26B A4BGoogle
12.4%
–
100 Magistral Medium 1.2Mistral AI
12.4%
–
102 Qwen3-235B-A22B-Thinking-2507Alibaba
12.0%
$0.45 / $3.50
102 Mistral Large 3Mistral AI
12.0%
$0.50 / $1.50
102 Solar Pro 3Upstage
12.0%
–
105 GPT-4.1 miniOpenAI
10.1%
$0.40 / $1.60
106 Ministral 3 14BMistral AI
9.7%
$0.20 / $0.20
107 Llama 4 MaverickMeta
7.9%
$0.35 / $1
108 Qwen3-Next-80B-A3B-ThinkingAlibaba
6.7%
–
108 Nemotron 3 Nano 30B A3B (reasoning on)NVIDIA
6.7%
$0.06 / $0.24
110 Mistral Small 3.2 24BMistral AI
5.6%
$0.094 / $0.25
110 GPT-4o mini (2024-07-18)OpenAI
5.6%
$0.15 / $0.60
112 Qwen3-32B (reasoning on)Alibaba
5.2%
$0.14 / $0.40
113 Qwen3-14B (reasoning on)Alibaba
4.9%
$0.10 / $0.22
113 Llama 3.3 70B InstructMeta
4.9%
$0.71 / $0.71
115 Gemma 3 27BGoogle
4.5%
$0.08 / $0.35
115 Magistral Small 1.2Mistral AI
4.5%
–
115 o3-mini (high reasoning)OpenAI
4.5%
–
118 Ministral 3 8BMistral AI
4.1%
$0.15 / $0.15
119 Llama 4 ScoutMeta
3.7%
$0.10 / $0.30
119 GPT-4.1 nanoOpenAI
3.7%
$0.10 / $0.40
119 GPT-5 mini (high reasoning)OpenAI
3.7%
$0.25 / $2
122 Qwen3.5-2B (reasoning on)Alibaba · best of 2 settings
3.0%
–
123 Qwen3-8B (reasoning on)Alibaba
2.2%
–
124 Gemma 4 E4BGoogle
1.9%
–
125 Qwen3-30B-A3B-Thinking-2507Alibaba
1.5%
–
125 Llama 3.1 8B InstructMeta
1.5%
$0.05 / $0.08
127 Qwen3.5-0.8B (no reasoning)Alibaba · best of 2 settings
0.4%
–
127 Gemma 3 4BGoogle
0.4%
$0.04 / $0.05
127 Gemma 4 E2BGoogle
0.4%
–
130 Gemma 3 12BGoogle
0.0%
$0.05 / $0.15
130 Ministral 3 3BMistral AI
0.0%
$0.10 / $0.10

Swipe the table sideways for more columns.

Source: Artificial Analysis, via Spring Prompt, springprompt.com/benchmarks/artificial-analysis. Test your own prompt on these models →

Ranks follow the score as shown, so equal numbers share a rank. Each model is shown at its best setting; show every setting. Results as published by Artificial Analysis; we do not re-run them.

Progress over time

Terminal-Bench 2.1, plotted at each model's release date, higher is better. Each model counts once, at its best setting. Scores are today's, not launch day's. Hover or tap a mark for its model.

Best score so far

Every model at its release date. The line moves each time a new release beats every earlier one, and each new best is named where there is room.

-50%0%50%100% Jul 20242025Jul2026Jul GPT-4o mini (2024-07-18)DeepSeek-V3DeepSeek-R1Mistral Small 3.1 24BGemini 2.5 ProClaude Sonnet 4DeepSeek-V3.1-TerminusClaude Sonnet 4.5Gemini 3.1 Pro PreviewGPT-5.4GPT-5.5Claude Opus 4.8GPT-5.6 SolClaude Fable 5.1

Best score by lab

Each lab's best so far, stepping whenever one of its releases beats its own earlier models. The 6 labs with the best scores today; 14 others not drawn.

AnthropicOpenAIAlibabaxAIGoogleMeta

0%50%100% Jul 20242025Jul2026Jul AnthropicOpenAIAlibabaxAIGoogleMeta

What it measures

Share of Terminal-Bench 2.1 tasks completed, as run by Artificial Analysis with its own harness and prompts.

What it does not measure

Not other harnesses or tools; superseded by 4.0 for newer models.

291 results from Artificial Analysis not ranked here · show why

We rank a result only when we can tie it to a specific model you can use. These are left out:

  • Not on sale through the API providers we track: 276
  • A different snapshot or variant from the model we list: 14
  • An unusual combination of settings: 1

Data sourced from Artificial Analysis. Licence: Artificial Analysis commercial data licence.