Benchmarks / Artificial Analysis

Reported by Artificial Analysis

Artificial Analysis

Share of terminal tasks (software, system and data jobs) an agent completes, as run by Artificial Analysis with its own harness and prompts.

Last updated 20 Oct 2026

Results dated
20 Oct 2026
Results
193 configurations of 118 models
Unit
% of tasks
Licence
Artificial Analysis commercial data licence

Terminal-Bench 4.0: MiniMax M3.1 Flash

Top 15 of 193 results · % of tasks, higher is better. Choose a model to highlight it.Clear highlight

  1. 1 Claude Sonnet 5.5 (max reasoning)Anthropic 63.6%
  2. 2 Claude Opus 5.5 (max reasoning)Anthropic 59.6%
  3. 2 Claude Opus 5.5 (extra-high reasoning)Anthropic 59.6%
  4. 2 GPT-6 Astra (extra-high reasoning)OpenAI 59.6%
  5. 5 GPT-6 Astra (max reasoning)OpenAI 59.1%
  6. 6 Claude Sonnet 5.5 (extra-high reasoning)Anthropic 57.1%
  7. 6 Gemini 4 Argon (high reasoning)Google 57.1%
  8. 8 Claude Opus 5.5 (high reasoning)Anthropic 56.6%
  9. 9 GPT-6.1 Sol (max reasoning)OpenAI 56.1%
  10. 10 Claude Fable 5.1 (extra-high reasoning)Anthropic 55.1%
  11. 11 GPT-6.1 Sol (extra-high reasoning)OpenAI 54.0%
  12. 11 GPT-6 Astra (high reasoning)OpenAI 54.0%
  13. 13 Claude Opus 5.5 (medium reasoning)Anthropic 52.5%
  14. 14 Claude Fable 5.1 (high reasoning)Anthropic 52.0%
  15. 14 Claude Fable 5.1 (max reasoning)Anthropic 52.0%
  16. 49 MiniMax M3.1 FlashMiniMax 25.8%

Full results

Artificial Analysis: Terminal-Bench 4.0, % of tasks, higher is better
#ModelTerminal-Bench 4.0
% of tasks, higher is better		Price
$ per million tokens, in / out
1 Claude Sonnet 5.5 (max reasoning)Anthropic
63.6%
$2 / $10
2 Claude Opus 5.5 (max reasoning)Anthropic
59.6%
$4 / $20
2 Claude Opus 5.5 (extra-high reasoning)Anthropic
59.6%
$4 / $20
2 GPT-6 Astra (extra-high reasoning)OpenAI
59.6%
$10 / $50
5 GPT-6 Astra (max reasoning)OpenAI
59.1%
$10 / $50
6 Claude Sonnet 5.5 (extra-high reasoning)Anthropic
57.1%
$2 / $10
6 Gemini 4 Argon (high reasoning)Google
57.1%
$2 / $10
8 Claude Opus 5.5 (high reasoning)Anthropic
56.6%
$4 / $20
9 GPT-6.1 Sol (max reasoning)OpenAI
56.1%
$2 / $10
10 Claude Fable 5.1 (extra-high reasoning)Anthropic
55.1%
$10 / $50
11 GPT-6.1 Sol (extra-high reasoning)OpenAI
54.0%
$2 / $10
11 GPT-6 Astra (high reasoning)OpenAI
54.0%
$10 / $50
13 Claude Opus 5.5 (medium reasoning)Anthropic
52.5%
$4 / $20
14 Claude Fable 5.1 (high reasoning)Anthropic
52.0%
$10 / $50
14 Claude Fable 5.1 (max reasoning)Anthropic
52.0%
$10 / $50
14 Muse Wonder (max reasoning)Meta
52.0%
$4 / $20
17 GPT-6.1 Sol (high reasoning)OpenAI
51.5%
$2 / $10
18 GPT-6 Astra (medium reasoning)OpenAI
49.5%
$10 / $50
19 Claude Opus 5 (max reasoning)Anthropic
49.0%
$5 / $25
20 GPT-6.1 Sol (medium reasoning)OpenAI
48.0%
$2 / $10
21 Claude Opus 5 (extra-high reasoning)Anthropic
46.5%
$5 / $25
22 Claude Opus 5 (high reasoning)Anthropic
46.0%
$5 / $25
23 Claude Fable 5.1 (medium reasoning)Anthropic
44.9%
$10 / $50
24 Claude Sonnet 5.5 (high reasoning)Anthropic
43.9%
$2 / $10
24 GPT-6 Sol (max reasoning)OpenAI
43.9%
$2 / $10
26 Claude Fable 5 (max reasoning)Anthropic
42.4%
$10 / $50
27 GPT-6 Astra (low reasoning)OpenAI
41.9%
$10 / $50
27 GLM-5.3 (max reasoning)Z.ai
41.9%
$1.40 / $4.40
29 Claude Fable 5.1 (low reasoning)Anthropic
40.4%
$10 / $50
30 GPT-5.6 Sol (max reasoning)OpenAI
39.9%
$4 / $20
31 Qwen3.8-Max (0902)Alibaba
38.9%
$2 / $6
32 GPT-5.6 Terra (max reasoning)OpenAI
35.4%
$2 / $12
33 MiMo-V2.6-ProXiaomi
34.8%
$0.43 / $0.87
33 GLM-5.3 (low reasoning)Z.ai
34.8%
$1.40 / $4.40
35 Claude Opus 5 (medium reasoning)Anthropic
34.3%
$5 / $25
36 DeepSeek-V4.1-Flash-1017 (max reasoning)DeepSeek
33.8%
$0.30 / $1.20
37 Muse Spark 1.3 (max reasoning)Meta
33.3%
$1.25 / $4.25
38 Claude Haiku 5.5 (max reasoning)Anthropic
32.8%
$0.10 / $0.50
38 GLM-5.3-FlashZ.ai
32.8%
$0.15 / $0.50
40 Claude Opus 5.5 (low reasoning)Anthropic
31.3%
$4 / $20
41 GPT-6.1 Sol (low reasoning)OpenAI
30.8%
$2 / $10
42 GPT-6 Sol (extra-high reasoning)OpenAI
30.3%
$2 / $10
43 Claude Sonnet 5.5 (medium reasoning)Anthropic
29.8%
$2 / $10
44 Claude Haiku 5.5 (extra-high reasoning)Anthropic
29.3%
$0.10 / $0.50
45 DeepSeek-V4.1-Flash (max reasoning)DeepSeek
26.8%
$0.30 / $1.20
45 Mistral Large 4Mistral AI
26.8%
$1.36 / $4.18
47 Claude Opus 5 (low reasoning)Anthropic
26.3%
$5 / $25
47 GPT-6 Sol (high reasoning)OpenAI
26.3%
$2 / $10
49 MiniMax M3.1 FlashMiniMax
25.8%
$0.15 / $0.60
49 Grok 4.7 (extra-high reasoning)xAI
25.8%
$2 / $6
51 Qwen3.8-Flash-NextAlibaba
25.3%
–
52 GPT-5.6 Sol (extra-high reasoning)OpenAI
24.7%
$4 / $20
52 Grok 4.7 (high reasoning)xAI
24.7%
$2 / $6
54 MiMo-V2.6-FlashXiaomi
22.7%
$0.14 / $0.28
55 Claude Haiku 5.5 (high reasoning)Anthropic
21.7%
$0.10 / $0.50
55 Claude Opus 4.8 (max reasoning)Anthropic
21.7%
$5 / $25
57 Grok 4.6 (high reasoning)xAI
21.2%
$2 / $6
58 Claude Sonnet 5.5 (low reasoning)Anthropic
20.7%
$2 / $10
58 GPT-5.6 Sol (high reasoning)OpenAI
20.7%
$4 / $20
60 Gemini 3.8 Flash (high reasoning)Google
19.7%
$1.50 / $7.50
60 Gemini 3.8 Flash (medium reasoning)Google
19.7%
$1.50 / $7.50
62 Qwen3.8-Max (0803)Alibaba
18.7%
–
62 GPT-6 Sol (medium reasoning)OpenAI
18.7%
$2 / $10
64 Grok 4.6 (extra-high reasoning)xAI
17.2%
$2 / $6
65 Muse Spark 1.3 (extra-high reasoning)Meta
16.7%
$1.25 / $4.25
66 Grok 4.7 (low reasoning)xAI
16.2%
$2 / $6
67 Claude Haiku 5.5 (medium reasoning)Anthropic
15.2%
$0.10 / $0.50
68 DeepSeek-V4-Pro (0423, max reasoning)DeepSeek
14.6%
$1.42 / $2.83
68 GPT-5.5 (extra-high reasoning)OpenAI
14.6%
$5 / $30
68 GPT-5.6 Sol (medium reasoning)OpenAI
14.6%
$4 / $20
71 Claude Sonnet 5 (max reasoning)Anthropic
14.1%
$2 / $10
71 DeepSeek-V4-Pro (0813, max reasoning)DeepSeek
14.1%
$1.32 / $3.96
73 Gemini 3.7 Flash (high reasoning)Google
13.6%
$1.50 / $7.50
74 GPT-6 Sol (no reasoning)OpenAI
13.1%
$2 / $10
74 Grok 4.6 (medium reasoning)xAI
13.1%
$2 / $6
76 Claude Haiku 5.5 (low reasoning)Anthropic
12.6%
$0.10 / $0.50
76 Kimi K3 (low reasoning)Moonshot AI
12.6%
$3 / $15
76 Kimi K3 (max reasoning)Moonshot AI
12.6%
$3 / $15
76 GPT-5.5 Instant (2026-06-26)OpenAI
12.6%
–
76 GPT-6 Luna (max reasoning)OpenAI
12.6%
$0.10 / $0.50
81 DeepSeek-V4-Flash (0731, max reasoning)DeepSeek
12.1%
$0.14 / $0.28
81 DeepSeek-V4-Flash-Vision-Exp (max reasoning)DeepSeek
12.1%
$0.44 / $1.32
83 GPT-5.6 Luna (max reasoning)OpenAI
11.6%
$0.20 / $1.20
84 Qwen3.8-2.4T-A95BAlibaba
11.1%
$2 / $6
85 Grok 4.5 (high reasoning)xAI
10.6%
$2 / $6
86 Gemini 3.8 Flash (low reasoning)Google
10.1%
$1.50 / $7.50
86 GPT-5.6 Terra (extra-high reasoning)OpenAI
10.1%
$2 / $12
88 GPT-5.5 (high reasoning)OpenAI
9.1%
$5 / $30
88 GPT-6 Sol (low reasoning)OpenAI
9.1%
$2 / $10
90 Muse Spark Open (extra-high reasoning)Meta
8.1%
–
90 GPT-6 Luna (extra-high reasoning)OpenAI
8.1%
$0.10 / $0.50
92 Claude Sonnet 5 (extra-high reasoning)Anthropic
7.1%
$2 / $10
92 Gemini 3.6 Flash (high reasoning)Google
7.1%
$1.50 / $7.50
92 Muse Spark 1.2 (extra-high reasoning)Meta
7.1%
$1.25 / $4.25
95 Gemini 3.5 Flash (high reasoning)Google
6.6%
$1.50 / $9
96 Muse Spark 1.1 (extra-high reasoning)Meta
6.1%
$1.25 / $4.25
97 Qwen3.8-27B (extra-high reasoning)Alibaba
5.6%
$0.50 / $3
97 DeepSeek-V4.1-Flash (no reasoning)DeepSeek
5.6%
$0.30 / $1.20
99 Qwen3.8-27B (medium reasoning)Alibaba
5.1%
$0.50 / $3
99 Claude Sonnet 5 (high reasoning)Anthropic
5.1%
$2 / $10
99 GPT-5.5 (medium reasoning)OpenAI
5.1%
$5 / $30
102 GPT-6 Luna (high reasoning)OpenAI
4.5%
$0.10 / $0.50
103 Gemini 3.1 Pro PreviewGoogle
4.0%
$2 / $12
104 GPT-5.6 Luna (extra-high reasoning)OpenAI
3.5%
$0.20 / $1.20
105 Claude Sonnet 4.6 (max reasoning)Anthropic
3.0%
$3 / $15
105 DeepSeek-V4-Flash (0423, high reasoning)DeepSeek
3.0%
$0.14 / $0.28
105 Beam 1.0 501B A23BReflection
3.0%
–
105 Grok 4.6 (low reasoning)xAI
3.0%
$2 / $6
109 Qwen3.8-27B (low reasoning)Alibaba
2.5%
$0.50 / $3
109 Claude Sonnet 5 (low reasoning)Anthropic
2.5%
$2 / $10
109 DeepSeek-V4-Flash (0423, max reasoning)DeepSeek
2.5%
$0.14 / $0.28
109 GPT-5.6 Luna (high reasoning)OpenAI
2.5%
$0.20 / $1.20
109 GPT-6 Luna (medium reasoning)OpenAI
2.5%
$0.10 / $0.50
114 Claude Sonnet 5 (medium reasoning)Anthropic
2.0%
$2 / $10
114 MiniMax-M3MiniMax
2.0%
$0.30 / $1.20
114 GPT-5.4 mini (extra-high reasoning)OpenAI
2.0%
$0.75 / $4.50
114 GLM-5.1Z.ai
2.0%
$1.38 / $4.40
118 Qwen3.7-MaxAlibaba
1.5%
$1.48 / $4.42
118 DeepSeek-V4-Pro (0813, no reasoning)DeepSeek
1.5%
$1.32 / $3.96
118 GPT-5.6 Terra (high reasoning)OpenAI
1.5%
$2 / $12
118 GPT-5.6 Terra (low reasoning)OpenAI
1.5%
$2 / $12
118 GPT-6 Luna (no reasoning)OpenAI
1.5%
$0.10 / $0.50
123 Qwen3.7-PlusAlibaba
1.0%
$0.32 / $1.28
123 Gemini 3.5 Flash-LiteGoogle
1.0%
$0.30 / $2.50
123 Kimi K2.7 CodeMoonshot AI
1.0%
$0.95 / $4
123 GPT-5.6 Luna (no reasoning)OpenAI
1.0%
$0.20 / $1.20
123 GPT-5.6 Sol (low reasoning)OpenAI
1.0%
$4 / $20
123 GPT-5.6 Terra (medium reasoning)OpenAI
1.0%
$2 / $12
123 Inkling SmallThinking Machines
1.0%
$0.45 / $1.20
123 Inkling (extra-high reasoning)Thinking Machines
1.0%
$0.95 / $4.05
123 GLM-5.2 (max reasoning)Z.ai
1.0%
$1.40 / $4.40
132 Qwen3.5-9BAlibaba
0.5%
$0.10 / $0.15
132 Trinity Large ThinkingArcee AI
0.5%
$0.25 / $0.80
132 Gemini 3.1 Flash-Lite PreviewGoogle
0.5%
$0.25 / $1.50
132 Muse Glimmer (high reasoning)Meta
0.5%
–
132 Kimi K2.6Moonshot AI
0.5%
$0.95 / $4
132 GPT-5.4 nano (extra-high reasoning)OpenAI
0.5%
$0.20 / $1.25
132 GPT-5.6 Luna (medium reasoning)OpenAI
0.5%
$0.20 / $1.20
132 GPT-5.6 Terra (no reasoning)OpenAI
0.5%
$2 / $12
132 Hy3Tencent
0.5%
$0.14 / $0.58
141 Qwen3-14B (reasoning on)Alibaba
0.0%
$0.12 / $0.24
141 Qwen3-235B-A22B-Thinking-2507Alibaba
0.0%
$0.30 / $3
141 Qwen3-30B-A3B-Thinking-2507Alibaba
0.0%
$0.20 / $2.40
141 Qwen3-32B (reasoning on)Alibaba
0.0%
$0.14 / $0.40
141 Qwen3.5-122B-A10BAlibaba
0.0%
$0.26 / $2.08
141 Qwen3.5-397B-A17BAlibaba
0.0%
$0.55 / $3.50
141 Qwen3.6-27BAlibaba
0.0%
$0.30 / $3.20
141 Qwen3.6-35B-A3BAlibaba
0.0%
$0.10 / $1
141 Qwen3.8-27B (no reasoning)Alibaba
0.0%
$0.50 / $3
141 Qwen3-8B (reasoning on)Alibaba
0.0%
$0.12 / $0.46
141 Qwen3-Coder-NextAlibaba
0.0%
$0.18 / $0.90
141 Claude Haiku 4.5 (reasoning on)Anthropic
0.0%
$1 / $5
141 Claude Sonnet 4.5 (reasoning on)Anthropic
0.0%
$3 / $15
141 DeepSeek-V3-0324DeepSeek
0.0%
$0.25 / $1
141 DeepSeek-V3DeepSeek
0.0%
$0.26 / $1.03
141 DeepSeek-R1DeepSeek
0.0%
$0.70 / $2.50
141 DeepSeek-V3.1-Terminus (reasoning on)DeepSeek
0.0%
$0.27 / $1
141 Gemini 2.5 ProGoogle
0.0%
$1.25 / $10
141 Gemma 3 12BGoogle
0.0%
$0.05 / $0.15
141 Gemma 3 27BGoogle
0.0%
$0.12 / $0.20
141 Gemma 4 31BGoogle
0.0%
$0.14 / $0.40
141 Gemma 4 E4BGoogle
0.0%
–
141 Granite 4.2 8BIBM
0.0%
$0.06 / $0.25
141 Mercury 2Inception
0.0%
$0.25 / $0.75
141 Llama 4 MaverickMeta
0.0%
$0.27 / $0.85
141 Llama 4 ScoutMeta
0.0%
$0.18 / $0.59
141 MiniMax-M2.7MiniMax
0.0%
$0.30 / $1.20
141 Devstral 2Mistral AI
0.0%
$0.40 / $2
141 Devstral Small 2Mistral AI
0.0%
–
141 Ministral 3 14BMistral AI
0.0%
$0.20 / $0.20
141 Ministral 3 3BMistral AI
0.0%
$0.10 / $0.10
141 Ministral 3 8BMistral AI
0.0%
$0.15 / $0.15
141 Mistral Large 3Mistral AI
0.0%
$0.50 / $1.50
141 Mistral Medium 3.1Mistral AI
0.0%
$0.40 / $2
141 Mistral Medium 3.5Mistral AI
0.0%
$1.50 / $7.50
141 Mistral Small 4Mistral AI
0.0%
$0.15 / $0.60
141 Mistral Small 3.1 24BMistral AI
0.0%
$0.35 / $0.56
141 Mistral Small 3.2 24BMistral AI
0.0%
$0.094 / $0.25
141 Nemotron 3.5 Super VLNVIDIA
0.0%
–
141 Nemotron 3 Nano 30B A3B (reasoning on)NVIDIA
0.0%
$0.05 / $0.20
141 Nemotron 3 Super 120B A12BNVIDIA
0.0%
$0.085 / $0.40
141 GPT-5.6 Luna (low reasoning)OpenAI
0.0%
$0.20 / $1.20
141 GPT-5 mini (high reasoning)OpenAI
0.0%
$0.25 / $2
141 GPT-6 Luna (low reasoning)OpenAI
0.0%
$0.10 / $0.50
141 gpt-oss-120b (high reasoning)OpenAI
0.0%
$0.15 / $0.60
141 gpt-oss-20b (high reasoning)OpenAI
0.0%
$0.03 / $0.15
141 o3-mini (high reasoning)OpenAI
0.0%
$1.10 / $4.40
141 Solar Pro 3Upstage
0.0%
$0.15 / $0.60
141 Solar Pro 4Upstage
0.0%
$0.09 / $0.36
141 Grok 4.3 (high reasoning)xAI
0.0%
$1.25 / $2.50
141 Grok 4.3 (no reasoning)xAI
0.0%
$1.25 / $2.50
141 MiMo-V2.5-Pro (reasoning on)Xiaomi
0.0%
$0.43 / $0.87
141 MiMo-V2.5Xiaomi
0.0%
$0.17 / $0.34

Swipe the table sideways for more columns.

Ranks follow the score as shown, so equal numbers share a rank. Results as published by Artificial Analysis; we do not re-run them.

Progress over time

Terminal-Bench 4.0, plotted at each model's release date, higher is better. Each model counts once, at its best setting. Scores are today's, not launch day's. Hover or tap a mark for its model.

Best score so far

Every model at its release date. The line moves each time a new release beats every earlier one, and each new best is named where there is room.

-20%0%20%40%60%80% 2025Jul2026Jul DeepSeek-V3Claude Sonnet 4.6Gemini 3.1 Pro PreviewGPT-5.5Claude Opus 4.8Claude Fable 5Claude Opus 5Claude Fable 5.1GPT-6 AstraClaude Sonnet 5.5

Best score by lab

Each lab's best so far, stepping whenever one of its releases beats its own earlier models. The 6 labs with the best scores today; 14 others not drawn.

AnthropicOpenAIGoogleMetaZ.aiAlibaba

-20%0%20%40%60%80% 2025Jul2026Jul AnthropicOpenAIGoogleMetaZ.aiAlibaba

What it measures

Share of terminal tasks (software, system and data jobs) an agent completes, as run by Artificial Analysis with its own harness and prompts.

What it does not measure

Not other harnesses or tools; one attempt per task.

Source

Artificial Analysis · Methodology

285 results from Artificial Analysis not ranked here · show why

We rank a result only when we can tie it to a specific model you can use. These are left out:

  • Not on sale through the API providers we track: 270
  • A different snapshot or variant from the model we list: 14
  • An unusual combination of settings: 1

Data sourced from Artificial Analysis. Licence: Artificial Analysis commercial data licence.