Benchmarks / Artificial Analysis

Reported by Artificial Analysis

Artificial Analysis

Share of scientific coding sub-problems solved, as run by Artificial Analysis with its own harness and prompts.

Last updated 20 Oct 2026

Results dated
20 Oct 2026
Results
197 configurations of 119 models
Unit
% of problems
Licence
Artificial Analysis commercial data licence

SciCode: Beam 1.0 501B A23B

Top 15 of 197 results · % of problems, higher is better. Choose a model to highlight it.Clear highlight

  1. 1 Claude Opus 5.5 (max reasoning)Anthropic 66.9%
  2. 2 Claude Opus 5.5 (extra-high reasoning)Anthropic 65.0%
  3. 3 Claude Fable 5.1 (max reasoning)Anthropic 63.1%
  4. 4 Gemini 4 Argon (high reasoning)Google 61.8%
  5. 5 Claude Fable 5 (max reasoning)Anthropic 61.0%
  6. 5 Claude Sonnet 5.5 (max reasoning)Anthropic 61.0%
  7. 7 Claude Fable 5.1 (extra-high reasoning)Anthropic 60.9%
  8. 7 MiMo-V2.6-ProXiaomi 60.9%
  9. 9 Claude Opus 5.5 (high reasoning)Anthropic 60.4%
  10. 10 Muse Wonder (max reasoning)Meta 60.3%
  11. 11 Gemini 3.7 Flash (medium reasoning)Google 59.8%
  12. 12 Muse Spark 1.3 (extra-high reasoning)Meta 59.7%
  13. 13 Kimi K3 (max reasoning)Moonshot AI 59.5%
  14. 14 Claude Opus 5.5 (medium reasoning)Anthropic 59.3%
  15. 15 GLM-5.3 (max reasoning)Z.ai 59.0%
  16. 120 Beam 1.0 501B A23BReflection 48.5%

Full results

Artificial Analysis: SciCode, % of problems, higher is better
#ModelSciCode
% of problems, higher is better		Price
$ per million tokens, in / out
1 Claude Opus 5.5 (max reasoning)Anthropic
66.9%
$4 / $20
2 Claude Opus 5.5 (extra-high reasoning)Anthropic
65.0%
$4 / $20
3 Claude Fable 5.1 (max reasoning)Anthropic
63.1%
$10 / $50
4 Gemini 4 Argon (high reasoning)Google
61.8%
$2 / $10
5 Claude Fable 5 (max reasoning)Anthropic
61.0%
$10 / $50
5 Claude Sonnet 5.5 (max reasoning)Anthropic
61.0%
$2 / $10
7 Claude Fable 5.1 (extra-high reasoning)Anthropic
60.9%
$10 / $50
7 MiMo-V2.6-ProXiaomi
60.9%
$0.43 / $0.87
9 Claude Opus 5.5 (high reasoning)Anthropic
60.4%
$4 / $20
10 Muse Wonder (max reasoning)Meta
60.3%
$4 / $20
11 Gemini 3.7 Flash (medium reasoning)Google
59.8%
$1.50 / $7.50
12 Muse Spark 1.3 (extra-high reasoning)Meta
59.7%
$1.25 / $4.25
13 Kimi K3 (max reasoning)Moonshot AI
59.5%
$3 / $15
14 Claude Opus 5.5 (medium reasoning)Anthropic
59.3%
$4 / $20
15 GLM-5.3 (max reasoning)Z.ai
59.0%
$1.40 / $4.40
16 Muse Spark 1.1 (extra-high reasoning)Meta
58.8%
$1.25 / $4.25
16 Muse Spark 1.3 (max reasoning)Meta
58.8%
$1.25 / $4.25
18 Claude Fable 5.1 (high reasoning)Anthropic
58.7%
$10 / $50
18 Gemini 3.1 Pro PreviewGoogle
58.7%
$2 / $12
20 Claude Opus 5.5 (low reasoning)Anthropic
58.6%
$4 / $20
21 Muse Spark Open (extra-high reasoning)Meta
58.4%
–
22 GPT-5.6 Sol (high reasoning)OpenAI
57.8%
$4 / $20
22 Grok 4.7 (high reasoning)xAI
57.8%
$2 / $6
24 GPT-6 Sol (max reasoning)OpenAI
57.6%
$2 / $10
25 Muse Spark 1.2 (extra-high reasoning)Meta
57.4%
$1.25 / $4.25
25 GPT-5.6 Sol (medium reasoning)OpenAI
57.4%
$4 / $20
25 Grok 4.7 (extra-high reasoning)xAI
57.4%
$2 / $6
28 Claude Sonnet 5.5 (extra-high reasoning)Anthropic
57.3%
$2 / $10
29 Gemini 3.7 Flash (high reasoning)Google
57.2%
$1.50 / $7.50
30 GPT-5.6 Sol (max reasoning)OpenAI
57.1%
$4 / $20
30 GPT-5.6 Sol (extra-high reasoning)OpenAI
57.1%
$4 / $20
32 Claude Fable 5.1 (low reasoning)Anthropic
56.7%
$10 / $50
33 Gemini 3.8 Flash (high reasoning)Google
56.6%
$1.50 / $7.50
34 GPT-6 Astra (max reasoning)OpenAI
56.5%
$10 / $50
34 Grok 4.6 (high reasoning)xAI
56.5%
$2 / $6
36 Claude Fable 5.1 (medium reasoning)Anthropic
56.4%
$10 / $50
36 Claude Opus 5 (max reasoning)Anthropic
56.4%
$5 / $25
36 GPT-5.6 Sol (low reasoning)OpenAI
56.4%
$4 / $20
39 GPT-5.5 (high reasoning)OpenAI
56.1%
$5 / $30
40 DeepSeek-V4.1-Flash-1017 (max reasoning)DeepSeek
55.9%
$0.30 / $1.20
40 Grok 4.6 (medium reasoning)xAI
55.9%
$2 / $6
42 GPT-5.5 (extra-high reasoning)OpenAI
55.8%
$5 / $30
42 GPT-6.1 Sol (high reasoning)OpenAI
55.8%
$2 / $10
44 Claude Opus 5 (extra-high reasoning)Anthropic
55.7%
$5 / $25
44 Gemini 3.7 Flash (low reasoning)Google
55.7%
$1.50 / $7.50
44 GPT-6.1 Sol (extra-high reasoning)OpenAI
55.7%
$2 / $10
44 GPT-6 Astra (extra-high reasoning)OpenAI
55.7%
$10 / $50
48 Claude Opus 5 (high reasoning)Anthropic
55.4%
$5 / $25
48 GPT-6 Astra (high reasoning)OpenAI
55.4%
$10 / $50
50 Gemini 3.8 Flash (medium reasoning)Google
55.1%
$1.50 / $7.50
50 GPT-6 Sol (extra-high reasoning)OpenAI
55.1%
$2 / $10
52 Claude Haiku 5.5 (max reasoning)Anthropic
55.0%
$0.10 / $0.50
52 Gemini 3.8 Flash (low reasoning)Google
55.0%
$1.50 / $7.50
52 GPT-5.6 Terra (max reasoning)OpenAI
55.0%
$2 / $12
52 Grok 4.5 (high reasoning)xAI
55.0%
$2 / $6
56 GPT-6 Sol (high reasoning)OpenAI
54.9%
$2 / $10
56 Grok 4.7 (low reasoning)xAI
54.9%
$2 / $6
58 GPT-6 Luna (max reasoning)OpenAI
54.6%
$0.10 / $0.50
59 GPT-5.5 (medium reasoning)OpenAI
54.5%
$5 / $30
60 Claude Opus 4.8 (max reasoning)Anthropic
54.4%
$5 / $25
61 Claude Sonnet 5 (high reasoning)Anthropic
54.3%
$2 / $10
61 Claude Sonnet 5 (max reasoning)Anthropic
54.3%
$2 / $10
63 Mistral Large 4Mistral AI
54.2%
$1.36 / $4.18
63 GPT-6.1 Sol (max reasoning)OpenAI
54.2%
$2 / $10
63 GPT-6 Astra (medium reasoning)OpenAI
54.2%
$10 / $50
66 Qwen3.8-2.4T-A95BAlibaba
54.1%
$2 / $6
66 Claude Sonnet 5 (extra-high reasoning)Anthropic
54.1%
$2 / $10
66 GPT-6 Astra (low reasoning)OpenAI
54.1%
$10 / $50
69 Gemini 3.5 Flash (high reasoning)Google
53.9%
$1.50 / $9
70 GPT-6 Sol (medium reasoning)OpenAI
53.8%
$2 / $10
71 Claude Sonnet 5.5 (high reasoning)Anthropic
53.7%
$2 / $10
72 GPT-5.6 Luna (max reasoning)OpenAI
53.6%
$0.20 / $1.20
73 Gemini 3.6 Flash (high reasoning)Google
53.4%
$1.50 / $7.50
73 MiniMax M3.1 FlashMiniMax
53.4%
$0.15 / $0.60
75 Qwen3.8-Max (0803)Alibaba
53.2%
–
75 GPT-6.1 Sol (low reasoning)OpenAI
53.2%
$2 / $10
75 GPT-6.1 Sol (medium reasoning)OpenAI
53.2%
$2 / $10
78 Grok 4.6 (extra-high reasoning)xAI
53.0%
$2 / $6
79 Claude Sonnet 5.5 (medium reasoning)Anthropic
52.9%
$2 / $10
80 Kimi K3 (low reasoning)Moonshot AI
52.7%
$3 / $15
81 GPT-5.5 Instant (2026-06-26)OpenAI
52.5%
–
82 GPT-5.6 Terra (high reasoning)OpenAI
52.4%
$2 / $12
83 GPT-5.6 Terra (extra-high reasoning)OpenAI
52.3%
$2 / $12
84 Qwen3.8-Max (0902)Alibaba
52.1%
$2 / $6
84 GPT-5.4 mini (extra-high reasoning)OpenAI
52.1%
$0.75 / $4.50
86 DeepSeek-V4.1-Flash (max reasoning)DeepSeek
51.9%
$0.30 / $1.20
87 Claude Haiku 5.5 (extra-high reasoning)Anthropic
51.7%
$0.10 / $0.50
87 GPT-6 Luna (extra-high reasoning)OpenAI
51.7%
$0.10 / $0.50
89 Claude Sonnet 5 (medium reasoning)Anthropic
51.6%
$2 / $10
89 GPT-5.6 Luna (high reasoning)OpenAI
51.6%
$0.20 / $1.20
89 GLM-5.3-FlashZ.ai
51.6%
$0.15 / $0.50
92 Claude Opus 5 (medium reasoning)Anthropic
51.5%
$5 / $25
92 Kimi K2.6Moonshot AI
51.5%
$0.95 / $4
94 MiMo-V2.6-FlashXiaomi
51.3%
$0.14 / $0.28
95 GLM-5.2 (max reasoning)Z.ai
51.2%
$1.40 / $4.40
96 DeepSeek-V4-Pro (0813, max reasoning)DeepSeek
51.0%
$1.32 / $3.96
97 GPT-6 Luna (medium reasoning)OpenAI
50.9%
$0.10 / $0.50
98 DeepSeek-V4-Pro (0423, max reasoning)DeepSeek
50.8%
$1.42 / $2.83
99 Qwen3.8-Flash-NextAlibaba
50.6%
–
99 MiMo-V2.5-Pro (reasoning on)Xiaomi
50.6%
$0.43 / $0.87
101 GPT-5.6 Luna (extra-high reasoning)OpenAI
50.5%
$0.20 / $1.20
101 GPT-5.6 Terra (medium reasoning)OpenAI
50.5%
$2 / $12
103 DeepSeek-V4-Flash (0731, max reasoning)DeepSeek
50.3%
$0.14 / $0.28
103 GPT-6 Luna (high reasoning)OpenAI
50.3%
$0.10 / $0.50
105 GPT-6 Sol (low reasoning)OpenAI
50.2%
$2 / $10
106 Claude Sonnet 4.6 (max reasoning)Anthropic
50.1%
$3 / $15
106 Claude Sonnet 5 (low reasoning)Anthropic
50.1%
$2 / $10
106 MiniMax-M2.7MiniMax
50.1%
$0.30 / $1.20
109 GPT-5.6 Terra (low reasoning)OpenAI
49.9%
$2 / $12
110 DeepSeek-V4-Flash-Vision-Exp (max reasoning)DeepSeek
49.7%
$0.44 / $1.32
110 Inkling SmallThinking Machines
49.7%
$0.45 / $1.20
112 Qwen3.7-MaxAlibaba
49.5%
$1.48 / $4.42
113 Grok 4.6 (low reasoning)xAI
49.4%
$2 / $6
114 Claude Haiku 5.5 (low reasoning)Anthropic
49.2%
$0.10 / $0.50
114 Claude Opus 5 (low reasoning)Anthropic
49.2%
$5 / $25
116 Claude Sonnet 5.5 (low reasoning)Anthropic
49.1%
$2 / $10
117 Claude Haiku 5.5 (medium reasoning)Anthropic
49.0%
$0.10 / $0.50
118 Claude Haiku 5.5 (high reasoning)Anthropic
48.7%
$0.10 / $0.50
119 Hy3Tencent
48.6%
$0.14 / $0.58
120 Beam 1.0 501B A23BReflection
48.5%
–
121 Grok 4.3 (high reasoning)xAI
48.3%
$1.25 / $2.50
122 Solar Pro 4Upstage
48.0%
$0.09 / $0.36
123 Kimi K2.7 CodeMoonshot AI
47.8%
$0.95 / $4
123 Nemotron 3.5 Super VLNVIDIA
47.8%
–
125 GPT-5.6 Sol (no reasoning)OpenAI
47.7%
$4 / $20
126 GPT-6 Sol (no reasoning)OpenAI
47.3%
$2 / $10
127 GPT-5.4 nano (extra-high reasoning)OpenAI
47.2%
$0.20 / $1.25
128 MiniMax-M3MiniMax
47.1%
$0.30 / $1.20
129 Inkling (extra-high reasoning)Thinking Machines
47.0%
$0.95 / $4.05
130 GPT-6 Luna (low reasoning)OpenAI
46.9%
$0.10 / $0.50
131 GPT-5.6 Luna (medium reasoning)OpenAI
46.8%
$0.20 / $1.20
132 Qwen3.8-27B (extra-high reasoning)Alibaba
46.6%
$0.50 / $3
133 Gemini 2.5 ProGoogle
46.3%
$1.25 / $10
134 Qwen3.7-PlusAlibaba
46.1%
$0.32 / $1.28
134 GPT-5.6 Luna (low reasoning)OpenAI
46.1%
$0.20 / $1.20
136 Claude Sonnet 4.5 (reasoning on)Anthropic
45.7%
$3 / $15
137 Gemma 4 31BGoogle
45.5%
$0.14 / $0.40
138 DeepSeek-V4-Flash (0423, max reasoning)DeepSeek
45.3%
$0.14 / $0.28
139 GPT-5.6 Terra (no reasoning)OpenAI
45.1%
$2 / $12
140 Muse Glimmer (high reasoning)Meta
44.9%
–
141 Qwen3.5-397B-A17BAlibaba
44.8%
$0.55 / $3.50
141 GLM-5.1Z.ai
44.8%
$1.38 / $4.40
143 MiMo-V2.5Xiaomi
43.9%
$0.17 / $0.34
144 Gemini 3.1 Flash-Lite PreviewGoogle
43.4%
$0.25 / $1.50
145 GPT-6 Luna (no reasoning)OpenAI
43.1%
$0.10 / $0.50
146 Qwen3.6-27BAlibaba
42.8%
$0.30 / $3.20
146 o3-mini (high reasoning)OpenAI
42.8%
$1.10 / $4.40
148 Claude Haiku 4.5 (reasoning on)Anthropic
42.2%
$1 / $5
149 GLM-5.3 (low reasoning)Z.ai
42.0%
$1.40 / $4.40
150 Qwen3-235B-A22B-Thinking-2507Alibaba
41.4%
$0.30 / $3
151 Gemini 3.5 Flash-LiteGoogle
41.3%
$0.30 / $2.50
152 Trinity Large ThinkingArcee AI
40.6%
$0.25 / $0.80
153 GPT-5.6 Luna (no reasoning)OpenAI
40.4%
$0.20 / $1.20
154 DeepSeek-V4-Flash (0423, high reasoning)DeepSeek
40.2%
$0.14 / $0.28
154 Mistral Medium 3.5Mistral AI
40.2%
$1.50 / $7.50
156 Qwen3.8-27B (low reasoning)Alibaba
40.0%
$0.50 / $3
156 Gemma 4 26B A4BGoogle
40.0%
$0.10 / $0.30
158 DeepSeek-V4-Pro (0813, no reasoning)DeepSeek
39.9%
$1.32 / $3.96
159 Qwen3.5-122B-A10BAlibaba
39.7%
$0.26 / $2.08
160 Grok 4.3 (no reasoning)xAI
39.4%
$1.25 / $2.50
161 Qwen3.8-27B (medium reasoning)Alibaba
39.0%
$0.50 / $3
161 DeepSeek-V3-0324DeepSeek
39.0%
$0.25 / $1
161 GPT-5 mini (high reasoning)OpenAI
39.0%
$0.25 / $2
164 gpt-oss-20b (high reasoning)OpenAI
38.9%
$0.03 / $0.15
165 Mistral Small 4Mistral AI
38.8%
$0.15 / $0.60
166 DeepSeek-R1DeepSeek
38.3%
$0.70 / $2.50
167 DeepSeek-V3.1-Terminus (reasoning on)DeepSeek
38.0%
$0.27 / $1
168 Mercury 2Inception
37.7%
$0.25 / $0.75
169 Qwen3.6-35B-A3BAlibaba
36.6%
$0.10 / $1
169 Mistral Large 3Mistral AI
36.6%
$0.50 / $1.50
171 Qwen3.8-27B (no reasoning)Alibaba
36.2%
$0.50 / $3
171 Qwen3-Coder-NextAlibaba
36.2%
$0.18 / $0.90
171 Nemotron 3 Super 120B A12BNVIDIA
36.2%
$0.085 / $0.40
174 Qwen3-32B (reasoning on)Alibaba
36.0%
$0.14 / $0.40
175 DeepSeek-V3DeepSeek
35.8%
$0.26 / $1.03
176 DeepSeek-V4.1-Flash (no reasoning)DeepSeek
35.5%
$0.30 / $1.20
177 gpt-oss-120b (high reasoning)OpenAI
34.0%
$0.15 / $0.60
178 Qwen3-30B-A3B-Thinking-2507Alibaba
33.0%
$0.20 / $2.40
179 Devstral 2Mistral AI
32.8%
$0.40 / $2
180 Devstral Small 2Mistral AI
32.4%
–
180 Mistral Medium 3.1Mistral AI
32.4%
$0.40 / $2
182 Llama 4 MaverickMeta
31.7%
$0.27 / $0.85
183 Granite 4.2 8BIBM
31.5%
$0.06 / $0.25
184 Qwen3-14B (reasoning on)Alibaba
30.7%
$0.12 / $0.24
185 Nemotron 3 Nano 30B A3B (reasoning on)NVIDIA
30.6%
$0.05 / $0.20
186 Qwen3.5-9BAlibaba
29.5%
$0.10 / $0.15
187 Mistral Small 3.2 24BMistral AI
28.6%
$0.094 / $0.25
188 Qwen3-8B (reasoning on)Alibaba
27.9%
$0.12 / $0.46
189 Mistral Small 3.1 24BMistral AI
27.8%
$0.35 / $0.56
190 Solar Pro 3Upstage
25.5%
$0.15 / $0.60
191 Gemma 4 E4BGoogle
24.4%
–
192 Ministral 3 14BMistral AI
23.8%
$0.20 / $0.20
193 Gemma 3 27BGoogle
23.3%
$0.12 / $0.20
194 Llama 4 ScoutMeta
21.3%
$0.18 / $0.59
195 Ministral 3 8BMistral AI
20.7%
$0.15 / $0.15
196 Gemma 3 12BGoogle
16.4%
$0.05 / $0.15
197 Ministral 3 3BMistral AI
15.3%
$0.10 / $0.10

Swipe the table sideways for more columns.

Ranks follow the score as shown, so equal numbers share a rank. Results as published by Artificial Analysis; we do not re-run them.

Progress over time

SciCode, plotted at each model's release date, higher is better. Each model counts once, at its best setting. Scores are today's, not launch day's. Hover or tap a mark for its model.

Best score so far

Every model at its release date. The line moves each time a new release beats every earlier one, and each new best is named where there is room.

0%20%40%60%80% 2025Jul2026Jul DeepSeek-V3DeepSeek-R1o3-miniGemini 2.5 ProClaude Sonnet 4.6Gemini 3.1 Pro PreviewClaude Fable 5Claude Fable 5.1Claude Opus 5.5

Best score by lab

Each lab's best so far, stepping whenever one of its releases beats its own earlier models. The 6 labs with the best scores today; 14 others not drawn.

AnthropicGoogleXiaomiMetaMoonshot AIZ.ai

20%30%40%50%60%70% 2025Jul2026Jul AnthropicGoogleXiaomiMetaMoonshot AIZ.ai

What it measures

Share of scientific coding sub-problems solved, as run by Artificial Analysis with its own harness and prompts.

What it does not measure

Not general software engineering.

Source

Artificial Analysis · Methodology

285 results from Artificial Analysis not ranked here · show why

We rank a result only when we can tie it to a specific model you can use. These are left out:

  • Not on sale through the API providers we track: 270
  • A different snapshot or variant from the model we list: 14
  • An unusual combination of settings: 1

Data sourced from Artificial Analysis. Licence: Artificial Analysis commercial data licence.