Benchmarks / Artificial Analysis

Reported by Artificial Analysis

Artificial Analysis

Share of scientific coding sub-problems solved, as run by Artificial Analysis with its own harness and prompts.

Last updated 20 Oct 2026

Results dated
20 Oct 2026
Results
197 configurations of 119 models
Unit
% of problems
Licence
Artificial Analysis commercial data licence

SciCode: Muse Spark Open

Top 15 of 119 results · % of problems, higher is better. Choose a model to highlight it.Clear highlight

  1. 1 Claude Opus 5.5 (max reasoning)Anthropic 66.9%
  2. 2 Claude Fable 5.1 (max reasoning)Anthropic 63.1%
  3. 3 Gemini 4 Argon (high reasoning)Google 61.8%
  4. 4 Claude Fable 5 (max reasoning)Anthropic 61.0%
  5. 4 Claude Sonnet 5.5 (max reasoning)Anthropic 61.0%
  6. 6 MiMo-V2.6-ProXiaomi 60.9%
  7. 7 Muse Wonder (max reasoning)Meta 60.3%
  8. 8 Gemini 3.7 Flash (medium reasoning)Google 59.8%
  9. 9 Muse Spark 1.3 (extra-high reasoning)Meta 59.7%
  10. 10 Kimi K3 (max reasoning)Moonshot AI 59.5%
  11. 11 GLM-5.3 (max reasoning)Z.ai 59.0%
  12. 12 Muse Spark 1.1 (extra-high reasoning)Meta 58.8%
  13. 13 Gemini 3.1 Pro PreviewGoogle 58.7%
  14. 14 Muse Spark Open (extra-high reasoning)Meta 58.4%
  15. 15 GPT-5.6 Sol (high reasoning)OpenAI 57.8%

Full results

Artificial Analysis: SciCode, % of problems, higher is better
#ModelSciCode
% of problems, higher is better		Price
$ per million tokens, in / out
1 Claude Opus 5.5 (max reasoning)Anthropic · best of 5 settings
66.9%
$4 / $20
2 Claude Fable 5.1 (max reasoning)Anthropic · best of 5 settings
63.1%
$10 / $50
3 Gemini 4 Argon (high reasoning)Google
61.8%
$2 / $10
4 Claude Fable 5 (max reasoning)Anthropic
61.0%
$10 / $50
4 Claude Sonnet 5.5 (max reasoning)Anthropic · best of 5 settings
61.0%
$2 / $10
6 MiMo-V2.6-ProXiaomi
60.9%
$0.43 / $0.87
7 Muse Wonder (max reasoning)Meta
60.3%
$4 / $20
8 Gemini 3.7 Flash (medium reasoning)Google · best of 3 settings
59.8%
$1.50 / $7.50
9 Muse Spark 1.3 (extra-high reasoning)Meta · best of 2 settings
59.7%
$1.25 / $4.25
10 Kimi K3 (max reasoning)Moonshot AI · best of 2 settings
59.5%
$3 / $15
11 GLM-5.3 (max reasoning)Z.ai · best of 2 settings
59.0%
$1.40 / $4.40
12 Muse Spark 1.1 (extra-high reasoning)Meta
58.8%
$1.25 / $4.25
13 Gemini 3.1 Pro PreviewGoogle
58.7%
$2 / $12
14 Muse Spark Open (extra-high reasoning)Meta
58.4%
–
15 GPT-5.6 Sol (high reasoning)OpenAI · best of 6 settings
57.8%
$4 / $20
15 Grok 4.7 (high reasoning)xAI · best of 3 settings
57.8%
$2 / $6
17 GPT-6 Sol (max reasoning)OpenAI · best of 6 settings
57.6%
$2 / $10
18 Muse Spark 1.2 (extra-high reasoning)Meta
57.4%
$1.25 / $4.25
19 Gemini 3.8 Flash (high reasoning)Google · best of 3 settings
56.6%
$1.50 / $7.50
20 GPT-6 Astra (max reasoning)OpenAI · best of 5 settings
56.5%
$10 / $50
20 Grok 4.6 (high reasoning)xAI · best of 4 settings
56.5%
$2 / $6
22 Claude Opus 5 (max reasoning)Anthropic · best of 5 settings
56.4%
$5 / $25
23 GPT-5.5 (high reasoning)OpenAI · best of 3 settings
56.1%
$5 / $30
24 DeepSeek-V4.1-Flash-1017 (max reasoning)DeepSeek
55.9%
$0.30 / $1.20
25 GPT-6.1 Sol (high reasoning)OpenAI · best of 5 settings
55.8%
$2 / $10
26 Claude Haiku 5.5 (max reasoning)Anthropic · best of 5 settings
55.0%
$0.10 / $0.50
26 GPT-5.6 Terra (max reasoning)OpenAI · best of 6 settings
55.0%
$2 / $12
26 Grok 4.5 (high reasoning)xAI
55.0%
$2 / $6
29 GPT-6 Luna (max reasoning)OpenAI · best of 6 settings
54.6%
$0.10 / $0.50
30 Claude Opus 4.8 (max reasoning)Anthropic
54.4%
$5 / $25
31 Claude Sonnet 5 (high reasoning)Anthropic · best of 5 settings
54.3%
$2 / $10
32 Mistral Large 4Mistral AI
54.2%
$1.36 / $4.18
33 Qwen3.8-2.4T-A95BAlibaba
54.1%
$2 / $6
34 Gemini 3.5 Flash (high reasoning)Google
53.9%
$1.50 / $9
35 GPT-5.6 Luna (max reasoning)OpenAI · best of 6 settings
53.6%
$0.20 / $1.20
36 Gemini 3.6 Flash (high reasoning)Google
53.4%
$1.50 / $7.50
36 MiniMax M3.1 FlashMiniMax
53.4%
$0.15 / $0.60
38 Qwen3.8-Max (0803)Alibaba
53.2%
–
39 GPT-5.5 Instant (2026-06-26)OpenAI
52.5%
–
40 Qwen3.8-Max (0902)Alibaba
52.1%
$2 / $6
40 GPT-5.4 mini (extra-high reasoning)OpenAI
52.1%
$0.75 / $4.50
42 DeepSeek-V4.1-Flash (max reasoning)DeepSeek · best of 2 settings
51.9%
$0.30 / $1.20
43 GLM-5.3-FlashZ.ai
51.6%
$0.15 / $0.50
44 Kimi K2.6Moonshot AI
51.5%
$0.95 / $4
45 MiMo-V2.6-FlashXiaomi
51.3%
$0.14 / $0.28
46 GLM-5.2 (max reasoning)Z.ai
51.2%
$1.40 / $4.40
47 DeepSeek-V4-Pro (0813, max reasoning)DeepSeek · best of 2 settings
51.0%
$1.32 / $3.96
48 DeepSeek-V4-Pro (0423, max reasoning)DeepSeek
50.8%
$1.42 / $2.83
49 Qwen3.8-Flash-NextAlibaba
50.6%
–
49 MiMo-V2.5-Pro (reasoning on)Xiaomi
50.6%
$0.43 / $0.87
51 DeepSeek-V4-Flash (0731, max reasoning)DeepSeek
50.3%
$0.14 / $0.28
52 Claude Sonnet 4.6 (max reasoning)Anthropic
50.1%
$3 / $15
52 MiniMax-M2.7MiniMax
50.1%
$0.30 / $1.20
54 DeepSeek-V4-Flash-Vision-Exp (max reasoning)DeepSeek
49.7%
$0.44 / $1.32
54 Inkling SmallThinking Machines
49.7%
$0.45 / $1.20
56 Qwen3.7-MaxAlibaba
49.5%
$1.48 / $4.42
57 Hy3Tencent
48.6%
$0.14 / $0.58
58 Beam 1.0 501B A23BReflection
48.5%
–
59 Grok 4.3 (high reasoning)xAI · best of 2 settings
48.3%
$1.25 / $2.50
60 Solar Pro 4Upstage
48.0%
$0.09 / $0.36
61 Kimi K2.7 CodeMoonshot AI
47.8%
$0.95 / $4
61 Nemotron 3.5 Super VLNVIDIA
47.8%
–
63 GPT-5.4 nano (extra-high reasoning)OpenAI
47.2%
$0.20 / $1.25
64 MiniMax-M3MiniMax
47.1%
$0.30 / $1.20
65 Inkling (extra-high reasoning)Thinking Machines
47.0%
$0.95 / $4.05
66 Qwen3.8-27B (extra-high reasoning)Alibaba · best of 4 settings
46.6%
$0.50 / $3
67 Gemini 2.5 ProGoogle
46.3%
$1.25 / $10
68 Qwen3.7-PlusAlibaba
46.1%
$0.32 / $1.28
69 Claude Sonnet 4.5 (reasoning on)Anthropic
45.7%
$3 / $15
70 Gemma 4 31BGoogle
45.5%
$0.14 / $0.40
71 DeepSeek-V4-Flash (0423, max reasoning)DeepSeek · best of 2 settings
45.3%
$0.14 / $0.28
72 Muse Glimmer (high reasoning)Meta
44.9%
–
73 Qwen3.5-397B-A17BAlibaba
44.8%
$0.55 / $3.50
73 GLM-5.1Z.ai
44.8%
$1.38 / $4.40
75 MiMo-V2.5Xiaomi
43.9%
$0.17 / $0.34
76 Gemini 3.1 Flash-Lite PreviewGoogle
43.4%
$0.25 / $1.50
77 Qwen3.6-27BAlibaba
42.8%
$0.30 / $3.20
77 o3-mini (high reasoning)OpenAI
42.8%
$1.10 / $4.40
79 Claude Haiku 4.5 (reasoning on)Anthropic
42.2%
$1 / $5
80 Qwen3-235B-A22B-Thinking-2507Alibaba
41.4%
$0.30 / $3
81 Gemini 3.5 Flash-LiteGoogle
41.3%
$0.30 / $2.50
82 Trinity Large ThinkingArcee AI
40.6%
$0.25 / $0.80
83 Mistral Medium 3.5Mistral AI
40.2%
$1.50 / $7.50
84 Gemma 4 26B A4BGoogle
40.0%
$0.10 / $0.30
85 Qwen3.5-122B-A10BAlibaba
39.7%
$0.26 / $2.08
86 DeepSeek-V3-0324DeepSeek
39.0%
$0.25 / $1
86 GPT-5 mini (high reasoning)OpenAI
39.0%
$0.25 / $2
88 gpt-oss-20b (high reasoning)OpenAI
38.9%
$0.03 / $0.15
89 Mistral Small 4Mistral AI
38.8%
$0.15 / $0.60
90 DeepSeek-R1DeepSeek
38.3%
$0.70 / $2.50
91 DeepSeek-V3.1-Terminus (reasoning on)DeepSeek
38.0%
$0.27 / $1
92 Mercury 2Inception
37.7%
$0.25 / $0.75
93 Qwen3.6-35B-A3BAlibaba
36.6%
$0.10 / $1
93 Mistral Large 3Mistral AI
36.6%
$0.50 / $1.50
95 Qwen3-Coder-NextAlibaba
36.2%
$0.18 / $0.90
95 Nemotron 3 Super 120B A12BNVIDIA
36.2%
$0.085 / $0.40
97 Qwen3-32B (reasoning on)Alibaba
36.0%
$0.14 / $0.40
98 DeepSeek-V3DeepSeek
35.8%
$0.26 / $1.03
99 gpt-oss-120b (high reasoning)OpenAI
34.0%
$0.15 / $0.60
100 Qwen3-30B-A3B-Thinking-2507Alibaba
33.0%
$0.20 / $2.40
101 Devstral 2Mistral AI
32.8%
$0.40 / $2
102 Devstral Small 2Mistral AI
32.4%
–
102 Mistral Medium 3.1Mistral AI
32.4%
$0.40 / $2
104 Llama 4 MaverickMeta
31.7%
$0.27 / $0.85
105 Granite 4.2 8BIBM
31.5%
$0.06 / $0.25
106 Qwen3-14B (reasoning on)Alibaba
30.7%
$0.12 / $0.24
107 Nemotron 3 Nano 30B A3B (reasoning on)NVIDIA
30.6%
$0.05 / $0.20
108 Qwen3.5-9BAlibaba
29.5%
$0.10 / $0.15
109 Mistral Small 3.2 24BMistral AI
28.6%
$0.094 / $0.25
110 Qwen3-8B (reasoning on)Alibaba
27.9%
$0.12 / $0.46
111 Mistral Small 3.1 24BMistral AI
27.8%
$0.35 / $0.56
112 Solar Pro 3Upstage
25.5%
$0.15 / $0.60
113 Gemma 4 E4BGoogle
24.4%
–
114 Ministral 3 14BMistral AI
23.8%
$0.20 / $0.20
115 Gemma 3 27BGoogle
23.3%
$0.12 / $0.20
116 Llama 4 ScoutMeta
21.3%
$0.18 / $0.59
117 Ministral 3 8BMistral AI
20.7%
$0.15 / $0.15
118 Gemma 3 12BGoogle
16.4%
$0.05 / $0.15
119 Ministral 3 3BMistral AI
15.3%
$0.10 / $0.10

Swipe the table sideways for more columns.

Ranks follow the score as shown, so equal numbers share a rank. Each model is shown at its best setting; show every setting. Results as published by Artificial Analysis; we do not re-run them.

Progress over time

SciCode, plotted at each model's release date, higher is better. Each model counts once, at its best setting. Scores are today's, not launch day's. Hover or tap a mark for its model.

Best score so far

Every model at its release date. The line moves each time a new release beats every earlier one, and each new best is named where there is room.

0%20%40%60%80% 2025Jul2026Jul DeepSeek-V3DeepSeek-R1o3-miniGemini 2.5 ProClaude Sonnet 4.6Gemini 3.1 Pro PreviewClaude Fable 5Claude Fable 5.1Claude Opus 5.5

Best score by lab

Each lab's best so far, stepping whenever one of its releases beats its own earlier models. The 6 labs with the best scores today; 14 others not drawn.

AnthropicGoogleXiaomiMetaMoonshot AIZ.ai

20%30%40%50%60%70% 2025Jul2026Jul AnthropicGoogleXiaomiMetaMoonshot AIZ.ai

What it measures

Share of scientific coding sub-problems solved, as run by Artificial Analysis with its own harness and prompts.

What it does not measure

Not general software engineering.

Source

Artificial Analysis · Methodology

285 results from Artificial Analysis not ranked here · show why

We rank a result only when we can tie it to a specific model you can use. These are left out:

  • Not on sale through the API providers we track: 270
  • A different snapshot or variant from the model we list: 14
  • An unusual combination of settings: 1

Data sourced from Artificial Analysis. Licence: Artificial Analysis commercial data licence.