Benchmarks / Artificial Analysis

Reported by Artificial Analysis

Artificial Analysis

Share of scientific coding sub-problems solved, as run by Artificial Analysis with its own harness and prompts.

Last updated 9 Nov 2026

Results dated
20 Oct 2026 to 9 Nov 2026
Results
205 configurations of 121 models
Unit
% of problems
Licence
Artificial Analysis commercial data licence

Artificial Analysis in brief

What it measures. Artificial Analysis is a third-party benchmark, shown here with credit. Measured here: Share of scientific coding sub-problems solved, as run by Artificial Analysis with its own harness and prompts.

Latest results. , 121 models ranked.

Top 3 on SciCode (% of problems, higher is better, each model at its best setting):

  1. Claude Opus 5.5 (max reasoning) (Anthropic): 66.9%
  2. Claude Fable 5.1 (max reasoning) (Anthropic): 63.1%
  3. Gemini 4 Argon (high reasoning) (Google): 61.8%

How to cite. Artificial Analysis, via Spring Prompt, results dated 9 November 2026. https://springprompt.com/benchmarks/artificial-analysis These results belong to Artificial Analysis, not to us. Licence: Artificial Analysis commercial data licence. Credit Artificial Analysis first.

SciCode: GPT-6 Astra

Top 15 of 121 results · % of problems, higher is better. Choose a model to highlight it.Clear highlight

  1. 1 Claude Opus 5.5 (max reasoning)Anthropic 66.9%
  2. 2 Claude Fable 5.1 (max reasoning)Anthropic 63.1%
  3. 3 Gemini 4 Argon (high reasoning)Google 61.8%
  4. 4 Claude Fable 5 (max reasoning)Anthropic 61.0%
  5. 4 Claude Sonnet 5.5 (max reasoning)Anthropic 61.0%
  6. 6 MiMo-V2.6-ProXiaomi 60.9%
  7. 7 Kimi K3.1 ProMoonshot AI 60.8%
  8. 8 Muse Wonder (max reasoning)Meta 60.3%
  9. 9 Gemini 3.7 Flash (medium reasoning)Google 59.8%
  10. 10 Muse Spark 1.3 (extra-high reasoning)Meta 59.7%
  11. 11 Kimi K3 (max reasoning)Moonshot AI 59.5%
  12. 12 GLM-5.3 (max reasoning)Z.ai 59.0%
  13. 13 Muse Spark 1.1 (extra-high reasoning)Meta 58.8%
  14. 14 Gemini 3.1 Pro PreviewGoogle 58.7%
  15. 15 Muse Spark Open (extra-high reasoning)Meta 58.4%
  16. 21 GPT-6 Astra (max reasoning)OpenAI 56.5%

Full results

Artificial Analysis: SciCode, % of problems, higher is better
#ModelSciCode
% of problems, higher is better
Price
$ per million tokens, in / out
1 Claude Opus 5.5 (max reasoning)Anthropic · best of 5 settings
66.9%
$4 / $20
2 Claude Fable 5.1 (max reasoning)Anthropic · best of 5 settings
63.1%
$10 / $50
3 Gemini 4 Argon (high reasoning)Google
61.8%
$2 / $10
4 Claude Fable 5 (max reasoning)Anthropic
61.0%
$10 / $50
4 Claude Sonnet 5.5 (max reasoning)Anthropic · best of 5 settings
61.0%
$2 / $10
6 MiMo-V2.6-ProXiaomi
60.9%
$0.43 / $0.87
7 Kimi K3.1 ProMoonshot AI
60.8%
$1.75 / $10
8 Muse Wonder (max reasoning)Meta
60.3%
$4 / $20
9 Gemini 3.7 Flash (medium reasoning)Google · best of 3 settings
59.8%
$1.50 / $7.50
10 Muse Spark 1.3 (extra-high reasoning)Meta · best of 2 settings
59.7%
$1.25 / $4.25
11 Kimi K3 (max reasoning)Moonshot AI · best of 2 settings
59.5%
$3 / $15
12 GLM-5.3 (max reasoning)Z.ai · best of 2 settings
59.0%
$1.40 / $4.40
13 Muse Spark 1.1 (extra-high reasoning)Meta
58.8%
$1.25 / $4.25
14 Gemini 3.1 Pro PreviewGoogle
58.7%
$2 / $12
15 Muse Spark Open (extra-high reasoning)Meta
58.4%
$1.25 / $4.25
16 GPT-5.6 Sol (high reasoning)OpenAI · best of 6 settings
57.8%
$4 / $20
16 Grok 4.7 (high reasoning)xAI · best of 3 settings
57.8%
$2 / $6
18 GPT-6 Sol (max reasoning)OpenAI · best of 6 settings
57.6%
$2 / $10
19 Muse Spark 1.2 (extra-high reasoning)Meta
57.4%
$1.25 / $4.25
20 Gemini 3.8 Flash (high reasoning)Google · best of 3 settings
56.6%
$1.50 / $7.50
21 GPT-6 Astra (max reasoning)OpenAI · best of 5 settings
56.5%
$10 / $50
21 Grok 4.6 (high reasoning)xAI · best of 4 settings
56.5%
$2 / $6
23 Claude Opus 5 (max reasoning)Anthropic · best of 5 settings
56.4%
$5 / $25
24 GPT-5.5 (high reasoning)OpenAI · best of 3 settings
56.1%
$5 / $30
25 DeepSeek-V4.1-Flash-1017 (max reasoning)DeepSeek · best of 2 settings
55.9%
$0.30 / $1.20
26 GPT-6.1 Sol (high reasoning)OpenAI · best of 5 settings
55.8%
$2 / $10
27 GPT-6.1 Terra (max reasoning)OpenAI · best of 6 settings
55.1%
$1 / $5
28 Claude Haiku 5.5 (max reasoning)Anthropic · best of 5 settings
55.0%
$0.10 / $0.50
28 GPT-5.6 Terra (max reasoning)OpenAI · best of 6 settings
55.0%
$2 / $12
28 Grok 4.5 (high reasoning)xAI
55.0%
$2 / $6
31 GPT-6 Luna (max reasoning)OpenAI · best of 6 settings
54.6%
$0.10 / $0.50
32 Claude Opus 4.8 (max reasoning)Anthropic
54.4%
$5 / $25
33 Claude Sonnet 5 (high reasoning)Anthropic · best of 5 settings
54.3%
$2 / $10
34 Mistral Large 4Mistral AI
54.2%
$1.36 / $4.18
35 Qwen3.8-2.4T-A95BAlibaba
54.1%
$2 / $6
36 Gemini 3.5 Flash (high reasoning)Google
53.9%
$1.50 / $9
37 GPT-5.6 Luna (max reasoning)OpenAI · best of 6 settings
53.6%
$0.20 / $1.20
38 Gemini 3.6 Flash (high reasoning)Google
53.4%
$1.50 / $7.50
38 MiniMax M3.1 FlashMiniMax
53.4%
$0.15 / $0.60
40 Qwen3.8-Max (0803)Alibaba
53.2%
–
41 GPT-5.5 Instant (2026-06-26)OpenAI
52.5%
–
42 Qwen3.8-Max (0902)Alibaba
52.1%
$2 / $6
42 GPT-5.4 mini (extra-high reasoning)OpenAI
52.1%
$0.75 / $4.50
44 DeepSeek-V4.1-Flash (max reasoning)DeepSeek · best of 2 settings
51.9%
$0.30 / $1.20
45 GLM-5.3-FlashZ.ai
51.6%
$0.15 / $0.50
46 Kimi K2.6Moonshot AI
51.5%
$0.95 / $4
47 MiMo-V2.6-FlashXiaomi
51.3%
$0.14 / $0.28
48 GLM-5.2 (max reasoning)Z.ai
51.2%
$1.40 / $4.40
49 DeepSeek-V4-Pro (0813, max reasoning)DeepSeek · best of 2 settings
51.0%
$1.32 / $3.96
50 DeepSeek-V4-Pro (0423, max reasoning)DeepSeek
50.8%
$1.42 / $2.83
51 Qwen3.8-Flash-NextAlibaba
50.6%
–
51 MiMo-V2.5-Pro (reasoning on)Xiaomi
50.6%
$0.43 / $0.87
53 DeepSeek-V4-Flash (0731, max reasoning)DeepSeek
50.3%
$0.13 / $0.40
54 Claude Sonnet 4.6 (max reasoning)Anthropic
50.1%
$3 / $15
54 MiniMax-M2.7MiniMax
50.1%
$0.30 / $1.20
56 DeepSeek-V4-Flash-Vision-Exp (max reasoning)DeepSeek
49.7%
$0.22 / $0.65
56 Inkling SmallThinking Machines
49.7%
$0.45 / $1.20
58 Qwen3.7-MaxAlibaba
49.5%
$1.48 / $4.42
59 Hy3Tencent
48.6%
$0.14 / $0.58
60 Beam 1.0 501B A23BReflection
48.5%
$0.40 / $1.20
61 Grok 4.3 (high reasoning)xAI · best of 2 settings
48.3%
$1.25 / $2.50
62 Solar Pro 4Upstage
48.0%
$0.15 / $0.60
63 Kimi K2.7 CodeMoonshot AI
47.8%
$0.95 / $4
63 Nemotron 3.5 Super VLNVIDIA
47.8%
$0.15 / $0.50
65 GPT-5.4 nano (extra-high reasoning)OpenAI
47.2%
$0.20 / $1.25
66 MiniMax-M3MiniMax
47.1%
$0.30 / $1.20
67 Inkling (extra-high reasoning)Thinking Machines
47.0%
$1 / $4.05
68 Qwen3.8-27B (extra-high reasoning)Alibaba · best of 4 settings
46.6%
$0.50 / $3
69 Gemini 2.5 ProGoogle
46.3%
$1.25 / $10
70 Qwen3.7-PlusAlibaba
46.1%
$0.32 / $1.28
71 Claude Sonnet 4.5 (reasoning on)Anthropic
45.7%
$3 / $15
72 Gemma 4 31BGoogle
45.5%
$0.15 / $0.40
73 DeepSeek-V4-Flash (0423, max reasoning)DeepSeek · best of 2 settings
45.3%
$0.05 / $0.45
74 Muse Glimmer (high reasoning)Meta
44.9%
–
75 Qwen3.5-397B-A17BAlibaba
44.8%
$0.55 / $3.50
75 GLM-5.1Z.ai
44.8%
$1.33 / $4.18
77 MiMo-V2.5Xiaomi
43.9%
$0.40 / $2
78 Gemini 3.1 Flash-Lite PreviewGoogle
43.4%
$0.25 / $1.50
79 Qwen3.6-27BAlibaba
42.8%
$0.30 / $3.20
79 o3-mini (high reasoning)OpenAI
42.8%
–
81 Claude Haiku 4.5 (reasoning on)Anthropic
42.2%
$1 / $5
82 Qwen3-235B-A22B-Thinking-2507Alibaba
41.4%
$0.45 / $3.50
83 Gemini 3.5 Flash-LiteGoogle
41.3%
$0.30 / $2.50
84 Trinity Large ThinkingArcee AI
40.6%
–
85 Mistral Medium 3.5Mistral AI
40.2%
$1.50 / $7.50
86 Gemma 4 26B A4BGoogle
40.0%
$0.09 / $0.30
87 Qwen3.5-122B-A10BAlibaba
39.7%
$0.26 / $2.08
88 DeepSeek-V3-0324DeepSeek
39.0%
$0.25 / $1
88 GPT-5 mini (high reasoning)OpenAI
39.0%
$0.25 / $2
90 gpt-oss-20b (high reasoning)OpenAI
38.9%
$0.03 / $0.14
91 Mistral Small 4Mistral AI
38.8%
$0.15 / $0.60
92 DeepSeek-R1DeepSeek
38.3%
–
93 DeepSeek-V3.1-Terminus (reasoning on)DeepSeek
38.0%
$0.27 / $1
94 Mercury 2Inception
37.7%
–
95 Qwen3.6-35B-A3BAlibaba
36.6%
$0.10 / $0.95
95 Mistral Large 3Mistral AI
36.6%
$0.50 / $1.50
97 Qwen3-Coder-NextAlibaba
36.2%
$0.12 / $0.80
97 Nemotron 3 Super 120B A12BNVIDIA
36.2%
$0.08 / $0.45
99 Qwen3-32B (reasoning on)Alibaba
36.0%
$0.14 / $0.40
100 DeepSeek-V3DeepSeek
35.8%
$0.26 / $1.03
101 gpt-oss-120b (high reasoning)OpenAI
34.0%
$0.10 / $0.50
102 Qwen3-30B-A3B-Thinking-2507Alibaba
33.0%
–
103 Devstral 2Mistral AI
32.8%
$0.40 / $2
104 Devstral Small 2Mistral AI
32.4%
–
104 Mistral Medium 3.1Mistral AI
32.4%
$0.40 / $2
106 Llama 4 MaverickMeta
31.7%
$0.35 / $1
107 Granite 4.2 8BIBM
31.5%
$0.06 / $0.15
108 Qwen3-14B (reasoning on)Alibaba
30.7%
$0.10 / $0.22
109 Nemotron 3 Nano 30B A3B (reasoning on)NVIDIA
30.6%
$0.06 / $0.24
110 Qwen3.5-9BAlibaba
29.5%
$0.10 / $0.15
111 Mistral Small 3.2 24BMistral AI
28.6%
$0.094 / $0.25
112 Qwen3-8B (reasoning on)Alibaba
27.9%
–
113 Mistral Small 3.1 24BMistral AI
27.8%
$0.35 / $0.56
114 Solar Pro 3Upstage
25.5%
–
115 Gemma 4 E4BGoogle
24.4%
–
116 Ministral 3 14BMistral AI
23.8%
$0.20 / $0.20
117 Gemma 3 27BGoogle
23.3%
$0.08 / $0.35
118 Llama 4 ScoutMeta
21.3%
$0.10 / $0.30
119 Ministral 3 8BMistral AI
20.7%
$0.15 / $0.15
120 Gemma 3 12BGoogle
16.4%
$0.05 / $0.15
121 Ministral 3 3BMistral AI
15.3%
$0.10 / $0.10

Swipe the table sideways for more columns.

Source: Artificial Analysis, via Spring Prompt, springprompt.com/benchmarks/artificial-analysis. Test your own prompt on these models →

Ranks follow the score as shown, so equal numbers share a rank. Each model is shown at its best setting; show every setting. Results as published by Artificial Analysis; we do not re-run them.

Progress over time

SciCode, plotted at each model's release date, higher is better. Each model counts once, at its best setting. Scores are today's, not launch day's. Hover or tap a mark for its model.

Best score so far

Every model at its release date. The line moves each time a new release beats every earlier one, and each new best is named where there is room.

0%20%40%60%80% 2025Jul2026Jul DeepSeek-V3DeepSeek-R1o3-miniGemini 2.5 ProClaude Sonnet 4.6Gemini 3.1 Pro PreviewClaude Fable 5Claude Fable 5.1Claude Opus 5.5

Best score by lab

Each lab's best so far, stepping whenever one of its releases beats its own earlier models. The 6 labs with the best scores today; 14 others not drawn.

AnthropicGoogleXiaomiMoonshot AIMetaZ.ai

20%30%40%50%60%70% 2025Jul2026Jul AnthropicGoogleXiaomiMoonshot AIMetaZ.ai

What it measures

Share of scientific coding sub-problems solved, as run by Artificial Analysis with its own harness and prompts.

What it does not measure

Not general software engineering.

291 results from Artificial Analysis not ranked here · show why

We rank a result only when we can tie it to a specific model you can use. These are left out:

  • Not on sale through the API providers we track: 276
  • A different snapshot or variant from the model we list: 14
  • An unusual combination of settings: 1

Data sourced from Artificial Analysis. Licence: Artificial Analysis commercial data licence.