Benchmarks / Artificial Analysis

Reported by Artificial Analysis

Artificial Analysis

Share of questions answered correctly over long documents, as run by Artificial Analysis with its own harness and prompts.

Last updated 8 Oct 2026

Results dated
8 Oct 2026
Results
393 configurations of 233 models
Unit
% of questions
Licence
Artificial Analysis commercial data licence

Long-context reasoning (AA-LCR): DiffusionGemma 26B A4B

Top 15 of 393 results · % of questions, higher is better. Choose a model to highlight it.Clear highlight

  1. 1 Kimi K3 (max reasoning)Moonshot AI 88.7%
  2. 2 MiMo-V2.6-ProXiaomi 86.3%
  3. 3 Claude Fable 5.1 (max reasoning)Anthropic 85.3%
  4. 4 Claude Fable 5.1 (medium reasoning)Anthropic 84.7%
  5. 4 Claude Opus 5.5 (max reasoning)Anthropic 84.7%
  6. 4 Claude Opus 5.5 (extra-high reasoning)Anthropic 84.7%
  7. 7 Claude Opus 5.5 (medium reasoning)Anthropic 84.3%
  8. 7 GPT-5.5 (high reasoning)OpenAI 84.3%
  9. 7 GPT-5.5 (extra-high reasoning)OpenAI 84.3%
  10. 10 DeepSeek-V4.1-Flash (max reasoning)DeepSeek 84.0%
  11. 10 Gemini 3.8 Flash (medium reasoning)Google 84.0%
  12. 10 GPT-5.6 Sol (max reasoning)OpenAI 84.0%
  13. 10 GPT-6.1 Sol (low reasoning)OpenAI 84.0%
  14. 14 Claude Fable 5.1 (high reasoning)Anthropic 83.7%
  15. 14 GPT-5.6 Luna (max reasoning)OpenAI 83.7%
  16. 351 DiffusionGemma 26B A4BGoogle 19.7%

Full results

Artificial Analysis: Long-context reasoning (AA-LCR), % of questions, higher is better
#ModelLong-context reasoning (AA-LCR)
% of questions, higher is better		Price
$ per million tokens, in / out
1 Kimi K3 (max reasoning)Moonshot AI
88.7%
$3 / $15
2 MiMo-V2.6-ProXiaomi
86.3%
$0.43 / $0.87
3 Claude Fable 5.1 (max reasoning)Anthropic
85.3%
$10 / $50
4 Claude Fable 5.1 (medium reasoning)Anthropic
84.7%
$10 / $50
4 Claude Opus 5.5 (max reasoning)Anthropic
84.7%
$4 / $20
4 Claude Opus 5.5 (extra-high reasoning)Anthropic
84.7%
$4 / $20
7 Claude Opus 5.5 (medium reasoning)Anthropic
84.3%
$4 / $20
7 GPT-5.5 (high reasoning)OpenAI
84.3%
$5 / $30
7 GPT-5.5 (extra-high reasoning)OpenAI
84.3%
$5 / $30
10 DeepSeek-V4.1-Flash (max reasoning)DeepSeek
84.0%
$0.30 / $1.20
10 Gemini 3.8 Flash (medium reasoning)Google
84.0%
$1.50 / $7.50
10 GPT-5.6 Sol (max reasoning)OpenAI
84.0%
$4 / $20
10 GPT-6.1 Sol (low reasoning)OpenAI
84.0%
$2 / $10
14 Claude Fable 5.1 (high reasoning)Anthropic
83.7%
$10 / $50
14 GPT-5.6 Luna (max reasoning)OpenAI
83.7%
$0.20 / $1.20
14 GPT-6 Sol (high reasoning)OpenAI
83.7%
$2 / $10
14 GPT-6 Sol (max reasoning)OpenAI
83.7%
$2 / $10
18 Muse Glimmer (high reasoning)Meta
83.3%
–
18 GPT-5.3-Codex (extra-high reasoning)OpenAI
83.3%
$1.75 / $14
18 GPT-6.1 Sol (medium reasoning)OpenAI
83.3%
$2 / $10
18 GPT-6 Luna (max reasoning)OpenAI
83.3%
$0.10 / $0.50
22 Claude Fable 5.1 (extra-high reasoning)Anthropic
83.0%
$10 / $50
22 Gemini 3.7 Flash (medium reasoning)Google
83.0%
$1.50 / $7.50
22 Muse Spark 1.3 (max reasoning)Meta
83.0%
$1.25 / $4.25
22 Muse Spark 1.3 (extra-high reasoning)Meta
83.0%
$1.25 / $4.25
22 MiniMax-M3MiniMax
83.0%
$0.30 / $1.20
22 GPT-5.5 (medium reasoning)OpenAI
83.0%
$5 / $30
22 GPT-5.6 Terra (max reasoning)OpenAI
83.0%
$2 / $12
22 GPT-6.1 Sol (max reasoning)OpenAI
83.0%
$2 / $10
30 Claude Haiku 5.5 (max reasoning)Anthropic
82.7%
$0.10 / $0.50
30 Claude Opus 5.5 (high reasoning)Anthropic
82.7%
$4 / $20
30 Claude Sonnet 5.5 (max reasoning)Anthropic
82.7%
$2 / $10
30 GPT-5.2 (extra-high reasoning)OpenAI
82.7%
$1.75 / $14
34 Claude Fable 5.1 (low reasoning)Anthropic
82.3%
$10 / $50
34 Claude Fable 5 (max reasoning)Anthropic
82.3%
$10 / $50
34 GPT-5.2-Codex (extra-high reasoning)OpenAI
82.3%
$1.75 / $14
34 GPT-5.6 Sol (extra-high reasoning)OpenAI
82.3%
$4 / $20
34 GPT-6.1 Sol (high reasoning)OpenAI
82.3%
$2 / $10
34 GPT-6 Sol (medium reasoning)OpenAI
82.3%
$2 / $10
40 Qwen3.8-27B (extra-high reasoning)Alibaba
82.0%
$0.50 / $3
40 Claude Opus 5 (medium reasoning)Anthropic
82.0%
$5 / $25
40 Claude Sonnet 5 (max reasoning)Anthropic
82.0%
$2 / $10
40 Gemini 3.1 Pro PreviewGoogle
82.0%
$2 / $12
40 GPT-5.4 (extra-high reasoning)OpenAI
82.0%
$2.50 / $15
45 Gemini 3.7 Flash (high reasoning)Google
81.7%
$1.50 / $7.50
45 GPT-5.6 Luna (extra-high reasoning)OpenAI
81.7%
$0.20 / $1.20
45 GPT-5.6 Sol (high reasoning)OpenAI
81.7%
$4 / $20
48 Claude Opus 5 (low reasoning)Anthropic
81.3%
$5 / $25
48 DeepSeek-V4-Flash-Vision-Exp (max reasoning)DeepSeek
81.3%
$0.44 / $1.32
48 Gemini 3.8 Flash (high reasoning)Google
81.3%
$1.50 / $7.50
48 Mistral Large 4Mistral AI
81.3%
$1.36 / $4.18
48 GPT-6 Sol (extra-high reasoning)OpenAI
81.3%
$2 / $10
53 Kimi K2.6Moonshot AI
81.0%
$0.95 / $4
53 GPT-5.5 (low reasoning)OpenAI
81.0%
$5 / $30
53 Grok 4.6 (medium reasoning)xAI
81.0%
$2 / $6
53 Grok 4.6 (extra-high reasoning)xAI
81.0%
$2 / $6
57 Qwen3.6-Max-PreviewAlibaba
80.7%
$1.03 / $6.16
57 Claude Opus 5.5 (low reasoning)Anthropic
80.7%
$4 / $20
57 Gemini 3.8 Flash (low reasoning)Google
80.7%
$1.50 / $7.50
57 GPT-6 Astra (max reasoning)OpenAI
80.7%
$10 / $50
57 Grok 4.6 (low reasoning)xAI
80.7%
$2 / $6
62 Qwen3.8-2.4T-A95BAlibaba
80.3%
$2 / $6
62 Qwen3.8-Max (0902)Alibaba
80.3%
$2 / $6
62 Claude Opus 5 (extra-high reasoning)Anthropic
80.3%
$5 / $25
62 DeepSeek-V4-Pro (0813, max reasoning)DeepSeek
80.3%
$1.32 / $3.96
62 GPT-5.6 Luna (high reasoning)OpenAI
80.3%
$0.20 / $1.20
62 GPT-5.6 Sol (medium reasoning)OpenAI
80.3%
$4 / $20
62 Grok 4.6 (high reasoning)xAI
80.3%
$2 / $6
69 Claude Sonnet 4.6 (max reasoning)Anthropic
80.0%
$3 / $15
69 Gemini 3.6 Flash (high reasoning)Google
80.0%
$1.50 / $7.50
69 GPT-5.1 (high reasoning)OpenAI
80.0%
$1.25 / $10
69 GPT-6 Astra (high reasoning)OpenAI
80.0%
$10 / $50
69 GPT-6 Astra (low reasoning)OpenAI
80.0%
$10 / $50
69 GPT-6 Astra (extra-high reasoning)OpenAI
80.0%
$10 / $50
69 GPT-6 Luna (extra-high reasoning)OpenAI
80.0%
$0.10 / $0.50
69 Grok 4.7 (low reasoning)xAI
80.0%
$2 / $6
69 GLM-5.3-FlashZ.ai
80.0%
$0.15 / $0.50
78 Qwen3.8-27B (medium reasoning)Alibaba
79.7%
$0.50 / $3
78 Qwen3.8-Flash-NextAlibaba
79.7%
–
78 Claude Sonnet 5.5 (extra-high reasoning)Anthropic
79.7%
$2 / $10
78 DeepSeek-V4-Flash (0731, max reasoning)DeepSeek
79.7%
$0.14 / $0.28
78 Gemini 4 Argon (high reasoning)Google
79.7%
–
78 GPT-6.1 Sol (extra-high reasoning)OpenAI
79.7%
$2 / $10
78 GPT-6 Astra (medium reasoning)OpenAI
79.7%
$10 / $50
78 MiMo-V2.5-Pro (reasoning on)Xiaomi
79.7%
$0.43 / $0.87
78 GLM-5.3 (max reasoning)Z.ai
79.7%
$1.40 / $4.40
87 Claude Opus 5 (max reasoning)Anthropic
79.3%
$5 / $25
87 Kimi K2.7 CodeMoonshot AI
79.3%
$0.95 / $4
87 Kimi K3 (low reasoning)Moonshot AI
79.3%
$3 / $15
87 GPT-6 Luna (high reasoning)OpenAI
79.3%
$0.10 / $0.50
87 GPT-6 Sol (low reasoning)OpenAI
79.3%
$2 / $10
87 Grok 4.5 (high reasoning)xAI
79.3%
$2 / $6
93 Qwen3.7-MaxAlibaba
79.0%
$1.48 / $4.42
93 Claude Opus 5 (high reasoning)Anthropic
79.0%
$5 / $25
93 Muse Spark 1.2 (extra-high reasoning)Meta
79.0%
$1.25 / $4.25
93 GPT-5.6 Terra (extra-high reasoning)OpenAI
79.0%
$2 / $12
93 Hy3Tencent
79.0%
$0.14 / $0.58
98 Claude Opus 4.7 (max reasoning)Anthropic
78.7%
$5 / $25
98 Gemini 3.7 Flash (low reasoning)Google
78.7%
$1.50 / $7.50
100 Qwen3.6-PlusAlibaba
78.3%
$0.33 / $1.95
100 Qwen3.8-Max (0803)Alibaba
78.3%
–
100 Claude Haiku 5.5 (extra-high reasoning)Anthropic
78.3%
$0.10 / $0.50
100 MiniMax-M2.7MiniMax
78.3%
$0.30 / $1.20
100 GPT-6 Luna (medium reasoning)OpenAI
78.3%
$0.10 / $0.50
100 GLM-5.2 (max reasoning)Z.ai
78.3%
$1.40 / $4.40
106 GPT-5 (high reasoning)OpenAI
78.2%
$1.25 / $10
107 Claude Opus 4.6 (max reasoning)Anthropic
78.0%
$5 / $25
107 Claude Sonnet 5.5 (high reasoning)Anthropic
78.0%
$2 / $10
107 Gemini 3 Flash Preview (reasoning on)Google
78.0%
$0.50 / $3
107 Muse SparkMeta
78.0%
–
107 Kimi K2.5Moonshot AI
78.0%
$0.57 / $2.85
107 GPT-5.6 Sol (low reasoning)OpenAI
78.0%
$4 / $20
113 Qwen3.5-27BAlibaba
77.7%
$0.27 / $2.16
113 Claude Opus 4.8 (max reasoning)Anthropic
77.7%
$5 / $25
113 Muse Spark 1.1 (extra-high reasoning)Meta
77.7%
$1.25 / $4.25
113 GPT-5.6 Terra (high reasoning)OpenAI
77.7%
$2 / $12
117 Qwen3.5-397B-A17BAlibaba
77.3%
$0.55 / $3.50
117 Qwen3.6-27BAlibaba
77.3%
$0.30 / $3.20
117 Qwen3.8-27B (low reasoning)Alibaba
77.3%
$0.50 / $3
117 Claude Haiku 5.5 (high reasoning)Anthropic
77.3%
$0.10 / $0.50
117 Claude Haiku 5.5 (medium reasoning)Anthropic
77.3%
$0.10 / $0.50
117 Claude Opus 4.5 (reasoning on)Anthropic
77.3%
$5 / $25
117 Inkling (extra-high reasoning)Thinking Machines
77.3%
$0.95 / $4.05
124 GPT-5.4 mini (extra-high reasoning)OpenAI
77.0%
$0.75 / $4.50
124 Grok 4.7 (high reasoning)xAI
77.0%
$2 / $6
126 Claude Sonnet 5 (high reasoning)Anthropic
76.7%
$2 / $10
126 Claude Sonnet 5 (extra-high reasoning)Anthropic
76.7%
$2 / $10
126 GPT-5.4 nano (extra-high reasoning)OpenAI
76.7%
$0.20 / $1.25
126 GPT-5.4 (low reasoning)OpenAI
76.7%
$2.50 / $15
126 Grok 4.7 (extra-high reasoning)xAI
76.7%
$2 / $6
131 Qwen3.5-122B-A10BAlibaba
76.3%
$0.26 / $2.08
131 Claude Sonnet 5.5 (medium reasoning)Anthropic
76.3%
$2 / $10
133 Claude Opus 4.1 (reasoning on)Anthropic
76.0%
$15 / $75
133 Claude Sonnet 5.5 (low reasoning)Anthropic
76.0%
$2 / $10
133 Gemini 3.5 Flash-LiteGoogle
76.0%
$0.30 / $2.50
133 Gemini 3 Pro Preview (high reasoning)Google
76.0%
–
133 GPT-5 (medium reasoning)OpenAI
76.0%
$1.25 / $10
138 Claude Opus 4.7 (no reasoning)Anthropic
75.7%
$5 / $25
138 Inkling SmallThinking Machines
75.7%
$0.45 / $1.20
138 GLM-5Z.ai
75.7%
$0.95 / $2.55
141 GPT-5.6 Luna (medium reasoning)OpenAI
75.0%
$0.20 / $1.20
141 Grok 4.3 (medium reasoning)xAI
75.0%
$1.25 / $2.50
143 DeepSeek-V4-Pro (0423, max reasoning)DeepSeek
74.7%
$1.42 / $2.83
143 o3OpenAI
74.7%
$2 / $8
143 Grok Build 0.1xAI
74.7%
$1 / $2
146 Qwen3-Max-ThinkingAlibaba
74.3%
$0.78 / $3.90
146 Claude Haiku 4.5 (reasoning on)Anthropic
74.3%
$1 / $5
146 DeepSeek-V4-Flash (0423, max reasoning)DeepSeek
74.3%
$0.14 / $0.28
146 Gemini 3.1 Flash-Lite PreviewGoogle
74.3%
$0.25 / $1.50
146 Gemini 3.5 Flash (medium reasoning)Google
74.3%
$1.50 / $9
146 MiMo-V2.6-FlashXiaomi
74.3%
$0.14 / $0.28
152 Gemini 3 Pro Preview (low reasoning)Google
74.0%
–
152 GPT-5.6 Terra (medium reasoning)OpenAI
74.0%
$2 / $12
152 GPT-6 Luna (low reasoning)OpenAI
74.0%
$0.10 / $0.50
152 Solar Pro 4Upstage
74.0%
$0.09 / $0.36
152 Grok 4.1 Fast (reasoning)xAI
74.0%
–
152 Grok 4.3 (low reasoning)xAI
74.0%
$1.25 / $2.50
158 Claude Sonnet 5 (medium reasoning)Anthropic
73.7%
$2 / $10
158 Grok 4 Fast (reasoning)xAI
73.7%
–
158 GLM-5.1Z.ai
73.7%
$1.38 / $4.40
161 DeepSeek-V3.2 (reasoning on)DeepSeek
73.3%
$0.30 / $0.96
161 Gemini 3.5 Flash (high reasoning)Google
73.3%
$1.50 / $9
161 MiniMax-M2.5MiniMax
73.3%
$0.30 / $1.20
164 Qwen3.7-PlusAlibaba
73.0%
$0.32 / $1.28
164 Grok 4.3 (high reasoning)xAI
73.0%
$1.25 / $2.50
164 MiMo-V2.5Xiaomi
73.0%
$0.17 / $0.34
167 GLM-5.3 (low reasoning)Z.ai
72.7%
$1.40 / $4.40
168 Claude Sonnet 4.5 (reasoning on)Anthropic
72.3%
$3 / $15
168 DeepSeek-V3.2-Exp (reasoning on)DeepSeek
72.3%
$0.27 / $0.41
168 GPT-5 mini (high reasoning)OpenAI
72.3%
$0.25 / $2
168 GPT-5 mini (medium reasoning)OpenAI
72.3%
$0.25 / $2
172 Qwen3-235B-A22B-Thinking-2507Alibaba
72.0%
$0.30 / $3
172 Qwen3.5-35B-A3BAlibaba
72.0%
$0.16 / $1.30
172 Claude Haiku 5.5 (low reasoning)Anthropic
72.0%
$0.10 / $0.50
172 DeepSeek-V4-Flash (0423, high reasoning)DeepSeek
72.0%
$0.14 / $0.28
172 Kimi K2 ThinkingMoonshot AI
72.0%
$0.60 / $2.50
177 Qwen3.6-35B-A3BAlibaba
71.7%
$0.10 / $1
177 GPT-5-Codex (high reasoning)OpenAI
71.7%
–
177 GLM-5-TurboZ.ai
71.7%
$1.20 / $4
180 GPT-5.6 Terra (low reasoning)OpenAI
71.3%
$2 / $12
181 Gemini 2.5 Flash Preview (09-2025, reasoning on)Google
71.0%
–
181 GLM-4.7Z.ai
71.0%
$0.54 / $1.98
183 Claude Opus 4.5 (no reasoning)Anthropic
70.7%
$5 / $25
184 Claude Sonnet 4 (reasoning on)Anthropic
70.3%
$3 / $15
184 DeepSeek-V4-Pro (0423, high reasoning)DeepSeek
70.3%
$1.42 / $2.83
184 GPT-5.2 (medium reasoning)OpenAI
70.3%
$1.75 / $14
184 GLM-5V-TurboZ.ai
70.3%
$1.20 / $4
188 Qwen3.5-9BAlibaba
70.0%
$0.10 / $0.15
188 Claude Sonnet 5 (no reasoning)Anthropic
70.0%
$2 / $10
188 DeepSeek-V3.2-SpecialeDeepSeek
70.0%
–
188 GPT-5.5 Instant (2026-06-26)OpenAI
70.0%
–
188 GPT-5.6 Luna (low reasoning)OpenAI
70.0%
$0.20 / $1.20
193 Gemma 4 31BGoogle
69.7%
$0.14 / $0.40
193 Kimi K2.6 (no reasoning)Moonshot AI
69.7%
$0.95 / $4
195 Qwen3.8-27B (no reasoning)Alibaba
69.3%
$0.50 / $3
195 DeepSeek-V3.1-Terminus (reasoning on)DeepSeek
69.3%
$0.27 / $1
195 Mistral Medium 3.5Mistral AI
69.3%
$1.50 / $7.50
195 GPT-5.1-Codex (high reasoning)OpenAI
69.3%
$1.25 / $10
199 Gemini 2.5 ProGoogle
69.0%
$1.25 / $10
199 Grok 4.20xAI
69.0%
$1.25 / $2.50
201 Claude Sonnet 4.6 (no reasoning)Anthropic
68.3%
$3 / $15
201 GPT-4.1OpenAI
68.3%
$2 / $8
203 Grok 4xAI
68.0%
–
204 MiniMax-M2.1MiniMax
67.7%
$0.30 / $1.20
204 Grok 4.20 (0309, reasoning)xAI
67.7%
–
206 Claude Sonnet 5 (low reasoning)Anthropic
67.3%
$2 / $10
206 Kimi K2.5 (no reasoning)Moonshot AI
67.3%
$0.57 / $2.85
206 GPT-5.4 nano (medium reasoning)OpenAI
67.3%
$0.20 / $1.25
209 Claude Opus 4.6 (no reasoning)Anthropic
67.0%
$5 / $25
209 GPT-5.4 mini (medium reasoning)OpenAI
67.0%
$0.75 / $4.50
211 Qwen3.6-27B (no reasoning)Alibaba
66.7%
$0.30 / $3.20
211 GPT-5.1-Codex-Mini (high reasoning)OpenAI
66.7%
$0.25 / $2
213 GPT-5.5 Instant (2026-05-26)OpenAI
66.3%
–
214 Gemma 4 26B A4BGoogle
65.7%
$0.10 / $0.30
214 Nemotron 3 Super 120B A12BNVIDIA
65.7%
$0.085 / $0.40
216 Gemini 2.5 Flash (reasoning on)Google
65.3%
$0.30 / $2.50
217 GPT-5 ChatOpenAI
65.0%
–
217 o1OpenAI
65.0%
$15 / $60
219 Gemini 2.5 Flash-Lite Preview (09-2025, reasoning on)Google
64.7%
–
219 Hy3 Preview (reasoning on)Tencent
64.7%
$0.18 / $0.60
221 Qwen3.5-397B-A17B (no reasoning)Alibaba
64.3%
$0.55 / $3.50
221 Qwen3.6-35B-A3B (no reasoning)Alibaba
64.3%
$0.10 / $1
221 MiniMax-M2MiniMax
64.3%
$0.30 / $1.20
224 Qwen3.5-27B (no reasoning)Alibaba
64.0%
$0.27 / $2.16
224 GPT-5.5 (no reasoning)OpenAI
64.0%
$5 / $30
224 GPT-6 Sol (no reasoning)OpenAI
64.0%
$2 / $10
227 Qwen3.5-Omni-PlusAlibaba
63.7%
–
227 Qwen3-Next-80B-A3B-ThinkingAlibaba
63.7%
$0.15 / $1.20
227 Qwen3-VL-235B-A22B-ThinkingAlibaba
63.7%
$0.40 / $4
227 Gemma 4 12BGoogle
63.7%
–
231 Qwen3.5-35B-A3B (no reasoning)Alibaba
63.0%
$0.16 / $1.30
231 Qwen3.5-4B (reasoning on)Alibaba
63.0%
–
233 GPT-5.6 Sol (no reasoning)OpenAI
62.3%
$4 / $20
234 Qwen3-Max-Thinking-PreviewAlibaba
62.0%
–
235 Qwen3-30B-A3B-Thinking-2507Alibaba
61.3%
$0.20 / $2.40
235 Qwen3.5-122B-A10B (no reasoning)Alibaba
61.3%
$0.26 / $2.08
235 Gemini 3.5 Flash (minimal reasoning)Google
61.3%
$1.50 / $9
238 o4-mini (high reasoning)OpenAI
61.0%
$1.10 / $4.40
239 Nova 2 Lite (high reasoning)Amazon
60.3%
$0.30 / $2.50
240 Nova 2 Lite (medium reasoning)Amazon
60.0%
$0.30 / $2.50
240 Gemini 2.5 Flash Preview (09-2025, no reasoning)Google
60.0%
–
242 GPT-5.6 Terra (no reasoning)OpenAI
58.7%
$2 / $12
243 GPT-5.4 (no reasoning)OpenAI
58.3%
$2.50 / $15
244 DeepSeek-R1DeepSeek
57.7%
$0.70 / $2.50
245 DeepSeek-V3.1 (reasoning on)DeepSeek
56.7%
$0.55 / $1.65
246 DeepSeek-R1-0528DeepSeek
55.7%
$0.50 / $2.18
246 Gemini 2.5 Flash-Lite (reasoning on)Google
55.7%
$0.10 / $0.40
248 Qwen3-VL-32B-ThinkingAlibaba
55.3%
–
248 DeepSeek-V4.1-Flash (no reasoning)DeepSeek
55.3%
$0.30 / $1.20
248 Gemini 3 Flash Preview (no reasoning)Google
55.3%
$0.50 / $3
251 Nova 2 Lite (low reasoning)Amazon
54.3%
$0.30 / $2.50
252 Claude Sonnet 4.5 (no reasoning)Anthropic
54.0%
$3 / $15
252 GLM-4.6 (reasoning on)Z.ai
54.0%
$0.50 / $2
254 GLM-5.1 (no reasoning)Z.ai
53.3%
$1.38 / $4.40
255 DeepSeek-V4-Pro (0423, no reasoning)DeepSeek
53.0%
$1.42 / $2.83
255 Magistral Medium 1.2Mistral AI
53.0%
–
255 Kimi K2 (0905)Moonshot AI
53.0%
$0.60 / $2.50
255 Kimi K2 (0711)Moonshot AI
53.0%
$0.57 / $2.30
255 Grok Code Fast 1xAI
53.0%
–
260 Qwen3-Next-80B-A3B-InstructAlibaba
52.7%
$0.10 / $1.10
260 GLM-4.5Z.ai
52.7%
$0.60 / $2.20
262 Qwen3.5-Omni-FlashAlibaba
52.0%
–
262 gpt-oss-120b (high reasoning)OpenAI
52.0%
$0.15 / $0.60
264 DeepSeek-V4-Pro (0813, no reasoning)DeepSeek
50.7%
$1.32 / $3.96
265 Qwen3-MaxAlibaba
50.0%
$0.78 / $3.90
265 Llama 4 MaverickMeta
50.0%
$0.27 / $0.85
265 Step 3.5 FlashStepFun
50.0%
$0.10 / $0.30
268 Gemini 2.5 Flash (no reasoning)Google
49.9%
$0.30 / $2.50
269 Claude Haiku 4.5 (no reasoning)Anthropic
49.7%
$1 / $5
269 Gemini 2.5 Flash-Lite Preview (09-2025, no reasoning)Google
49.7%
–
269 Mistral Small 4Mistral AI
49.7%
$0.15 / $0.60
272 GLM-4.6V (reasoning on)Z.ai
48.7%
$0.30 / $0.90
273 Qwen3-Coder-NextAlibaba
47.0%
$0.18 / $0.90
273 DeepSeek-V3.1 (no reasoning)DeepSeek
47.0%
$0.55 / $1.65
275 Gemma 4 31B (no reasoning)Google
46.7%
$0.14 / $0.40
275 GLM-4.5-AirZ.ai
46.7%
$0.14 / $0.86
277 Qwen3.5-9B (no reasoning)Alibaba
46.0%
$0.10 / $0.15
277 gpt-oss-120b (low reasoning)OpenAI
46.0%
$0.15 / $0.60
279 Qwen3-Coder-480B-A35BAlibaba
45.7%
$0.35 / $1.50
279 DeepSeek-V3.2 (no reasoning)DeepSeek
45.7%
$0.30 / $0.96
279 GPT-5.2 (no reasoning)OpenAI
45.7%
$1.75 / $14
282 DeepSeek-V3.1-Terminus (no reasoning)DeepSeek
45.3%
$0.27 / $1
283 Granite 4.2 8BIBM
45.0%
$0.06 / $0.25
283 GPT-5.1 (no reasoning)OpenAI
45.0%
$1.25 / $10
283 GPT-5 nano (high reasoning)OpenAI
45.0%
$0.05 / $0.40
286 DeepSeek-V3.2-Exp (no reasoning)DeepSeek
44.7%
$0.27 / $0.41
287 Claude Sonnet 4 (no reasoning)Anthropic
44.0%
$3 / $15
287 GPT-4.1 miniOpenAI
44.0%
$0.40 / $1.60
289 Mercury 2Inception
43.7%
$0.25 / $0.75
289 GPT-5 nano (medium reasoning)OpenAI
43.7%
$0.05 / $0.40
289 GLM-5 (no reasoning)Z.ai
43.7%
$0.95 / $2.55
292 Qwen3-Max-PreviewAlibaba
43.0%
–
292 o3-mini (high reasoning)OpenAI
43.0%
$1.10 / $4.40
294 Mistral Medium 3.1Mistral AI
42.7%
$0.40 / $2
294 GPT-5.6 Luna (no reasoning)OpenAI
42.7%
$0.20 / $1.20
296 Gemma 4 26B A4B (no reasoning)Google
42.3%
$0.10 / $0.30
296 GLM-5.2 (no reasoning)Z.ai
42.3%
$1.40 / $4.40
298 Hy3 Preview (no reasoning)Tencent
42.0%
$0.18 / $0.60
299 DeepSeek-V4-Flash (0423, no reasoning)DeepSeek
41.7%
$0.14 / $0.28
299 MiMo-V2.5-Pro (no reasoning)Xiaomi
41.7%
$0.43 / $0.87
299 GLM-4.7-FlashZ.ai
41.7%
$0.06 / $0.40
302 GPT-4o (2024-08-06)OpenAI
41.0%
$2.50 / $10
303 DeepSeek-V3-0324DeepSeek
40.7%
$0.25 / $1
303 GLM-4.7 (no reasoning)Z.ai
40.7%
$0.54 / $1.98
305 GPT-6 Luna (no reasoning)OpenAI
39.7%
$0.10 / $0.50
306 GPT-5 mini (minimal reasoning)OpenAI
39.3%
$0.25 / $2
307 Trinity Large ThinkingArcee AI
38.0%
$0.25 / $0.80
307 Nemotron 3 Nano 30B A3B (reasoning on)NVIDIA
38.0%
$0.05 / $0.20
309 Qwen3-4B-Thinking-2507Alibaba
37.3%
–
310 GPT-5.4 mini (no reasoning)OpenAI
37.0%
$0.75 / $4.50
311 Mistral Large 3Mistral AI
36.0%
$0.50 / $1.50
312 Gemma 4 12B (no reasoning)Google
35.0%
–
313 Qwen3.5-4B (no reasoning)Alibaba
34.7%
–
313 gpt-oss-20b (high reasoning)OpenAI
34.7%
$0.03 / $0.15
315 Qwen3-235B-A22B-Instruct-2507Alibaba
33.9%
$0.15 / $0.75
316 Qwen3-VL-8B-ThinkingAlibaba
33.3%
$0.18 / $2.10
317 Qwen3-Coder-30B-A3B-InstructAlibaba
32.7%
$0.07 / $0.28
317 Qwen3-VL-235B-A22B-InstructAlibaba
32.7%
$0.30 / $1.50
319 Devstral 2Mistral AI
32.3%
$0.40 / $2
319 Solar Pro 3Upstage
32.3%
$0.15 / $0.60
319 Grok 4.3 (no reasoning)xAI
32.3%
$1.25 / $2.50
322 Gemini 2.5 Flash-Lite (no reasoning)Google
32.0%
$0.10 / $0.40
322 Gemma 4 E4BGoogle
32.0%
–
322 Devstral MediumMistral AI
32.0%
–
325 Mistral Medium 3Mistral AI
31.3%
$0.40 / $2
325 Grok 4.1 Fast (non-reasoning, no reasoning)xAI
31.3%
–
327 gpt-oss-20b (low reasoning)OpenAI
31.0%
$0.03 / $0.15
328 GPT-5.4 nano (no reasoning)OpenAI
29.7%
$0.20 / $1.25
329 DeepSeek-V3DeepSeek
29.3%
$0.26 / $1.03
330 Mistral Small 4 (no reasoning)Mistral AI
28.3%
$0.15 / $0.60
331 Devstral Small 2Mistral AI
28.0%
–
331 Kimi Linear 48B A3B InstructMoonshot AI
28.0%
–
333 Llama 4 ScoutMeta
27.7%
$0.18 / $0.59
334 Qwen3-30B-A3B-Instruct-2507Alibaba
26.3%
$0.09 / $0.30
334 Ministral 3 14BMistral AI
26.3%
$0.20 / $0.20
334 GLM-4.6 (no reasoning)Z.ai
26.3%
$0.50 / $2
337 Ministral 3 8BMistral AI
25.7%
$0.15 / $0.15
338 Grok 4.20 (0309, non-reasoning, no reasoning)xAI
25.3%
–
339 Gemma 4 E4B (no reasoning)Google
24.0%
–
339 Grok 4 Fast (non-reasoning, no reasoning)xAI
24.0%
–
341 Grok 4.20 (no reasoning)xAI
23.3%
$1.25 / $2.50
342 Mistral Small 3.1 24BMistral AI
22.3%
$0.35 / $0.56
343 Qwen3-VL-4B-ThinkingAlibaba
21.3%
–
343 Command ACohere
21.3%
$2.50 / $10
345 Qwen3.5-2B (reasoning on)Alibaba
21.0%
–
345 Nova Pro 1.0Amazon
21.0%
$0.80 / $3.20
347 Mistral Small 3.2 24BMistral AI
20.3%
$0.094 / $0.25
347 GPT-4.1 nanoOpenAI
20.3%
$0.10 / $0.40
347 GLM-4.7-Flash (no reasoning)Z.ai
20.3%
$0.06 / $0.40
350 GPT-5 nano (minimal reasoning)OpenAI
20.0%
$0.05 / $0.40
351 DiffusionGemma 26B A4BGoogle
19.7%
–
352 Magistral Small 1.2Mistral AI
19.3%
–
353 Nova Lite 1.0Amazon
19.0%
$0.06 / $0.24
354 Nova 2 Lite (no reasoning)Amazon
18.7%
$0.30 / $2.50
354 Devstral Small 1.1Mistral AI
18.7%
–
356 Llama 3.1 8B InstructMeta
18.0%
$0.05 / $0.08
357 Ministral 3 3BMistral AI
17.0%
$0.10 / $0.10
357 GLM-4.6V (no reasoning)Z.ai
17.0%
$0.30 / $0.90
359 Qwen3-VL-8B-InstructAlibaba
16.7%
$0.12 / $0.46
360 Gemma 4 E2B (no reasoning)Google
16.3%
–
360 Gemma 4 E2BGoogle
16.3%
–
362 Llama 3.3 70B InstructMeta
15.7%
$0.59 / $0.79
363 Qwen3.5-2B (no reasoning)Alibaba
14.0%
–
363 Qwen3-VL-4B-InstructAlibaba
14.0%
–
365 Nova Micro 1.0Amazon
13.0%
$0.035 / $0.14
366 Qwen3-4B-Instruct-2507Alibaba
11.3%
–
367 Nemotron 3 Nano 30B A3B (no reasoning)NVIDIA
10.7%
$0.05 / $0.20
368 Qwen3.5-0.8B (reasoning on)Alibaba
9.0%
–
369 Gemma 3 12BGoogle
8.3%
$0.05 / $0.15
370 Qwen3.5-0.8B (no reasoning)Alibaba
8.0%
–
371 Gemma 3 27BGoogle
7.3%
$0.12 / $0.20
372 Gemma 3 4BGoogle
6.7%
$0.05 / $0.10
372 Llama 3.2 1B InstructMeta
6.7%
$0.027 / $0.20
374 Llama 3.2 3B InstructMeta
4.3%
$0.05 / $0.33
375 Mistral Large 2 (2407)Mistral AI
2.0%
$2 / $6
376 Qwen3-14B (no reasoning)Alibaba
0.0%
$0.12 / $0.24
376 Qwen3-14B (reasoning on)Alibaba
0.0%
$0.12 / $0.24
376 Qwen3-235B-A22B (no reasoning)Alibaba
0.0%
$0.46 / $1.82
376 Qwen3-235B-A22B (reasoning on)Alibaba
0.0%
$0.46 / $1.82
376 Qwen3-30B-A3B (no reasoning)Alibaba
0.0%
$0.12 / $0.50
376 Qwen3-30B-A3B (reasoning on)Alibaba
0.0%
$0.12 / $0.50
376 Qwen3-32B (no reasoning)Alibaba
0.0%
$0.14 / $0.40
376 Qwen3-32B (reasoning on)Alibaba
0.0%
$0.14 / $0.40
376 Qwen3-8B (no reasoning)Alibaba
0.0%
$0.12 / $0.46
376 Qwen3-8B (reasoning on)Alibaba
0.0%
$0.12 / $0.46
376 Qwen3-Omni-30B-A3B-InstructAlibaba
0.0%
–
376 Qwen3-Omni-30B-A3B-ThinkingAlibaba
0.0%
–
376 Gemma 3 270MGoogle
0.0%
–
376 Phi-4Microsoft
0.0%
$0.07 / $0.14
376 Mistral Small 3Mistral AI
0.0%
$0.05 / $0.08
376 Reka Flash 3Reka AI
0.0%
$0.10 / $0.20
376 GLM-4.5V (no reasoning)Z.ai
0.0%
$0.60 / $1.80
376 GLM-4.5V (reasoning on)Z.ai
0.0%
$0.60 / $1.80

Swipe the table sideways for more columns.

Ranks follow the score as shown, so equal numbers share a rank. Results as published by Artificial Analysis; we do not re-run them.

What it measures

Share of questions answered correctly over long documents, as run by Artificial Analysis with its own harness and prompts.

What it does not measure

Not retrieval over your own document store.

Source

Artificial Analysis · Methodology

282 results from Artificial Analysis not ranked here · show why

We rank a result only when we can tie it to a specific model you can use. These are left out:

  • Not on sale through the API providers we track: 268
  • A different snapshot or variant from the model we list: 13
  • An unusual combination of settings: 1

Data sourced from Artificial Analysis. Licence: Artificial Analysis commercial data licence.