Benchmarks / Artificial Analysis

Reported by Artificial Analysis

Artificial Analysis

Share of precise, checkable formatting instructions followed, as run by Artificial Analysis with its own harness and prompts.

Last updated 8 Oct 2026

Results dated
8 Oct 2026
Results
288 configurations of 195 models
Unit
% of instructions
Licence
Artificial Analysis commercial data licence

IFBench: Qwen3.5-2B

Top 15 of 288 results · % of instructions, higher is better. Choose a model to highlight it.Clear highlight

  1. 1 Grok 4.3 (medium reasoning)xAI 83.3%
  2. 2 Grok 4.20 (0309, reasoning)xAI 82.9%
  3. 2 MiniMax-M3MiniMax 82.9%
  4. 4 Grok 4.3 (high reasoning)xAI 81.3%
  5. 5 Grok 4.20xAI 81.2%
  6. 6 Grok 4.3 (low reasoning)xAI 81.0%
  7. 7 Qwen3.7-MaxAlibaba 80.5%
  8. 8 MiMo-V2.5-Pro (reasoning on)Xiaomi 79.9%
  9. 9 DeepSeek-V4-Flash (0423, max reasoning)DeepSeek 79.2%
  10. 10 Qwen3.5-397B-A17BAlibaba 78.8%
  11. 11 Qwen3.7-PlusAlibaba 78.0%
  12. 11 Gemini 3 Flash Preview (reasoning on)Google 78.0%
  13. 13 GPT-5.2-Codex (extra-high reasoning)OpenAI 77.6%
  14. 14 Gemini 3.1 Flash-Lite PreviewGoogle 77.2%
  15. 15 Gemini 3.1 Pro PreviewGoogle 77.1%
  16. 261 Qwen3.5-2B (reasoning on)Alibaba 31.5%
  17. 272 Qwen3.5-2B (no reasoning)Alibaba 29.1%

Full results

Artificial Analysis: IFBench, % of instructions, higher is better
#ModelIFBench
% of instructions, higher is better		Price
$ per million tokens, in / out
1 Grok 4.3 (medium reasoning)xAI
83.3%
$1.25 / $2.50
2 Grok 4.20 (0309, reasoning)xAI
82.9%
–
2 MiniMax-M3MiniMax
82.9%
$0.30 / $1.20
4 Grok 4.3 (high reasoning)xAI
81.3%
$1.25 / $2.50
5 Grok 4.20xAI
81.2%
$1.25 / $2.50
6 Grok 4.3 (low reasoning)xAI
81.0%
$1.25 / $2.50
7 Qwen3.7-MaxAlibaba
80.5%
$1.48 / $4.42
8 MiMo-V2.5-Pro (reasoning on)Xiaomi
79.9%
$0.43 / $0.87
9 DeepSeek-V4-Flash (0423, max reasoning)DeepSeek
79.2%
$0.14 / $0.28
10 Qwen3.5-397B-A17BAlibaba
78.8%
$0.55 / $3.50
11 Qwen3.7-PlusAlibaba
78.0%
$0.32 / $1.28
11 Gemini 3 Flash Preview (reasoning on)Google
78.0%
$0.50 / $3
13 GPT-5.2-Codex (extra-high reasoning)OpenAI
77.6%
$1.75 / $14
14 Gemini 3.1 Flash-Lite PreviewGoogle
77.2%
$0.25 / $1.50
15 Gemini 3.1 Pro PreviewGoogle
77.1%
$2 / $12
16 Qwen3.6-Max-PreviewAlibaba
76.6%
$1.03 / $6.16
17 DeepSeek-V4-Pro (0423, max reasoning)DeepSeek
76.5%
$1.42 / $2.83
18 Gemini 3.5 Flash (high reasoning)Google
76.3%
$1.50 / $9
18 GLM-5.1Z.ai
76.3%
$1.38 / $4.40
20 Kimi K2.6Moonshot AI
76.0%
$0.95 / $4
21 Muse SparkMeta
75.9%
–
21 GPT-5.4 nano (extra-high reasoning)OpenAI
75.9%
$0.20 / $1.25
21 GPT-5.5 (extra-high reasoning)OpenAI
75.9%
$5 / $30
24 Qwen3.5-122B-A10BAlibaba
75.7%
$0.26 / $2.08
24 MiniMax-M2.7MiniMax
75.7%
$0.30 / $1.20
26 Qwen3.5-27BAlibaba
75.6%
$0.27 / $2.16
26 Gemma 4 31BGoogle
75.6%
$0.14 / $0.40
28 GPT-5.2 (extra-high reasoning)OpenAI
75.4%
$1.75 / $14
28 GPT-5 mini (high reasoning)OpenAI
75.4%
$0.25 / $2
28 GPT-5.3-Codex (extra-high reasoning)OpenAI
75.4%
$1.75 / $14
31 Qwen3.6-PlusAlibaba
75.2%
$0.33 / $1.95
32 Gemini 3.5 Flash (medium reasoning)Google
74.6%
$1.50 / $9
33 GPT-5-Codex (high reasoning)OpenAI
74.2%
–
34 GPT-5.4 (extra-high reasoning)OpenAI
73.9%
$2.50 / $15
35 Gemma 4 12BGoogle
73.5%
–
35 DeepSeek-V4-Flash (0423, high reasoning)DeepSeek
73.5%
$0.14 / $0.28
37 GLM-5.2 (max reasoning)Z.ai
73.3%
$1.40 / $4.40
37 GPT-5.4 mini (extra-high reasoning)OpenAI
73.3%
$0.75 / $4.50
39 GLM-5-TurboZ.ai
73.2%
$1.20 / $4
40 GPT-5 (high reasoning)OpenAI
73.1%
$1.25 / $10
41 GPT-5.1 (high reasoning)OpenAI
72.9%
$1.25 / $10
42 GPT-5.6 Sol (max reasoning)OpenAI
72.7%
$4 / $20
43 Qwen3.5-35B-A3BAlibaba
72.5%
$0.16 / $1.30
44 Gemma 4 26B A4BGoogle
72.4%
$0.10 / $0.30
45 MiniMax-M2MiniMax
72.3%
$0.30 / $1.20
45 GLM-5Z.ai
72.3%
$0.95 / $2.55
47 MiniMax-M2.5MiniMax
71.6%
$0.30 / $1.20
47 GPT-5.5 (high reasoning)OpenAI
71.6%
$5 / $30
49 Nemotron 3 Super 120B A12BNVIDIA
71.5%
$0.085 / $0.40
49 GPT-5.5 Instant (2026-05-26)OpenAI
71.5%
–
51 o3OpenAI
71.4%
$2 / $8
52 DeepSeek-V4-Pro (0423, high reasoning)DeepSeek
71.3%
$1.42 / $2.83
53 GPT-5.6 Terra (max reasoning)OpenAI
71.2%
$2 / $12
53 Solar Pro 3Upstage
71.2%
$0.15 / $0.60
53 GPT-5 mini (medium reasoning)OpenAI
71.2%
$0.25 / $2
56 Nemotron 3 Nano 30B A3B (reasoning on)NVIDIA
71.1%
$0.05 / $0.20
57 GPT-5.6 Sol (extra-high reasoning)OpenAI
71.0%
$4 / $20
57 GPT-5.5 (medium reasoning)OpenAI
71.0%
$5 / $30
59 Qwen3-Max-ThinkingAlibaba
70.7%
$0.78 / $3.90
59 Nova 2 Lite (high reasoning)Amazon
70.7%
$0.30 / $2.50
61 GPT-5 (medium reasoning)OpenAI
70.6%
$1.25 / $10
62 Gemini 3 Pro Preview (high reasoning)Google
70.4%
–
63 o1OpenAI
70.3%
$15 / $60
64 Kimi K2.5Moonshot AI
70.2%
$0.57 / $2.85
65 GPT-5.1-Codex (high reasoning)OpenAI
70.0%
$1.25 / $10
66 MiniMax-M2.1MiniMax
69.9%
$0.30 / $1.20
67 Mercury 2Inception
69.8%
$0.25 / $0.75
68 GPT-5.6 Sol (medium reasoning)OpenAI
69.6%
$4 / $20
69 GPT-5.6 Sol (high reasoning)OpenAI
69.2%
$4 / $20
70 gpt-oss-120b (high reasoning)OpenAI
69.0%
$0.15 / $0.60
71 Mistral Medium 3.5Mistral AI
68.8%
$1.50 / $7.50
72 o4-mini (high reasoning)OpenAI
68.7%
$1.10 / $4.40
73 Nova 2 Lite (medium reasoning)Amazon
68.5%
$0.30 / $2.50
74 Kimi K2 ThinkingMoonshot AI
68.1%
$0.60 / $2.50
75 GPT-5.1-Codex-Mini (high reasoning)OpenAI
67.9%
$0.25 / $2
75 GLM-4.7Z.ai
67.9%
$0.54 / $1.98
77 Qwen3.6-27BAlibaba
67.6%
$0.30 / $3.20
77 GPT-5 nano (high reasoning)OpenAI
67.6%
$0.05 / $0.40
79 o3-mini (high reasoning)OpenAI
67.1%
$1.10 / $4.40
79 MiMo-V2.5Xiaomi
67.1%
$0.17 / $0.34
81 Qwen3.5-9BAlibaba
66.7%
$0.10 / $0.15
82 GPT-5 (low reasoning)OpenAI
66.6%
$1.25 / $10
83 GPT-5.6 Sol (low reasoning)OpenAI
66.5%
$4 / $20
84 GPT-5.6 Terra (extra-high reasoning)OpenAI
66.3%
$2 / $12
85 GPT-5.4 (low reasoning)OpenAI
65.9%
$2.50 / $15
85 GPT-5 nano (medium reasoning)OpenAI
65.9%
$0.05 / $0.40
87 GPT-5.2 (medium reasoning)OpenAI
65.2%
$1.75 / $14
88 gpt-oss-20b (high reasoning)OpenAI
65.1%
$0.03 / $0.15
89 GPT-5.4 mini (medium reasoning)OpenAI
64.8%
$0.75 / $4.50
90 Step 3.5 FlashStepFun
64.6%
$0.10 / $0.30
91 GPT-5.4 nano (medium reasoning)OpenAI
64.4%
$0.20 / $1.25
91 GPT-5.6 Terra (high reasoning)OpenAI
64.4%
$2 / $12
91 Qwen3.6-35B-A3BAlibaba
64.4%
$0.10 / $1
91 GPT-5.5 (low reasoning)OpenAI
64.4%
$5 / $30
95 DeepSeek-V3.2-SpecialeDeepSeek
63.9%
–
96 Claude Fable 5 (max reasoning)Anthropic
63.5%
$10 / $50
97 Kimi K2.7 CodeMoonshot AI
63.1%
$0.95 / $4
97 Hy3 Preview (reasoning on)Tencent
63.1%
$0.18 / $0.60
99 Claude Opus 4.8 (max reasoning)Anthropic
62.2%
$5 / $25
99 GPT-5.6 Terra (medium reasoning)OpenAI
62.2%
$2 / $12
101 Nova 2 Lite (low reasoning)Amazon
61.2%
$0.30 / $2.50
102 GLM-5V-TurboZ.ai
61.1%
$1.20 / $4
103 GLM-4.7-FlashZ.ai
60.8%
$0.06 / $0.40
104 Qwen3-Next-80B-A3B-ThinkingAlibaba
60.7%
$0.15 / $1.20
104 DeepSeek-V3.2 (reasoning on)DeepSeek
60.7%
$0.30 / $0.96
106 GPT-5.6 Terra (low reasoning)OpenAI
59.7%
$2 / $12
107 DiffusionGemma 26B A4BGoogle
59.5%
–
108 Qwen3-VL-32B-ThinkingAlibaba
59.4%
–
109 Claude Opus 4.7 (max reasoning)Anthropic
58.6%
$5 / $25
110 gpt-oss-120b (low reasoning)OpenAI
58.3%
$0.15 / $0.60
111 Claude Opus 4.5 (reasoning on)Anthropic
58.0%
$5 / $25
112 gpt-oss-20b (low reasoning)OpenAI
57.8%
$0.03 / $0.15
113 Claude Sonnet 4.5 (reasoning on)Anthropic
57.3%
$3 / $15
114 DeepSeek-V3.1-Terminus (reasoning on)DeepSeek
57.0%
$0.27 / $1
115 Claude Sonnet 4.6 (max reasoning)Anthropic
56.6%
$3 / $15
116 Qwen3-VL-235B-A22B-ThinkingAlibaba
56.5%
$0.40 / $4
117 Trinity Large ThinkingArcee AI
56.3%
$0.25 / $0.80
118 Claude Opus 4.1 (reasoning on)Anthropic
55.4%
$15 / $75
119 GLM-5 (no reasoning)Z.ai
55.2%
$0.95 / $2.55
120 Gemini 3 Flash Preview (no reasoning)Google
55.1%
$0.50 / $3
121 Claude Sonnet 4 (reasoning on)Anthropic
54.7%
$3 / $15
122 GLM-4.7 (no reasoning)Z.ai
54.6%
$0.54 / $1.98
123 Claude Haiku 4.5 (reasoning on)Anthropic
54.3%
$1 / $5
124 DeepSeek-V3.2-Exp (reasoning on)DeepSeek
54.2%
$0.27 / $0.41
125 Qwen3-Max-Thinking-PreviewAlibaba
53.8%
–
126 Grok 4xAI
53.7%
–
127 Gemma 4 31B (no reasoning)Google
53.5%
$0.14 / $0.40
128 Claude Opus 4.6 (max reasoning)Anthropic
53.1%
$5 / $25
129 Grok 4.1 Fast (reasoning)xAI
52.7%
–
130 Gemini 2.5 Flash-Lite Preview (09-2025, reasoning on)Google
52.6%
–
131 Gemini 2.5 Flash Preview (09-2025, reasoning on)Google
52.3%
–
132 Qwen3.5-4B (reasoning on)Alibaba
52.0%
–
132 GLM-5.1 (no reasoning)Z.ai
52.0%
$1.38 / $4.40
134 Qwen3.5-397B-A17B (no reasoning)Alibaba
51.6%
$0.55 / $3.50
135 Qwen3-235B-A22B-Thinking-2507Alibaba
51.2%
$0.30 / $3
135 Qwen3.5-Omni-PlusAlibaba
51.2%
–
137 Qwen3.5-122B-A10B (no reasoning)Alibaba
50.8%
$0.26 / $2.08
138 Qwen3-30B-A3B-Thinking-2507Alibaba
50.7%
$0.20 / $2.40
139 Grok 4 Fast (reasoning)xAI
50.5%
–
140 Gemini 2.5 Flash (reasoning on)Google
50.3%
$0.30 / $2.50
141 Gemini 2.5 Flash-Lite (reasoning on)Google
49.9%
$0.10 / $0.40
142 Qwen3-4B-Thinking-2507Alibaba
49.8%
–
143 Gemini 3 Pro Preview (low reasoning)Google
49.7%
–
144 Grok 4.20 (no reasoning)xAI
49.3%
$1.25 / $2.50
145 DeepSeek-V3.2 (no reasoning)DeepSeek
49.0%
$0.30 / $0.96
146 Gemini 2.5 ProGoogle
48.7%
$1.25 / $10
147 GPT-5.4 (no reasoning)OpenAI
48.4%
$2.50 / $15
148 Mistral Small 4Mistral AI
48.2%
$0.15 / $0.60
149 Qwen3-Max-PreviewAlibaba
48.0%
–
149 Hy3 Preview (no reasoning)Tencent
48.0%
$0.18 / $0.60
151 Grok 4.20 (0309, non-reasoning, no reasoning)xAI
47.8%
–
152 Grok 4.3 (no reasoning)xAI
47.6%
$1.25 / $2.50
153 GPT-5.2 (no reasoning)OpenAI
47.4%
$1.75 / $14
154 Gemini 3.5 Flash (minimal reasoning)Google
47.3%
$1.50 / $9
155 DeepSeek-V4-Flash (0423, no reasoning)DeepSeek
47.2%
$0.14 / $0.28
156 Llama 3.3 70B InstructMeta
47.1%
$0.59 / $0.79
157 Qwen3.5-27B (no reasoning)Alibaba
46.9%
$0.27 / $2.16
158 GLM-4.7-Flash (no reasoning)Z.ai
46.3%
$0.06 / $0.40
159 GPT-5.5 (no reasoning)OpenAI
46.1%
$5 / $30
159 Qwen3-235B-A22B-Instruct-2507Alibaba
46.1%
$0.15 / $0.75
161 DeepSeek-V4-Pro (0423, no reasoning)DeepSeek
45.8%
$1.42 / $2.83
162 Qwen3.6-27B (no reasoning)Alibaba
45.7%
$0.30 / $3.20
163 GPT-5 mini (minimal reasoning)OpenAI
45.6%
$0.25 / $2
163 GPT-5 (minimal reasoning)OpenAI
45.6%
$1.25 / $10
165 Gemma 4 26B A4B (no reasoning)Google
45.4%
$0.10 / $0.30
165 Claude Sonnet 4 (no reasoning)Anthropic
45.4%
$3 / $15
167 Gemma 4 12B (no reasoning)Google
45.2%
–
168 Qwen3-VL-30B-A3B-ThinkingAlibaba
45.1%
$0.29 / $1
169 GPT-5 ChatOpenAI
45.0%
–
170 Claude Opus 4.6 (no reasoning)Anthropic
44.6%
$5 / $25
171 Qwen3.5-35B-A3B (no reasoning)Alibaba
44.5%
$0.16 / $1.30
172 Magistral Small 1.2Mistral AI
44.4%
–
173 Kimi K2.6 (no reasoning)Moonshot AI
44.3%
$0.95 / $4
174 Gemma 4 E4BGoogle
44.2%
–
174 Qwen3-MaxAlibaba
44.2%
$0.78 / $3.90
176 GLM-4.5Z.ai
44.1%
$0.60 / $2.20
177 Kimi K2.5 (no reasoning)Moonshot AI
43.7%
$0.57 / $2.85
178 Claude Opus 4.7 (no reasoning)Anthropic
43.6%
$5 / $25
179 Gemini 2.5 Flash Preview (09-2025, no reasoning)Google
43.5%
–
180 Qwen3-Omni-30B-A3B-ThinkingAlibaba
43.4%
–
180 GLM-4.6 (reasoning on)Z.ai
43.4%
$0.50 / $2
182 GPT-5.1 (no reasoning)OpenAI
43.2%
$1.25 / $10
183 DeepSeek-V3.2-Exp (no reasoning)DeepSeek
43.1%
$0.27 / $0.41
184 Claude Opus 4.5 (no reasoning)Anthropic
43.0%
$5 / $25
184 Llama 4 MaverickMeta
43.0%
$0.27 / $0.85
184 Magistral Medium 1.2Mistral AI
43.0%
–
184 GPT-4.1OpenAI
43.0%
$2 / $8
188 MiMo-V2.5-Pro (no reasoning)Xiaomi
42.7%
$0.43 / $0.87
188 Qwen3-VL-235B-A22B-InstructAlibaba
42.7%
$0.30 / $1.50
188 Claude Sonnet 4.5 (no reasoning)Anthropic
42.7%
$3 / $15
191 Claude Haiku 4.5 (no reasoning)Anthropic
42.0%
$1 / $5
192 Gemini 2.5 Flash-Lite Preview (09-2025, no reasoning)Google
41.8%
–
193 Kimi K2 (0905)Moonshot AI
41.7%
$0.60 / $2.50
194 Qwen3-30B-A3B (reasoning on)Alibaba
41.5%
$0.12 / $0.50
194 DeepSeek-V3.1 (reasoning on)DeepSeek
41.5%
$0.55 / $1.65
194 Kimi K2 (0711)Moonshot AI
41.5%
$0.57 / $2.30
197 Grok Code Fast 1xAI
41.4%
–
198 DeepSeek-V3.1-Terminus (no reasoning)DeepSeek
41.2%
$0.27 / $1
198 Claude Sonnet 4.6 (no reasoning)Anthropic
41.2%
$3 / $15
200 DeepSeek-V3-0324DeepSeek
41.0%
$0.25 / $1
201 Qwen3-14B (reasoning on)Alibaba
40.5%
$0.12 / $0.24
201 Nova 2 Lite (no reasoning)Amazon
40.5%
$0.30 / $2.50
201 Gemma 4 E4B (no reasoning)Google
40.5%
–
201 Qwen3-Coder-480B-A35BAlibaba
40.5%
$0.35 / $1.50
205 Qwen3-VL-8B-ThinkingAlibaba
39.9%
$0.18 / $2.10
206 Mistral Medium 3.1Mistral AI
39.8%
$0.40 / $2
207 Qwen3-Next-80B-A3B-InstructAlibaba
39.7%
$0.10 / $1.10
208 DeepSeek-R1-0528DeepSeek
39.6%
$0.50 / $2.18
209 Llama 4 ScoutMeta
39.5%
$0.18 / $0.59
210 Mistral Medium 3Mistral AI
39.3%
$0.40 / $2
211 Qwen3-VL-32B-InstructAlibaba
39.2%
$0.10 / $0.42
212 DeepSeek-R1DeepSeek
39.0%
$0.70 / $2.50
212 Gemini 2.5 Flash (no reasoning)Google
39.0%
$0.30 / $2.50
214 GPT-5.4 mini (no reasoning)OpenAI
38.8%
$0.75 / $4.50
215 Qwen3-235B-A22B (reasoning on)Alibaba
38.7%
$0.46 / $1.82
216 GPT-4.1 miniOpenAI
38.3%
$0.40 / $1.60
217 Nova Pro 1.0Amazon
38.1%
$0.80 / $3.20
217 Devstral 2Mistral AI
38.1%
$0.40 / $2
219 Gemma 4 E2BGoogle
38.0%
–
219 Qwen3.5-Omni-FlashAlibaba
38.0%
–
221 Qwen3.5-9B (no reasoning)Alibaba
37.8%
$0.10 / $0.15
221 DeepSeek-V3.1 (no reasoning)DeepSeek
37.8%
$0.55 / $1.65
223 Grok 4 Fast (non-reasoning, no reasoning)xAI
37.7%
–
224 GLM-4.5-AirZ.ai
37.6%
$0.14 / $0.86
225 Nemotron 3 Nano 30B A3B (no reasoning)NVIDIA
37.5%
$0.05 / $0.20
226 Qwen2.5-72B-InstructAlibaba
36.9%
$0.36 / $0.40
227 Gemma 3 12BGoogle
36.7%
$0.05 / $0.15
227 GLM-4.6 (no reasoning)Z.ai
36.7%
$0.50 / $2
229 Qwen3-235B-A22B (no reasoning)Alibaba
36.6%
$0.46 / $1.82
229 Qwen3-VL-4B-ThinkingAlibaba
36.6%
–
231 Command ACohere
36.5%
$2.50 / $10
231 Grok 4.1 Fast (non-reasoning, no reasoning)xAI
36.5%
–
233 Qwen3-32B (reasoning on)Alibaba
36.3%
$0.14 / $0.40
234 Qwen3.6-35B-A3B (no reasoning)Alibaba
36.2%
$0.10 / $1
234 Mistral Large 3Mistral AI
36.2%
$0.50 / $1.50
236 GPT-4o (2024-08-06)OpenAI
36.0%
$2.50 / $10
237 Qwen3-Coder-NextAlibaba
35.2%
$0.18 / $0.90
238 DeepSeek-V3DeepSeek
34.8%
$0.26 / $1.03
239 Devstral Small 1.1Mistral AI
34.6%
–
240 Llama 3.1 70B InstructMeta
34.4%
$0.40 / $0.40
241 GLM-4.5V (reasoning on)Z.ai
34.2%
$0.60 / $1.80
241 Nova Lite 1.0Amazon
34.2%
$0.06 / $0.24
243 Gemma 4 E2B (no reasoning)Google
33.6%
–
244 Qwen3-4B-Instruct-2507Alibaba
33.5%
–
244 Qwen3-8B (reasoning on)Alibaba
33.5%
$0.12 / $0.46
244 Mistral Small 3.2 24BMistral AI
33.5%
$0.094 / $0.25
247 Qwen3.5-4B (no reasoning)Alibaba
33.3%
–
248 Qwen3-VL-30B-A3B-InstructAlibaba
33.1%
$0.15 / $0.60
248 Qwen3-30B-A3B-Instruct-2507Alibaba
33.1%
$0.09 / $0.30
250 Mistral Small 4 (no reasoning)Mistral AI
32.8%
$0.15 / $0.60
251 GPT-5.4 nano (no reasoning)OpenAI
32.7%
$0.20 / $1.25
251 Qwen3-Coder-30B-A3B-InstructAlibaba
32.7%
$0.07 / $0.28
253 GPT-5 nano (minimal reasoning)OpenAI
32.5%
$0.05 / $0.40
254 Qwen3-VL-8B-InstructAlibaba
32.3%
$0.12 / $0.46
255 Ministral 3 14BMistral AI
32.0%
$0.20 / $0.20
255 GPT-4.1 nanoOpenAI
32.0%
$0.10 / $0.40
257 Qwen3-30B-A3B (no reasoning)Alibaba
31.9%
$0.12 / $0.50
258 Qwen3-VL-4B-InstructAlibaba
31.8%
–
258 Gemma 3 27BGoogle
31.8%
$0.12 / $0.20
260 Mistral Large 2 (2407)Mistral AI
31.6%
$2 / $6
261 Qwen3-32B (no reasoning)Alibaba
31.5%
$0.14 / $0.40
261 Qwen3.5-2B (reasoning on)Alibaba
31.5%
–
261 Gemini 2.5 Flash-Lite (no reasoning)Google
31.5%
$0.10 / $0.40
264 Qwen3-Omni-30B-A3B-InstructAlibaba
31.2%
–
264 Devstral Small 2Mistral AI
31.2%
–
266 GPT-4o mini (2024-07-18)OpenAI
31.0%
$0.15 / $0.60
267 Reka Flash 3Reka AI
30.4%
$0.10 / $0.20
268 GLM-4.6V (reasoning on)Z.ai
30.1%
$0.30 / $0.90
269 Devstral MediumMistral AI
29.9%
–
269 Mistral Small 3.1 24BMistral AI
29.9%
$0.35 / $0.56
271 Nova Micro 1.0Amazon
29.4%
$0.035 / $0.14
272 Qwen3.5-2B (no reasoning)Alibaba
29.1%
–
272 Ministral 3 8BMistral AI
29.1%
$0.15 / $0.15
274 GLM-4.5V (no reasoning)Z.ai
28.6%
$0.60 / $1.80
274 Qwen3-8B (no reasoning)Alibaba
28.6%
$0.12 / $0.46
274 Llama 3.1 8B InstructMeta
28.6%
$0.05 / $0.08
277 Gemma 3 4BGoogle
28.3%
$0.05 / $0.10
278 Kimi Linear 48B A3B InstructMoonshot AI
28.1%
–
279 GLM-4.6V (no reasoning)Z.ai
27.9%
$0.30 / $0.90
280 Ministral 3 3BMistral AI
26.8%
$0.10 / $0.10
281 Mistral Small 3Mistral AI
26.4%
$0.05 / $0.08
282 Llama 3.2 3B InstructMeta
26.2%
$0.05 / $0.33
283 Qwen3-14B (no reasoning)Alibaba
23.9%
$0.12 / $0.24
284 Phi-4Microsoft
23.5%
$0.07 / $0.14
285 Llama 3.2 1B InstructMeta
22.8%
$0.027 / $0.20
286 Qwen3.5-0.8B (no reasoning)Alibaba
21.6%
–
287 Qwen3.5-0.8B (reasoning on)Alibaba
21.5%
–
288 Gemma 3 270MGoogle
12.1%
–

Swipe the table sideways for more columns.

Ranks follow the score as shown, so equal numbers share a rank. Results as published by Artificial Analysis; we do not re-run them.

What it measures

Share of precise, checkable formatting instructions followed, as run by Artificial Analysis with its own harness and prompts.

What it does not measure

Not judgement about what an instruction meant.

Source

Artificial Analysis · Methodology

282 results from Artificial Analysis not ranked here · show why

We rank a result only when we can tie it to a specific model you can use. These are left out:

  • Not on sale through the API providers we track: 268
  • A different snapshot or variant from the model we list: 13
  • An unusual combination of settings: 1

Data sourced from Artificial Analysis. Licence: Artificial Analysis commercial data licence.