Models / Qwen3-4B-Instruct-2507
Qwen3-4B-Instruct-2507
Qwen3-4B-Instruct-2507 trails most of the field on what it has been measured on; it is not on sale through OpenRouter, so it is here for reference.
Results 16 Nov 2026 · 26 results · 3 sources
- Price per million tokens
- Not available through OpenRouter
- Speed
- Not measured
- Intelligence Index
- 6.7
Artificial Analysis - Spring Prompt overall
- Not ranked yet
needs our benchmarks and two groups of results - Developer
- Alibaba, based in China
Weights not published - Released
- 6 Aug 2025 (Artificial Analysis)
Where it stands among the models you could choose
Available models grouped by price, intelligence, speed and developer location. Retired models are excluded.
| Among | Intelligence Index | Spring Prompt overall | Price, blended | Speed |
|---|---|---|---|---|
| Models available today | 167th of 186Claude Opus 5.5 | – | – | – |
| Models from developers based in China | 74th of 76MiMo-V2.6-Pro | – | – | – |
The name under each rank is the leader of that group; green is the top quarter of the group and red the bottom quarter, by rank. Price is shown as a percentile: the share of the group that costs less, so lower is cheaper. "Sets this group" marks the measure the group is defined by. Spring Prompt overall is our score out of 100 across every source (how it works). Price is the developer's list price (or the typical OpenRouter host where we haven't read one) for three input tokens to one output; intelligence and speed are from Artificial Analysis (data sourced from Artificial Analysis); "similar intelligence" means within 4 points on its Intelligence Index. Developer locations are where each company is based, not where a model is served. Groups under 3 models are not ranked.
How good is it, and for what?
Each use case ranks the available models its benchmarks measured. The bar shows its percentile in that field, best to the right. Open a row for the results behind it.
Product listingsTurning a sparse product feed and photos into listings that can go live
- CatalogBench: has not measured this model.
Decks from an analysisTurning a finished analysis into a deck you could present as it is
- DeckBench: has not measured this model.
User surveysPlanning a user survey and reading its results without being misled
- SurveyBench: has not measured this model.
Marketing planningPlanning a year of ad spend without overspending
- ROASBench: has not measured this model.
Agents and tool useMulti-step tasks with tools: support desks, coding agents, function calls
| Benchmark | Qwen3-4B-Instruct-2507 | Rank among available | Best available |
|---|---|---|---|
| Berkeley Function Calling Leaderboard (BFCL) V4BFCL: overall accuracy · reported | 35.7% | 22nd of 37 | Claude Opus 4.5 77.5% |
- tau2-bench: has not measured this model.
- OpenHands Index: has not measured this model.
- Microsoft STATE-Bench: has not measured this model.
- Artificial Analysis: has not measured this model.
- Vending-Bench 2: has not measured this model.
Professional workReal tasks from banking, consulting and law, business documents and freelance projects
- APEX-Agents: has not measured this model.
- GDP.pdf: has not measured this model.
- Remote Labor Index: has not measured this model.
Reasoning and knowledgeHard questions across science, maths and general knowledge
| Benchmark | Qwen3-4B-Instruct-2507 | Rank among available | Best available |
|---|---|---|---|
| Artificial AnalysisArtificial Analysis: Artificial Analysis Intelligence Index · reported | 6.7 | 167th of 186 | Claude Opus 5.5 57.6 |
| Artificial AnalysisArtificial Analysis: Humanity's Last Exam · reported | 4.5% | 156th of 187 | Claude Opus 5.5 61.4% |
| Artificial AnalysisArtificial Analysis: GPQA Diamond · reported | 51.7% | 145th of 169 | GPT-6 Astra 96.3% |
WritingWhat people prefer in blind comparisons, and judged writing quality
| Benchmark | Qwen3-4B-Instruct-2507 | Rank among available | Best available |
|---|---|---|---|
| UGI LeaderboardUGI: writing score · reported | 29.9 | 105th of 125 | Gemini 3.8 Flash 78.6 |
- Arena (formerly LMArena): has not measured this model.
Sticking to the factsSummarising without inventing things, and factual answers
- Vectara Hallucination Leaderboard: has not measured this model.
- Arena (formerly LMArena): has not measured this model.
- SimpleQA Verified (Epoch AI): has not measured this model.
Speed under pressureGood decisions against a real clock (fast chess)
- BulletBench: has not measured this model.
Nearest alternatives
- Alibaba's newer, at least as capable modelQwen3.8-Max (0902)$3.00 blended · Intelligence Index 45.4
- Open weights at similar intelligenceGemma 3 4B$0.04 blended · Intelligence Index 4.8Compare with Qwen3-4B-Instruct-2507 →
Against Qwen3-4B-Instruct-2507, Intelligence Index 6.7. "Similar intelligence" means within 4 points or better.
Against the alternatives
The models you would most likely weigh it against: the leaders of the groups above.
Scroll sideways to see every alternative.
| Qwen3-4B-Instruct-2507 | Claude Opus 5.5 most intelligent available today | MiMo-V2.6-Pro most intelligent from China | Claude Fable 5.2 leads overall | Qwen4-Max Alibaba's best other model | |
|---|---|---|---|---|---|
| Price per million tokens | – | $8.00 | $0.54 | $16.00 | $2.25 |
| Tokens a second | – | 96 | 43 | – | – |
| Intelligence Index | 6.7 | 57.6 | 46.3 | – | – |
| Spring Prompt overall | – | 82 | – | 90 | 56 |
| Rank among available models, by use case | |||||
| Product listings | – | 13th | – | 5th | 19th |
| Decks from an analysis | – | 7th | – | 4th | 1st |
| User surveys | – | 4th | – | 1st | 5th |
| Marketing planning | – | 9th | – | 1st | 15th |
| Agents and tool use | 93rd of 160 | 3rd | 23rd | – | – |
| Professional work | – | 3rd | – | – | – |
| Reasoning and knowledge | 159th of 188 | 1st | 13th | – | – |
| Writing | 143rd of 170 | 5th | 23rd | – | – |
| Sticking to the facts | – | 3rd | 10th | – | – |
| Speed under pressure | – | – | – | 8th | 26th |
| Side by side → | |||||
Shaded figures are better than Qwen3-4B-Instruct-2507's. A dash means no result.
Where it has been measured
Compared with the models available today (it is not on sale itself), in the bottom quarter for reasoning and knowledge, and writing.
Artificial Analysis lists it but has not published a speed for it yet
- Artificial Analysis7 results
- Berkeley Function Calling Leaderboard (BFCL) V416 results
- UGI Leaderboard3 results
- BulletBenchNot measured
- CatalogBenchNot measured
- DeckBenchNot measured
- ROASBenchNot measured
- SurveyBenchNot measured
- APEX-AgentsNot measured
- Arena (formerly LMArena)Not measured
- GDP.pdfNot measured
- Microsoft STATE-BenchNot measured
- OpenHands IndexNot measured
- Remote Labor IndexNot measured
- SimpleQA Verified (Epoch AI)Not measured
- tau2-benchNot measured
- Vectara Hallucination LeaderboardNot measured
- Vending-Bench 2Not measured
Every result
26 published results from 3 sources, each in the source's own units, with the configuration that produced it.
Artificial Analysis · reported by Artificial Analysis · 7 results
| Measure | Value | Rank | Configuration | Dated |
|---|---|---|---|---|
| Artificial Analysis: Artificial Analysis Intelligence Indexindex score, higher is better | 6.7 | 381 of 413 | Qwen3-4B-Instruct-2507 | 20 Oct 2026 |
| Artificial Analysis: GPQA Diamond% of questions, higher is better | 51.7% | 317 of 360 | Qwen3-4B-Instruct-2507 | 20 Oct 2026 |
| Artificial Analysis: Humanity's Last Exam% of questions, higher is better | 4.5% | 362 of 419 | Qwen3-4B-Instruct-2507 | 20 Oct 2026 |
| Artificial Analysis: IFBench% of instructions, higher is better | 33.5% | 244 of 288 | Qwen3-4B-Instruct-2507 | 20 Oct 2026 |
| Artificial Analysis: Long-context reasoning (AA-LCR)% of questions, higher is better | 11.3% | 380 of 407 | Qwen3-4B-Instruct-2507 | 20 Oct 2026 |
| Artificial Analysis: Terminal-Bench Hard% of tasks, higher is better | 4.5% | 237 of 282 | Qwen3-4B-Instruct-2507 | 20 Oct 2026 |
| Artificial Analysis: τ²-bench telecom% of tasks, higher is better | 26.6% | 227 of 286 | Qwen3-4B-Instruct-2507 | 20 Oct 2026 |
Not shown: Not business work, and a blend: read the parts for any one task.
Berkeley Function Calling Leaderboard (BFCL) V4 · reported by Berkeley Function Calling Leaderboard (BFCL) V4 · 16 results
| Measure | Value | Rank | Configuration | Dated |
|---|---|---|---|---|
| BFCL: irrelevance detection% correct, higher is better | 84.9% | 34 of 109 | Qwen3-4B-Instruct-2507 (bfcl fc) | 16 Dec 2025 |
| BFCL: irrelevance detection% correct, higher is better | 75.9% | 69 of 109 | Qwen3-4B-Instruct-2507 (bfcl prompt) | 16 Dec 2025 |
| BFCL: memory% correct, higher is better | 17.6% | 51 of 109 | Qwen3-4B-Instruct-2507 (bfcl fc) | 16 Dec 2025 |
| BFCL: memory% correct, higher is better | 23.9% | 35 of 109 | Qwen3-4B-Instruct-2507 (bfcl prompt) | 16 Dec 2025 |
| BFCL: multi-turn tasks% correct, higher is better | 22.1% | 49 of 109 | Qwen3-4B-Instruct-2507 (bfcl fc) | 16 Dec 2025 |
| BFCL: multi-turn tasks% correct, higher is better | 20.5% | 51 of 109 | Qwen3-4B-Instruct-2507 (bfcl prompt) | 16 Dec 2025 |
| BFCL: overall accuracy% correct, higher is better | 35.7% | 54 of 109 | Qwen3-4B-Instruct-2507 (bfcl fc) | 16 Dec 2025 |
| BFCL: overall accuracy% correct, higher is better | 35.5% | 55 of 109 | Qwen3-4B-Instruct-2507 (bfcl prompt) | 16 Dec 2025 |
| BFCL: relevance detection% correct, higher is better | 87.5% | 27 of 109 | Qwen3-4B-Instruct-2507 (bfcl fc) | 16 Dec 2025 |
| BFCL: relevance detection% correct, higher is better | 87.5% | 27 of 109 | Qwen3-4B-Instruct-2507 (bfcl prompt) | 16 Dec 2025 |
| BFCL: single-turn calls (curated)% correct, higher is better | 87.9% | 23 of 109 | Qwen3-4B-Instruct-2507 (bfcl fc) | 16 Dec 2025 |
| BFCL: single-turn calls (curated)% correct, higher is better | 86.4% | 35 of 109 | Qwen3-4B-Instruct-2507 (bfcl prompt) | 16 Dec 2025 |
| BFCL: single-turn calls (user-contributed)% correct, higher is better | 76.4% | 34 of 109 | Qwen3-4B-Instruct-2507 (bfcl fc) | 16 Dec 2025 |
| BFCL: single-turn calls (user-contributed)% correct, higher is better | 74.7% | 41 of 109 | Qwen3-4B-Instruct-2507 (bfcl prompt) | 16 Dec 2025 |
| BFCL: web search% correct, higher is better | 3.0% | 69 of 109 | Qwen3-4B-Instruct-2507 (bfcl fc) | 16 Dec 2025 |
| BFCL: web search% correct, higher is better | 4.5% | 65 of 109 | Qwen3-4B-Instruct-2507 (bfcl prompt) | 16 Dec 2025 |
Not shown: Not general refusal or safety behaviour.
UGI Leaderboard · reported by UGI Leaderboard · 3 results
| Measure | Value | Rank | Configuration | Dated |
|---|---|---|---|---|
| UGI: requested-length error% off the requested word count, lower is better | 8.0% | 63 of 379 | Qwen3-4B-Instruct-2507 | 16 Oct 2025 |
| UGI: style adherencescore from 0 to 1, higher is better | 0.35 | 210 of 379 | Qwen3-4B-Instruct-2507 | 16 Oct 2025 |
| UGI: writing scorescore out of 100, higher is better | 29.9 | 281 of 379 | Qwen3-4B-Instruct-2507 | 16 Oct 2025 |
Not shown: Not other format limits such as character counts or bullet counts.
Test your own prompt.
Benchmarks aren't your task. Run your prompt on the models you could use, with every answer checked and costed.