DeepSeek-V4.1-Flash-1017 review: better at analysis, much worse at sticking to the product data

DeepSeek released a new snapshot of V4.1 Flash on 17 October, at the same price as the one it replaces, and said it was better on benchmarks, faster and shorter-winded. On our tests the first two are partly true. It is much better at planning and analysing surveys and at turning analysis into slide decks, and it answers in less than half the time.

The cost is on product data. It added three times as many claims the source data didn't support, and its share of reliably publishable product listings halved. We ran six of our benchmarks on 19 October through DeepSeek's own API, at the default thinking setting. Here is what we found.

DeepSeek-V4.1-Flash-1017 at a glance

• Use it for analysis and reporting drafts at very low cost, and for coding and agent work, where third-party results are strong.
• Don't use it for product content or anything customer-facing that must stick to supplied facts. The September Flash was twice as reliable, and GPT-6 Luna is better again for less.
• Price: $0.30/$1.20 per million input/output tokens, unchanged; half price off-peak. Open weights, MIT licence.
• Speed: about 2.6 times quicker per product listing than the September model, but still too slow per move for our speed chess clocks.

What DeepSeek says

DeepSeek's release note calls 1017 "a Pareto improvement in performance and cost", with stronger benchmarks, faster task completion and shorter outputs than the September model. It was trained further with reinforcement learning to build whole projects, from games to 3D scenes. The weights are on Hugging Face under the MIT licence: a mixture-of-experts model with about 552 billion parameters, 16 billion active per token, and a context of a million tokens.

Its comparison table shows gains over the September Flash on every row, the largest on agent and coding tests: Terminal-Bench 4.0 from 31.2 to 42.3, ProgramBench from 20.3 to 29.1 and Apex Bench from 64.6 to 72.1. It leads 4 of 11 rows. The rivals in the table are Claude Opus 5 and GPT-5.6 Sol, not the current Opus 5.5 and GPT-6.1 Sol. These are the developer's claims, mostly at maximum reasoning.

DeepSeek-V4.1-Flash-1017 benchmarks, measured by Spring Prompt

The model page has every result as it lands. How to read this table: each row is one benchmark, with the model's score, its rank among the models we have tested on it, and the best model in each group.

Scroll sideways for all 6 columns →

Ranked on the release we publish with this post. A group only counts models that benchmark has measured, and groups of fewer than three are not ranked. "Best" is the top model in that group, not necessarily one you would choose instead: see the alternatives. The price group starts at $0.25, so GPT-6 Luna and Claude Haiku 5.5 ($0.20 blended) are not in it.

Product data: twice as quick, half as reliable

CatalogBench gives a model 56 invented products, with source data and images, and asks for a listing that could go live with no human fix. We run each product three times. With a factual brief, 1017 passed 40.5% of listings and was right in all three repeats for 21.4% of products. The September Flash scored 63.7% and 41.1%. Its extraction was as accurate as before (95.6% of fields right); the difference is in the copy. It added about 95 claims per run that the product data didn't support, against 30 for the September model.

With a sales brief it was worse: 224 unsupported claims per run and not one product right in all three repeats, against 23.2% for the September model. In return it is much quicker and cheaper: 6.7 seconds per listing against 17.6, and $0.26 for 168 listings against $1.02. If your product content goes live without review, that trade is not worth it.

Surveys and slide decks: a real step up

On SurveyBench, which asks a model to plan a survey and then analyse supplied responses without falling for traps in the data, 1017 scored 89.9 out of 100, and 4 of its 12 tasks had no faults in either the plan or the analysis. In our unpublished run of the September Flash, that was 80.8 and none. Judged head to head with other models on the same tasks, it rated 1,214, 5th of 11, just behind Claude Haiku 5.5 (1,261) and well ahead of Gemini 4 Argon (994). It finished all 12 tasks in about three minutes, for $0.43.

On DeckBench, its eight slide decks traced 97% of numbers back to the source, but only one was judged ready to present: six failed on layout. Head to head it rated 1,154, 12th of 22, close to Haiku 5.5 (1,202) and ahead of Argon. Our unpublished September run rated far lower.

Ad budgets and chess against the clock

In ROASBench, a 12-month simulation of a skincare brand's ad budget, it scored 41.4, 11th of 22, a little above our unpublished September run (39.1). It lost money in one month and kept resetting the ad platforms' learning with abrupt budget changes. GPT-6 Luna scored 53.4 and Gemini 3.8 Flash 51.4. This is one simulated run per model, not evidence of real advertising returns.

BulletBench plays speed chess against a ladder of engines, so slow thinking loses on time. At the default setting 1017 lost its first four games on time at all three clocks, as the September model did. Its moves were quicker (a median 4.7 seconds in three-minute games, against 10.3), but long thinks on some moves still ran its clock down. At low effort with one minute a game it got through the screen, won one of eight games and lost seven on time, for a rating of 147. Gemini 3.8 Flash at low thinking leads that clock on 1,068. This measures decisions against a clock, not general intelligence.

In the playground

We also asked it every one of our 28 playground prompts. These are one-off tests, not a benchmark, but two answers stood out. Its cassette recorder works fully: the keys latch, the tape winds between the reels and stops at the end, and the cassette ejects and flips to side B. None of the other fast, cheap models on that page got all of that working. Its spinning hexagon fails: a sign error in the wall maths throws the ball out within a fraction of a second, where the September Flash got it right.

Third-party results

Third-party figures so far are at maximum reasoning, not the API default.

• Artificial Analysis (our fetch, 19 October): 45.1 on the Intelligence Index, up from 39.5 for the September Flash, which moves it from 29th to 16th of the 201 models available today. Its model page reports that it used about 16% fewer tokens to run the index and cost about half as much per task. Data sourced from Artificial Analysis.
• Arena (reported 18 October): the top open model in Code Arena's WebDev board, 12th overall with 1,670, 52 points above the September Flash. Its text rating is not out yet.

Pending: Arena's text and factuality ratings, tau2-bench and Vectara's hallucination leaderboard. Until Arena rates it, its overall score on our leaderboard is provisional.

What people are saying

A selection, with how we read each one:

DeepSeek-V4.1-Flash-1017 vs Claude Haiku 5.5, GPT-6 Luna and Gemini 3.8 Flash

These are the models people will weigh it against: its predecessor, the two cheapest strong models from the big labs, and the fastest-thinking Flash model in our chess tests. Prices are blended per million tokens (three input to one output) at list price.

Scroll sideways for all 7 columns →

For business writing and product data, GPT-6 Luna is the better choice at well under half the price: it beat 1017 on product listings, slide decks and ad budgets, and stayed far closer to the facts with a sales brief. Claude Haiku 5.5 is level with it overall and a little better at surveys and decks. 1017's case is that it is open-weight, scores higher than both on Artificial Analysis's index, and has strong third-party results on coding and agents, which our benchmarks don't measure.

Is DeepSeek-V4.1-Flash-1017 good? Who should use it

• Coding and agent work on a budget, especially if you want open weights or to run it yourself. Third-party results there are strong, though several testers report long jobs taking hours.
• Analysis drafts such as survey write-ups, where it improved most and is very cheap.
• Not for product content without a fact check. It made three times as many unsupported claims as its predecessor, and with a sales brief no product was right every time.
• Not for anything that must answer within a second or two. It still thinks for several seconds before most answers.
• If you call deepseek-flash, you are already on it. DeepSeek moved the alias to 1017 on 17 October; check your product copy now.

These benchmarks aren't your data, so test it on your own prompts before switching. Its 28 playground answers sit beside other models' answers to the same prompts.

What we'll update

We'll publish a week-one verdict once Arena has rated its writing and factuality and tau2-bench and Vectara have results. Until then, the DeepSeek-V4.1-Flash-1017 model page has every result as it lands.

Choosing or switching the model behind a product feature? Our custom evaluations test the models you are weighing on your own prompts and data, against your own definition of good. If AI writes your product content, the catalogue feed diagnostic measures how far you can trust it. Talk to us, or email hello@springprompt.com.

Quick answers