Experiment 03 · Tested 4 November 2026

Which model writes meeting notes you can trust?

The risk in meeting notes isn't a clumsy summary. It's a decision nobody made, or a task with an owner who never agreed. We tested seven models on invented meetings, where people change their minds.

Test your own prompt See who won ↓

Best value: $0.38
Mercury 3: 11 → 33

A test case, not a customer. We built this on invented data and ran it on 4 November 2026. Models and prices change, and we don't rerun these often, so check the date.

What we tested

2prompts: a plain first one, and one rewrite
24test cases, 18 kept unseen for scoring
7models, each at two thinking settings
6checks on every answer
756answers scored, on 4 November 2026

First prompt 173 characters

Summarise this meeting transcript. Return JSON with the keys summary, decisions and action_items, where each action item has task, owner and due.

Transcript: {{transcript}}

After one rewrite 1439 characters

Summarise this meeting transcript. Return only valid JSON, without markdown or other text, with the keys summary, decisions and action_items. Use a string for summary and arrays for decisions and action_items, where each action item has task, owner and due. Use empty arrays when there are no decisions or action items, even if the transcript is incomplete or contains only small talk.

List as decisions only outcomes explicitly agreed in the transcript. Do not list suggestions, possibilities, conditional offers, unsupported statements of agreement, or the absence of an agreement or assignment as decisions. Put relevant context about unresolved matters in the summary instead.

Include every action item someone clearly agreed to do, but do not turn suggestions, unmet needs or unaccepted assignments into action items. Name an owner only if the transcript explicitly identifies one; otherwise use an empty string, not "Unassigned". Give a due date only if the transcript explicitly states it for that task; otherwise use an empty string. Preserve the stated deadline without adding inferred days or borrowing deadlines from related tasks. Prerequisites and conditions are not due dates.

Reflect the final position when people revise or withdraw earlier statements. Do not carry forward superseded tasks, conditions or commitments, or attach a condition to a task unless the transcript explicitly does so.

Transcript: {{transcript}}

Who won

Best valueClaude Haiku 5.5, medium thinking

33 of 36 at $0.38 per 1,000 answers, at a median of 5.0 s.

Best qualityGPT-6.1 Sol, medium thinking

35 of 36 at $2.54. Opus on low also got 35, at $9.14.

Fastest worth usingMercury 3, medium thinking

33 of 36 in 2.9 s at $0.48.

Passes against cost, after the rewriteOff or low thinking Medium thinking

Hover over a point for the model and setting. Every run is in the table below.

All 14 runs

Model	Thinking	First prompt	After one rewrite	Cost per 1,000 answers	Median time	Thinking tokens
GPT-6.1 Sol	Medium	not run	35 of 36	$2.54	4.6 s	28
Claude Opus 5.5	Low	21	35 of 36	$9.14	4.9 s	59
DeepSeek V4.1 Flash	Medium	not run	34 of 36	$0.55	6.2 s	871
Claude Haiku 5.5	Medium	not run	33 of 36	$0.38	5.0 s	368
Mercury 3	Medium	not run	33 of 36	$0.48	2.9 s	1191
GPT-6.1 Sol	Low	25	32 of 36	$2.24	4.5 s	6
Gemini 3.8 Flash	Medium	not run	32 of 36	$4.01	7.7 s	862
Claude Opus 5.5	Medium	not run	32 of 36	$14.04	6.6 s	294
Gemini 3.8 Flash	Low	21	31 of 36	$0.83	3.0 s	6
Claude Haiku 5.5	Off	18	30 of 36	$0.18	2.3 s	0
Mercury 3	Off	11	29 of 36	$0.09	1.0 s	0
DeepSeek V4.1 Flash	Off	24	29 of 36	$0.11	1.3 s	0
Nex N2.5 Mini	Medium	not run	22 of 36	$0.09	4.1 s	796
Nex N2.5 Mini	Off	15	13 of 36	$0.02	1.7 s	0

Sorted by passes, then cost. First prompt was run at the lowest setting only. Thinking tokens are per answer, as each provider reports them. Cost is the model's price for the answer, per 1,000 answers, on the test date. A code fence around JSON counts as valid, but anything that doesn't parse is a fail.

What went wrong, and what the rewrite fixed

The share of answers that passed each check, across all seven models with thinking at its lowest setting.

First prompt After one rewrite

Lists as decisions only things the group actually decided, not ideas raised 77%88%

Names an owner only if the transcript names one, and otherwise leaves it empty 83%98%

Gives a due date only if the transcript states one 90%97%

Valid JSON with summary, decisions, action_items 95%98%

Misses no action item someone clearly agreed to do 97%93%

Reflects the final position when people changed their minds 100%99%

Real answers from the test

A recommendation, recorded as a decision

A hiring panel debrief where the chair recommends a practical assessment and Louis prefers another interview for Candidate B.

First prompt · Claude Opus 5.5, low thinking

“Both candidates will be invited to a practical assessment”

The judge found this listed as a decision when it wasn't one.

After the rewrite · Claude Haiku 5.5, medium thinking

“The chair recommended inviting both to a practical assessment.”

An owner nobody could hear

A noisy call where someone offers to report a flickering light, and nobody can tell whether it was Imran or Callum.

First prompt · Claude Opus 5.5, low thinking

“Unconfirmed (Imran or Callum)”

It wrote a guess into the owner field, where our rule asks for an empty value.

After the rewrite · Claude Haiku 5.5, medium thinking

“It was unclear whether that speaker was Imran or Callum”

An idea, not a task

Ana: Should we move launch to March? Ben: Maybe. Ana: Let's think about it. Ben: Someone should check with legal.

First prompt · Claude Opus 5.5, low thinking

“Check with legal about moving the launch to March”

It added a task nobody accepted, inside a code block that a parser can't read.

After the rewrite · Claude Haiku 5.5, medium thinking

“no one agreed to do it and no owner or due date was set”

What changed in the prompt

Define a decision
Say a decision is something explicitly agreed. Models listed ideas and clarifications as decisions.
Say what to do with no owner or date
Use an empty value, not a guess. Models filled these in with "Unassigned" or a likely date.
Later statements win
Say that the final position replaces earlier ones, and that a prerequisite isn't a due date.
Ask for raw JSON
Opus wrapped its JSON in Markdown fences and added a note after it, which breaks a parser.

Does more thinking help?

Medium thinking lifted most models, by 3 to 5 cases for Haiku, Mercury, DeepSeek and Sol, and it cost Mercury and DeepSeek about 5 times as much. Nex rose to 22 and stayed the weakest. Opus went the other way, from 35 on low to 32 on medium, at a higher price. Haiku on medium came within 2 of Opus on low at about a twentieth of the cost.

Off or low Medium Cost change

Nex N2.5 Mini13 → 224.5x

DeepSeek V4.1 Flash29 → 345.0x

Mercury 329 → 335.3x

GPT-6.1 Sol32 → 351.1x

Claude Haiku 5.530 → 332.1x

Gemini 3.8 Flash31 → 324.8x

Claude Opus 5.535 → 321.5x

How we tested it

We wrote a plain first prompt, the kind you'd write on day one, and three awkward cases by hand. A model drafted the rest, for 24 cases in all.
The prompt ran on 6 of those cases. GPT-6.1 Sol rewrote it once from what failed, which is what Spring Prompt's assistant does.
Both prompts then ran on the 18 cases the rewrite never saw, twice, so scores are out of 36. An answer passes only if it meets every must check.
We set thinking on purpose. Lowest means off where a model allows it, and low where it doesn't. We also ran the rewrite at medium.
GPT-6 Luna judged each answer with three votes. Word limits and JSON were checked by code, and we read a sample of answers by hand.

What this can't tell you

The cases are invented, so a real product will behave differently. Your prompt, data and customers matter more than this table.
18 cases is small. Gaps of two or three cases can be chance, so read the top few as level.
A model drafted most cases and a model judged them.
Costs are model prices for the answers alone, and they change often. Times come from one day's runs.

Invite only

Run this on your own prompt.

Paste a prompt, and Spring Prompt drafts the test cases and checks. You'll see which model passes, what it costs and how fast it is. We're opening it up by invitation, with test credits to start.

Request an invite Takes a minute. We reply by email.