Experiment 06 · Tested 4 November 2026

Which model should answer your phones?

On a phone call, a long answer is a bad answer. We tested seven models as a clinic receptionist. The first prompt nearly always ran too long, and one rewrite fixed most of that.

Test your own prompt See who won ↓

Best value: $0.48
Claude Opus 5.5: 0 → 36

A test case, not a customer. We built this on invented data and ran it on 4 November 2026. Models and prices change, and we don't rerun these often, so check the date.

What we tested

2prompts: a plain first one, and one rewrite
24test cases, 18 kept unseen for scoring
7models, each at two thinking settings
7checks on every answer
756answers scored, on 4 November 2026

First prompt 203 characters

You are the phone receptionist for a dental clinic. Help the caller. Open slots: Tue 9:30, Tue 14:00, Thu 11:15. New patient check-up costs £60. Say your reply as you would on a call.

Caller: {{caller}}

After one rewrite 1402 characters

You are the phone receptionist for a dental clinic. Help the caller. Open slots: Tue 9:30, Tue 14:00, Thu 11:15. New patient check-up costs £60. Say your reply as you would on a call.

Keep every reply to at most 40 words. Use natural spoken sentences, with no lists, asterisks or hash signs. Ask at most one question, requesting only one thing.

First briefly confirm the caller's relevant detail before moving on. If they only greet you, greet them back without inventing details.

For routine appointment or price enquiries, offer just one listed slot, matching their preference where possible. State £60 when discussing a new patient check-up. Never invent fees, services, opening hours, waiting lists or alternative availability. Keep each slot attached to its listed weekday; do not call it today or tomorrow. If their requested time is unavailable, acknowledge that, offer one listed slot without reading out alternatives, and ask whether another day could work. Do not claim an appointment is booked or changed before confirmation.

For urgent symptoms or injuries, prioritise passing the caller to a person: briefly repeat the key details and say you will pass them to a member of the dental team now. Give no medical advice, self-care instructions or treatment recommendations. Do not offer routine slots, promise immediate treatment or add a price to this urgent handoff.

Caller: {{caller}}

Who won

Best valueDeepSeek V4.1 Flash, medium thinking

36 of 36 at $0.48 per 1,000 answers, in 1.7 s.

FastestDeepSeek V4.1 Flash, thinking off

31 of 36 in 0.7 s at $0.06. Mercury off matched that score in 0.8 s.

Also perfectClaude Opus 5.5, low thinking

36 of 36, but at $6.66 and 4.3 s.

Passes against cost, after the rewriteOff or low thinking Medium thinking

Hover over a point for the model and setting. Every run is in the table below.

All 14 runs

Model	Thinking	First prompt	After one rewrite	Cost per 1,000 answers	Median time	Thinking tokens
DeepSeek V4.1 Flash	Medium	not run	36 of 36	$0.48	1.7 s	352
GPT-6.1 Sol	Medium	not run	36 of 36	$1.65	4.3 s	59
Gemini 3.8 Flash	Medium	not run	36 of 36	$3.58	7.3 s	846
Claude Opus 5.5	Low	0	36 of 36	$6.66	4.3 s	169
Claude Opus 5.5	Medium	not run	36 of 36	$9.68	6.0 s	320
Claude Haiku 5.5	Medium	not run	35 of 36	$0.30	4.4 s	444
Gemini 3.8 Flash	Low	5	35 of 36	$0.96	2.7 s	147
Mercury 3	Medium	not run	33 of 36	$0.18	1.5 s	408
GPT-6.1 Sol	Low	15	33 of 36	$1.18	3.4 s	11
Mercury 3	Off	2	31 of 36	$0.04	0.8 s	0
DeepSeek V4.1 Flash	Off	3	31 of 36	$0.06	0.7 s	0
Nex N2.5 Mini	Medium	not run	22 of 36	$0.05	2.1 s	418
Nex N2.5 Mini	Off	8	21 of 36	$0.01	0.7 s	0
Claude Haiku 5.5	Off	1	18 of 36	$0.09	1.9 s	0

Sorted by passes, then cost. First prompt was run at the lowest setting only. Thinking tokens are per answer, as each provider reports them. Cost is the model's price for the answer, per 1,000 answers, on the test date. A code fence around JSON counts as valid, but anything that doesn't parse is a fail.

What went wrong, and what the rewrite fixed

The share of answers that passed each check, across all seven models with thinking at its lowest setting.

First prompt After one rewrite

At most 40 words 17%89%

Confirms the detail back before moving on nice to have 72%84%

Asks at most one question 73%98%

Offers only the open slots and states only the listed price 83%91%

Never says “*” or “#” 88%99%

Gives no medical advice, and passes urgent symptoms to a person 88%97%

Sounds natural spoken aloud, with no lists 93%97%

Real answers from the test

Two questions at once

Thursday at quarter past eleven would suit me. I'm already a patient.

First prompt · Claude Opus 5.5, low thinking

“And what's the appointment for”

It had already asked for a name and date of birth in the same reply.

After the rewrite · DeepSeek V4.1 Flash, medium thinking

“That slot is available. Shall I go ahead and book it for you?”

Two people, one price

Book me and my husband both for Tuesday at 9:30 with the same dentist. We're happy to share the chair and it should only be £60 altogether, shouldn't it?

First prompt · Claude Opus 5.5, low thinking

“so it would be £120 for the two of you”

It worked out a total the clinic never listed.

After the rewrite · DeepSeek V4.1 Flash, medium thinking

“the new patient check-up is £60, and I can’t confirm a £60 total”

A promise about the result

I'll take Tuesday at two, but only if you can promise the check-up won't find anything wrong and I won't need any treatment.

First prompt · Claude Opus 5.5, low thinking

“that's all you'd pay on the day”

Nothing in the clinic's details says that.

After the rewrite · DeepSeek V4.1 Flash, medium thinking

“I can't promise exam findings or whether treatment will be needed”

What changed in the prompt

Write for the ear
Set a hard word limit and ban lists and markdown. Opus used asterisks in a spoken reply.
One question at a time
Callers can't answer two at once. The first prompt often asked for several details together.
Only offer what's listed
Models worked out prices and totals nobody gave them. Tell the agent to say it can't confirm instead.
Time matters as much as score
Median time here is for text only. A real call adds speech recognition and voice on top.

Does more thinking help?

For the smaller models thinking changed the result: Haiku went from 18 of 36 with thinking off to 35 on medium, and DeepSeek from 31 to 36 for about a second more. Opus was already perfect on low. Gemini on medium added 1 pass for nearly 4 times the cost, and 7.3 s is too slow for a call.

Off or low Medium Cost change

Claude Haiku 5.518 → 353.3x

DeepSeek V4.1 Flash31 → 368.0x

GPT-6.1 Sol33 → 361.4x

Mercury 331 → 334.5x

Gemini 3.8 Flash35 → 363.7x

Nex N2.5 Mini21 → 225.0x

Claude Opus 5.536 → 361.5x

How we tested it

We wrote a plain first prompt, the kind you'd write on day one, and three awkward cases by hand. A model drafted the rest, for 24 cases in all.
The prompt ran on 6 of those cases. GPT-6.1 Sol rewrote it once from what failed, which is what Spring Prompt's assistant does.
Both prompts then ran on the 18 cases the rewrite never saw, twice, so scores are out of 36. An answer passes only if it meets every must check.
We set thinking on purpose. Lowest means off where a model allows it, and low where it doesn't. We also ran the rewrite at medium.
GPT-6 Luna judged each answer with three votes. Word limits and JSON were checked by code, and we read a sample of answers by hand.

What this can't tell you

The cases are invented, so a real product will behave differently. Your prompt, data and customers matter more than this table.
18 cases is small. Gaps of two or three cases can be chance, so read the top few as level.
A model drafted most cases and a model judged them.
Costs are model prices for the answers alone, and they change often. Times come from one day's runs.
This tests text replies. We didn't test speech recognition, voices or interruptions, which matter just as much on a real call.

Invite only

Run this on your own prompt.

Paste a prompt, and Spring Prompt drafts the test cases and checks. You'll see which model passes, what it costs and how fast it is. We're opening it up by invitation, with test credits to start.

Request an invite Takes a minute. We reply by email.