Benchmarks / SurveyBench / How Ashgrove patients find online-first booking / Claude Fable 5.2

Measured by Spring Prompt

Claude Fable 5.2: How Ashgrove patients find online-first booking

The decision: whether to keep online-first booking as it is, restore a morning phone booking line for every patient, or add a dedicated phone line and in-person help at reception only for patients who need them (over-75s, patients with a disability, and patients with limited English). The brief and the data →

Verdict: ✓ Sound research
Research score: 100 of 100
Analysis rating: 1,683
Head to head, this task: won 8 of 9

Scorecard

Numbers 11 of 11 right

Question		Its answer	True
Among respondents who tried to book in the last three months, what percentage were satisfied (fairly or very) with the booking process?	✓	50.8%	50.8%
Weighting each age band's percentage satisfied (fairly or very, among those who tried to book) by that band's share of registered adults, what is the overall percentage satisfied?	✓	46.3%	46.3%
What percentage of all respondents said there was a time in the last three months when they needed an appointment but did not try to book one?	✓	24.3%	24.3%
Weighted to the age mix of registered adults, what percentage would prefer to book by phone?	✓	45%	45%
What is the 95% margin of error, in percentage points, on the unweighted percentage satisfied among respondents who tried to book?	✓	4.2 points	4.18 points
Among respondents who tried to book, is the difference in the percentage satisfied (fairly or very) between female and male respondents statistically significant at the 95% level? (1 for yes, 0 for no)	✓	No	No
How many of the open-text answers ask to be able to book by phone, or for a phone booking line?	✓	97	97
How many of the open-text answers from respondents aged 65 or over ask to be able to book by phone, or for a phone booking line?	✓	13	13
Weighted to the age mix of registered adults, about how many of the practice's 17,100 registered adults would say there was a time in the last three months when they needed an appointment but did not try to book one?	✓	4,392	4,393
Among respondents aged 18 to 44, what percentage would prefer to book using the online request form?	✓	58.4%	58.4%
What percentage of the patients sent the text message completed the survey?	✓	9%	9%

Traps in the data 7 of 7 handled

✓Patients aged 65 or over are 24% of registered adults but 9.8% of respondents (text link, online only). They are less satisfied and prefer the phone, so weighting to the age mix lowers satisfaction among those who tried to book from 50.8% to 46.3% and raises the phone preference from 40.7% to 45.0%.
✓Q2 satisfaction (50.8%) is asked only of the 549 who tried to book. 24.3% of all respondents needed an appointment but did not try to book (35.1% of those who did not try to book); those who did not try are left out of the satisfaction base, which flatters it.
✓Last year's 71% came from a paper survey handed out in the waiting room before the change, to patients who had got an appointment; this year's 50.8% is from an online survey by text of those who tried to book. Different method and different people.
✓The practice manager believes patients prefer the new system. Unweighted, the online form leads (46.2% against 40.7% for phone), but only 18-44s prefer it (58.4%); weighted to the age mix, preference is split (40.7% online, 45.0% phone, 6.7% in person).
✓Only 13 respondents need an interpreter; 61.5% of them would prefer to book in person.
✓Among those who tried to book, women look less satisfied than men (47.5% against 55.3%, bases 303 and 237), but the difference is not statistically significant at 95%.
✓Only 886 of 9,840 patients texted completed (about 9%), and the 7,260 registered adults without a mobile number on record were never invited; patients who cannot or do not go online are the least likely to have answered.

The analysis

Recommendation

Do not keep online-first as it is, and do not restore a universal 8am phone line on this evidence. Go with the targeted option, but widen it and pair it with fixes to the online route.

Introduce the dedicated phone line and in-person reception help, with eligibility based on need, not a hard 75 cut-off. Include all patients aged 65+, disabled patients, patients with limited English, and anyone who says they cannot manage the form, without asking for proof. The 65-74s are as dissatisfied as the over-75s, and a meaningful share of 45-64s struggle.

Make the in-person help real for the groups a phone line does not serve. That means interpreter or translated support, face-to-face help for patients with hearing loss, a BSL route, and the option for a carer or family member to book.

Fix the online route for everyone at the same time. Set a same-working-day reply standard, shorten the form, and fix its accessibility. Waiting for a reply is the main reason patients do not try to book, and no phone option addresses it.

What to tell the partners: online-first suits under-45s but is not working for the list as a whole. Run this as a three-month pilot with the measures below, and revisit a universal phone line when the pilot and the follow-up research report.

What would change it

Towards a phone line for everyone: follow-up research with patients without a mobile and with 45-74s shows widespread inability, not just preference, to use the form; or, after reply times and the form are fixed, satisfaction among 45-64s stays below about half and the put-off rate stays near one in four; or demand for the assisted line is so broad that gatekeeping it becomes unworkable or unfair.

Towards a narrower targeted service: a properly representative sample shows 65-74s coping well. This is unlikely, since the survey's biases run the other way.

Towards keeping the system as it is: only if representative evidence shows older, disabled and limited-English patients booking successfully and satisfied. Nothing here suggests that.

On women: if the satisfaction gap holds after controlling for age and in a larger sample, booking for women needs its own investigation, for example booking for dependants or particular appointment types.

Findings

Practice manager: no, you cannot tell the partners that patients prefer online-first. Patients are split, and once the sample's youth is corrected the balance tips towards phone.

Medium confidenceBase 886

Unweighted, 46.2% prefer the online form and 40.7% prefer phone (n=886). That 5.5-point gap is not statistically significant (z about 1.8). The sample is 52% aged 18-44 against 44% of the list, and under 10% aged 65+ against 24%. Weighted to the list's age mix, phone leads 45.0% to 40.7%, and phone plus in person reaches 51.8%. The survey went only by text to the 58% with a mobile on record, so even the weighted figure probably flatters online.
Online-first works for under-45s and is failing patients aged 65 and over. This is the firmest finding in the survey and rules out keeping the system as it is.

High confidenceBase 87

Satisfied with booking: 64.7% of 18-44s (n=272), 17.8% of 65-74s (n=45), 13.0% of 75+ (n=23). Across 65+ that is 11 of 68 satisfied and 44 of 68 dissatisfied. Prefer online: 58.4% of 18-44s (n=462), 16.7% of 65-74s (n=54), 6.1% of 75+ (n=33). The older bases are small, so the exact figures are loose, but the gap is far too large to be chance. These are also older patients who answer a text survey, so the true picture is probably worse.
The proposed targeted option draws its line too narrowly at 75. Patients aged 65-74 look as badly served as over-75s, and 45-64s lean towards phone and are lukewarm on booking.

Medium confidenceBase 391

65-74: 17.8% satisfied (n=45), 63.0% prefer phone and 14.8% in person (n=54). 45-64: 44.0% satisfied (n=209), significantly below the 64.7% for 18-44s; 49.0% prefer phone against 38.0% online (n=337). Over-75s are 11% of adults (about 1,900 people); 65-74s add 13% (about 2,200).
GP partner: this survey cannot tell you whether satisfaction has fallen from 71%. What it does show is that only about half of those who tried to book are satisfied now and a third are dissatisfied, which is a problem regardless of the trend.

Medium confidenceBase 549

50.8% satisfied, plus or minus 4.2 points (n=549); 33.9% dissatisfied. Age-weighted it is 46.3%, or about 44% if weighted to the people who actually try to book, since older patients book more. Last year's 71% came from 412 paper forms handed out in the waiting room, so it covered people who had got an appointment and turned up. This year's figure comes from a text survey that includes people who tried and failed. The two differ in mode, population and likely wording, so the 20-point difference cannot be read as a fall.
About one in four patients held back from booking when they needed an appointment. For most of them the deterrent was expected slowness or lack of appointments, not the online channel, and a phone line alone will not fix that.

Medium confidenceBase 215

24.3% said yes (215 of 886, plus or minus 2.8 points). Age-weighted this is 25.7%, or roughly 4,400 of 17,100 adults (about 3,900-4,900 on sampling error alone, before non-response bias). It was 35.1% among those who made no booking attempt (n=313), who are not in the satisfaction figure. Reasons (n=215, multi-select): expected to wait too long for a reply 43.7%, did not think I would get an appointment 28.8%, form difficult 16.7%, not confident online 9.3%. Not all of this is unmet need: 22.8% got help elsewhere and 17.7% said the problem went away.
Among the few older patients who were put off, the online channel itself is the main barrier. This is indicative only.

Low confidenceBase 29

Of 29 put-off respondents aged 65+, 12 said they were not confident going online and 8 said the form is difficult. Among under-65s (n=186) not confident online was 2-6%.
PPG chair, on women: women report lower satisfaction than men, but the gap is not statistically reliable and has not been checked for age. Treat it as a signal to investigate, not as established.

Low confidenceBase 540

47.5% of women (n=303) and 55.3% of men (n=237) were satisfied. The 7.8-point gap gives z about 1.8 (p about 0.07), below the 95% threshold. Very satisfied is identical (21.8% and 21.5%); the difference sits in fairly satisfied. There are no tables by sex for being put off or for preferred channel.
PPG chair, on interpreters: this survey cannot answer the question. The little evidence there is points to in-person help with language support, not a phone line.

Low confidenceBase 13

Only 13 respondents (1.5%) need an interpreter, reached through what appears to be an English-language text survey. 8 of the 13 prefer to book in person, 3 by phone, 1 online, 1 does not know. There is no satisfaction table for this group. Two open-text answers (A112, A756) ask for a form in Polish and for an Urdu speaker.
Disabled patients are much less likely to want the online route, and their needs are varied. A phone line helps many of them but not all.

Medium confidenceBase 102

23.5% of disabled respondents prefer online against 49.1% of others (n=102 and 784), a clearly significant gap; 50.0% prefer phone and 15.7% in person. Satisfaction is 40.0% against 52.3% (n=65 and 484), which is not significant (z about 1.9). In open text, 21 people describe a specific impairment. 10 want a person on the phone. The other 11 need something else: a screen-reader-compatible or large-print form, one-handed use, save and return, a family member booking for them, face-to-face reception because of hearing loss, or BSL.
Open-text requests for phone booking come mostly from working-age patients, and around a third are tied to complaints about the online route being slow or long. Fixing reply times and the form should reduce some of the phone demand, though not all of it.

Medium confidenceBase 246

Of 246 open-text answers, 97 ask to book by phone or speak to a person (94 name the phone explicitly). 84 of the 97 are from under-65s; 13 of the 18 commenters aged 65+ ask for it. 30 of the 97 also complain about reply time (11) or form length (19). Separately, 45 answers ask for faster replies (reported waits of 24 hours to 5 days), 29 want a shorter or simpler form, and 45 say the form is easy and nothing should change. Open text is self-selected (28% answered), so these counts show the range of issues, not how common they are.

Caveats

Coverage: the survey went only by text to the 58% of adults with a mobile on record. The other 42% (about 7,260 people) could not take part and are probably the least comfortable online. Results likely overstate how well online-first is working.
Response: 886 completes from 9,840 texts is a 9.0% response rate. Those who replied may differ from those who did not in ways age weighting cannot fix.
Age skew: respondents aged 65+ are under 10% of the sample against 24% of the list. The weighted figures use age only, and a quarter of each weighted result rests on 87 older respondents, so weighted estimates are less precise than the raw sample size suggests.
Small bases: 75+ (33; 23 for satisfaction; 11 for reasons), 65-74 (54; 45; 18), disability satisfaction (65) and interpreter (13) are too small for precise percentages. Read them as direction only.
Satisfaction was asked only of the 549 who tried to book. The 215 who were put off, 110 of whom made no booking attempt at all, are partly or wholly missing from it, so 50.8% is a generous reading.
Last year's 71% is not a baseline. It came from paper forms in the waiting room, from people who had succeeded in getting an appointment, before the change.
Stated preference for phone is not the same as being unable to use the online form. Respondents were also not asked to weigh phone against the old 8am queue; 10.2% of put-off patients already expected not to get through by phone.
Being put off is self-reported need. About 40% of mentions are 'got help elsewhere' or 'problem went away', some of which is appropriate.
The sex and disability satisfaction gaps are not significant at 95% and are not age-adjusted. No tables were run by sex for being put off or preferred channel, or by interpreter need for satisfaction.
Open text was answered by 246 of 886 and only 18 were aged 65+. Counts are mentions, not prevalence. Three answers in the phone count (A114, A433, A644) ask to speak to a person without naming the phone.

Next steps

decisionApprove a three-month pilot of the dedicated phone line and reception help, open to patients aged 65+, disabled patients, patients with limited English, and anyone self-declaring difficulty with the form. Publicise it by letter and in the surgery, not only by text.
experimentSet and publish a same-working-day reply standard for online requests, with an automatic acknowledgement. Pull actual reply times from the system logs first.
experimentShorten the request form (a quick version for simple requests) and fix its accessibility: screen reader labels, large text, one-handed use, save and return, and consented booking by a family member or carer. Test it with disabled patients before release.
researchRun research with the patients this survey missed: a postal or telephone survey of a random sample of the 7,260 adults without a mobile and of over-65s, and 12-15 interviews with patients who need an interpreter, run through interpreters or community groups. Find out what the current phone exception is and whether patients know about it.
researchRun extra cuts of the existing data: satisfaction by sex within age band, and being put off and preferred channel by sex, disability and interpreter need.
monitorSet a like-for-like tracker: repeat the same satisfaction and put-off questions every six months by the same mixed method (text plus post or phone for those without a mobile), weighted to the list. Stop comparing against the waiting-room 71%.
decisionAt the pilot review, decide whether to extend phone booking to all patients, using the tracker, the non-mobile research and the line's usage data.

Open-text themes it coded

Wants to book by phone or speak to a person 97Faster replies to online requests 45Shorter, simpler online form 29Happy with the online form, no change needed 45Disability and health-related access needs 21In-person help at reception 4Language and interpreting support 3Continuity: seeing the same GP (outside the booking-channel decision) 15No suggestion or rarely uses the practice 32

The survey it planned

2 screening questions and 12 questions, as the model wrote them.

S1
Are you registered as a patient at Ashgrove Medical Group? (If you are helping someone to fill this in, please answer every question for the patient who received the text message.)
One answer
- Yes
- No
- Not sure
Continues if Yes
S2
How old are you?
One answer
- Under 18
- 18 to 44
- 45 to 64
- 65 to 74
- 75 or over
Continues if 18 to 44; 45 to 64; 65 to 74; 75 or over
Q1
Since April 2026, in which of these ways have you asked for an appointment for yourself at Ashgrove Medical Group? Please select all that apply.
Any that apply
- I filled in the online request form myself
- Someone else filled in the online request form for me
- I phoned the practice
- I went to reception in person
- Another way
- I have not asked for an appointment since April 2026
- Don't remember
Q2
Overall, how satisfied or dissatisfied are you with the way you book appointments at Ashgrove Medical Group at the moment?
Scale

Scale 1-5: Very dissatisfied to Very satisfied

Routing Shown only if Q1 is: I filled in the online request form myself; Someone else filled in the online request form for me; I phoned the practice; I went to reception in person; Another way
Q3
Thinking about the last time you asked for an appointment at the practice, how easy or difficult was it to make your request?
Scale

Scale 1-5: Very difficult to Very easy

Routing Shown only if Q1 is: I filled in the online request form myself; Someone else filled in the online request form for me; I phoned the practice; I went to reception in person; Another way
Q4
Since April 2026, has there been a time when you wanted an appointment at Ashgrove Medical Group but decided not to ask for one, or gave up while trying to ask for one?
One answer
- Yes, once
- Yes, more than once
- No
- I have not wanted an appointment since April 2026
- Don't remember
Q5
Thinking about the most recent time this happened, why did you not ask for an appointment, or give up? Please select all that apply.
Any that apply
- I did not know how to ask for an appointment
- I do not have the internet or a device I can use
- I found the online form difficult to use
- The online form was closed or not available when I tried
- I could not get through on the phone
- I did not want to describe my health problem in writing
- I could not do it in a language I understand well
- I had no one to help me
- I did not think I would get an appointment soon enough
- It was not a convenient time for me
- My health problem got better
- Another reason
Routing Shown only if Q4 is: Yes, once; Yes, more than once
Q6
The practice has an online form that patients fill in to ask for an appointment. Which of these best describes you?
One answer
- I can use the form on my own without difficulty
- I can use the form on my own, but I find it difficult
- I can only use the form with help from someone else
- I am not able to use the form
- I have not tried to use the form
Q7
If you could choose, how would you most prefer to ask for an appointment at the practice?
One answer
- By filling in an online form
- By phoning the practice
- By going to reception in person
- By having someone else ask for me
- No preference
Q8
Which of these, if any, would make it easier for you to book an appointment at Ashgrove Medical Group? Please select up to three.
Any that apply
- A phone booking line open to all patients every morning
- A separate phone line for patients who find it hard to go online
- Help in person at reception to make an appointment request
- A shorter or simpler online form
- The online form being open for more hours of the day
- The online form being available in other languages
- Being able to pick an appointment time online myself
- A faster reply from the practice after I send a request
- Being able to choose how the practice replies to me (text, phone call or email)
- Clearer information about how to book
- Something else
- Nothing - booking is already easy for me
Q9
Is there anything else you would like to tell us about booking appointments at the practice, or what would make it easier for you? (Optional. Please do not include personal medical details.)
Open text
Q10
Patients who cannot use the online form can still ask for an appointment by phoning reception. Before today, did you know this?
One answer
- Yes
- No
- Not sure
Q11
Do you have any physical or mental health conditions or illnesses, lasting or expected to last 12 months or more, that reduce your ability to carry out day-to-day activities?
One answer
- Yes, a lot
- Yes, a little
- No
- Prefer not to say
Q12
When you contact or visit the practice, do you need an interpreter or someone to translate into another language for you?
One answer
- Yes, always
- Yes, sometimes
- No
- Prefer not to say

Sample plan

WHO IS INVITED: Every registered adult (18+) with a mobile number on record gets a text with a unique one-use link, plus one reminder to non-responders after about 5 days. The text says a relative or carer may help and should answer for the patient. Check the number of mobiles on record by age band before launch; of the 17,100 adults this is likely to be lowest for the 75+ band.

COMPLETES AND QUOTAS: About 900 completes, with a minimum target of 225 in each age band from S2 (18 to 44, 45 to 64, 65 to 74, 75 or over). There are no caps on the older bands. If the survey has to close at 900, cap 18 to 44 and 45 to 64 at 225 each. Send a second reminder only to bands that are short. Overall figures are weighted by age band to the practice list.

PRECISION, WHICH THE BRIEF'S TARGET DOES NOT SURVIVE: Plus or minus 5 points at 95% in each of four bands needs about 385 completes per band, about 1,540 in total. A finite population correction would bring the older bands down to roughly 320 each, so about 1,400 in total. This cannot be met with 900. At 225 per band the margin is about plus or minus 6.5 points. It is wider for Q2 and Q3, which only go to patients who have booked since April, and wider still for Q5. The 75+ band may fall short of 225, which would need a response rate of perhaps 15 to 20% from that group. The partners have three options:

Accept about plus or minus 6.5 points per band.
Leave the link open past 900 until each band reaches about 385. Everyone is invited anyway, so this adds little cost, and it is my recommendation.
Report two bands (under 65, and 65 or over) at about 450 each, which gives plus or minus 4.6 points.

COVERAGE BIAS, THE MAIN RISK: A text-to-web survey cannot reach patients with no mobile, no internet or limited English, and these are the patients the decision is about. Results will therefore flatter online-first, most of all among over-75s, disabled patients and those needing an interpreter. Weighting by age does not fix this. Treat figures for those groups as a best case. I strongly recommend a small supplement outside the stated constraint: the same questions on paper at reception and by phone to a random sample of patients aged 75+ or with no mobile on record, plus translated versions in the practice's main languages. Report the supplement separately.

FIELDWORK NOTES: Randomise option order in Q5, Q7 (first three options) and Q8, keeping 'Another reason', 'Something else', 'No preference' and 'Nothing' fixed at the end. Make 'I have not asked...' and 'Don't remember' in Q1, and 'Nothing' in Q8, exclusive. There is no attention-check question, because the survey is short and goes to patients rather than a panel; screen for speeding and duplicates instead. Estimated length is 4 to 5 minutes. Responses are anonymous and not linked to the medical record.

NEUTRALITY: The practice manager wants the survey to show online-first is working. The questions are worded neutrally and the survey is built to test that belief, not to demonstrate it. All results will be reported, whichever way they fall.