Claude Opus 5.5: How Ashgrove patients find online-first booking

The decision: whether to keep online-first booking as it is, restore a morning phone booking line for every patient, or add a dedicated phone line and in-person help at reception only for patients who need them (over-75s, patients with a disability, and patients with limited English). The brief and the data →

Verdict: ✗ Not sound
Research score: 97 of 100
Analysis rating: 1,646
Head to head, this task: won 11 of 11

Why it is not sound

The analysis

Missed a trap: Q2 satisfaction (50.8%) is asked only of the 549 who tried to book. 24.3% of all respondents needed an appointment but did not try to book (35.1% of those who did not try to book); those who did not try are left out of the satisfaction base, which flatters it.

Scorecard

Numbers 11 of 11 right

Question		Its answer	True
Among respondents who tried to book in the last three months, what percentage were satisfied (fairly or very) with the booking process?	✓	50.8%	50.8%
Weighting each age band's percentage satisfied (fairly or very, among those who tried to book) by that band's share of registered adults, what is the overall percentage satisfied?	✓	46.3%	46.3%
What percentage of all respondents said there was a time in the last three months when they needed an appointment but did not try to book one?	✓	24.3%	24.3%
Weighted to the age mix of registered adults, what percentage would prefer to book by phone?	✓	45%	45%
What is the 95% margin of error, in percentage points, on the unweighted percentage satisfied among respondents who tried to book?	✓	4.2 points	4.18 points
Among respondents who tried to book, is the difference in the percentage satisfied (fairly or very) between female and male respondents statistically significant at the 95% level? (1 for yes, 0 for no)	✓	No	No
How many of the open-text answers ask to be able to book by phone, or for a phone booking line?	✓	94	97
How many of the open-text answers from respondents aged 65 or over ask to be able to book by phone, or for a phone booking line?	✓	11	13
Weighted to the age mix of registered adults, about how many of the practice's 17,100 registered adults would say there was a time in the last three months when they needed an appointment but did not try to book one?	✓	4,392	4,393
Among respondents aged 18 to 44, what percentage would prefer to book using the online request form?	✓	58.4%	58.4%
What percentage of the patients sent the text message completed the survey?	✓	9%	9%

Traps in the data 6 of 7 handled

✓Patients aged 65 or over are 24% of registered adults but 9.8% of respondents (text link, online only). They are less satisfied and prefer the phone, so weighting to the age mix lowers satisfaction among those who tried to book from 50.8% to 46.3% and raises the phone preference from 40.7% to 45.0%.
✗Q2 satisfaction (50.8%) is asked only of the 549 who tried to book. 24.3% of all respondents needed an appointment but did not try to book (35.1% of those who did not try to book); those who did not try are left out of the satisfaction base, which flatters it.
✓Last year's 71% came from a paper survey handed out in the waiting room before the change, to patients who had got an appointment; this year's 50.8% is from an online survey by text of those who tried to book. Different method and different people.
✓The practice manager believes patients prefer the new system. Unweighted, the online form leads (46.2% against 40.7% for phone), but only 18-44s prefer it (58.4%); weighted to the age mix, preference is split (40.7% online, 45.0% phone, 6.7% in person).
✓Only 13 respondents need an interpreter; 61.5% of them would prefer to book in person.
✓Among those who tried to book, women look less satisfied than men (47.5% against 55.3%, bases 303 and 237), but the difference is not statistically significant at 95%.
✓Only 886 of 9,840 patients texted completed (about 9%), and the 7,260 registered adults without a mobile number on record were never invited; patients who cannot or do not go online are the least likely to have answered.

The analysis

Recommendation

Do not keep online-first as it is. It works well for many under-45s (64.7% satisfied, 58.4% prefer online) but is failing over-65s (about 16% satisfied). Weighted to the practice's age mix, it is not the preferred route overall: phone 45.0% against online 40.7%.

Act in three stages:

Now: introduce the targeted measures, a dedicated phone line and in-person help at reception for over-75s, disabled patients and patients with limited English. These groups have the clearest unmet need, and the measures are low-regret.
Alongside: fix the online route's two biggest failures.
Set and publish a reply standard, such as a same-working-day response or acknowledgement. 'Expected to wait too long' is the top reason patients held back (43.7%).
Cut and de-duplicate the form, and run an accessibility audit.
Decide on universal phone access through a time-limited trial, not now. Phone demand clearly extends beyond the targeted groups (49% of 45 to 64s prefer phone), so a targeted-only model will probably not be enough. But reinstating an 8am first-come phone scramble could recreate the access problems online-first was meant to solve. Trial a general phone option that feeds the same triage queue, with receptionists completing the request for the caller. Measure call waits, reply times, satisfaction and put-off rates.

Stakeholder answers:

Practice manager: no. Patients do not prefer the new system overall, and you should not tell the partners online-first is working, except for younger patients.
GP partner: satisfaction is now about 51% (46% age-weighted) against last year's 71%, but the methods differ too much to say how much, or even whether, it has fallen.
PPG chair: women are not significantly less satisfied than men. The survey cannot tell you about interpreter users. Older and disabled patients are the groups with evidence of difficulty.

What would change it

Each of these would shift the recommendation:

Towards targeted-only: if fixing reply times and the form brought satisfaction among 45 to 64s up to the level of 18 to 44s, and their phone preference fell, during the trial. That would show their phone demand was mainly a reaction to slow replies.
Towards universal phone booking: if the trial showed universal phone access can be staffed without long call queues, while satisfaction and put-off rates improve.
Towards universal phone booking: if a representative survey (postal or phone, including patients with no mobile, with language support) found phone preference or put-off rates well above the levels here.
Against the targeted design: if eligibility checks deter patients who need the service, or other excluded groups show comparable need. Examples include carers, patients with mental health conditions, and patients with low literacy who do not identify as disabled.
Towards expanding interpreter and in-person support: if practice records show interpreter-dependent patients are booking much less often than their health needs would predict.

Findings

Online-first is not the majority preference. Once the sample is reweighted to the practice's age mix, more patients prefer phone than the online form. Even this understates phone demand, because the 42% of adults with no mobile on record could not take part.

Medium confidenceBase 886

Q6, base 886. Unweighted: online 46.2%, phone 40.7%, in person 4.6%. Weighted to the registered-adult age mix: phone 45.0%, online about 40.7%, in person about 6.7%. The sample is 52.1% aged 18 to 44 against 44% of registered adults, and 3.7% aged 75 or over against 11%.
Only about half of recent bookers were satisfied with booking. The figure drops below half once age is corrected for.

Medium confidenceBase 549

Q2, base 549 who tried to book. 50.8% were satisfied (fairly or very), with a 95% margin of error of ±4.2 points, so roughly 47% to 55%. Weighted to the registered-adult age mix it is 46.3%. 33.9% were dissatisfied.
Booking is failing older patients badly. Fewer than one in five patients aged 65 or over who tried to book were satisfied, against more than half of those under 65. This is the clearest and most robust gap in the data.

High confidenceBase 549

Q2 satisfied: 18 to 44 64.7% (n=272); 45 to 64 44.0% (n=209); 65 to 74 17.8% (n=45); 75 or over 13.0% (n=23). Combined, 65 or over is about 16% (11 of 68) against about 56% under 65 (268 of 481). The gap is far beyond sampling error despite the small older bases. In the 75 or over group, 73.9% were dissatisfied.
Phone preference is not confined to the groups the targeted option would serve. Half of patients aged 45 to 64, and four in ten patients without a disability, would rather phone. A line limited to over-75s, disabled patients and patients with limited English would leave much of the stated demand unmet.

Medium confidenceBase 886

Q6 phone preference: 45 to 64 49.0% (n=337); 65 to 74 63.0% (n=54); 75 or over 72.7% (n=33); no disability 39.5% (n=784); disability 50.0% (n=102). In open text, 94 answers ask to book by phone. Six (A006, A039, A201, A574, A583, A659) explicitly reject a phone line 'just for some'.
About a quarter of patients went without booking when they needed care. The main reason was expecting a slow reply, not the phone or the form.

Medium confidenceBase 215

Q4, base 886: 24.3% put off. Weighted to the age mix this is about 25.7%, or roughly 4,400 of 17,100 registered adults, if non-respondents resemble respondents. Q5, base 215: expected to wait too long for a reply 43.7%; did not think I would get an appointment 28.8%; got help elsewhere 22.8%; form difficult 16.7%; not confident online 9.3%.
Older patients who held back mostly cite digital barriers. Under-65s cite the wait.

Low confidenceBase 29

Q5, 'not confident going online': 65 to 74 44.4% (n=18) and 75 or over 36.4% (n=11), against 2.4% to 5.9% under 65. 'Form difficult to use': about 27% of those aged 65 or over. The bases are very small, so the direction is credible but the sizes are not.
Slow replies and an overlong, repetitive form are the two biggest fixable problems within the online route. Both can be fixed whichever access option is chosen.

Medium confidenceBase 246

Open text, 246 answers: 45 complain about waiting for a reply, with waits of 24 hours to 5 days described. 29 say the form is too long, repetitive or confusing, and 3 say they abandoned it partway (A036, A300, A529).
Disabled patients describe specific access barriers in the online form. These need fixing regardless of channel. Their lower satisfaction score points the same way but is not statistically conclusive on its own.

Medium confidenceBase 65

Q2 satisfied: disability 40.0% (n=65) against no disability 52.3% (n=484). z≈1.86, not significant at 95%. Open text names screen-reader buttons read out only as 'button' (A237), small tap targets for tremor (A193, A443), one-handed use (A221), large print (A565, A713), saving a part-finished request (A358), BSL (A375), easy read (A262) and proxy booking (A437).
There is no significant difference in satisfaction between women and men.

Medium confidenceBase 540

Q2 satisfied: female 47.5% (n=303) against male 55.3% (n=237). The 7.8-point gap gives z≈1.80, p≈0.07, so it is not significant at 95%.
This survey cannot say whether patients who need an interpreter find booking harder. The few who answered mostly want face-to-face help.

Low confidenceBase 13

Q10: only 13 respondents (1.5%) need an interpreter. The survey was English-only and sent by text. Of the 13, 61.5% (8) prefer booking in person and 7.7% (1) online. Satisfaction is not reported for this group. Open text asks for a Polish form (A112) and Urdu help (A756).
The fall from 71% to 51% satisfied cannot be read as a measured decline. The two surveys differ in method too much to compare.

Low confidenceBase 549

October 2025: paper questionnaire handed out in the waiting room, 412 returned, 71% satisfied, before the change. September 2026: text-message survey to mobile holders, 549 recent bookers, 50.8% satisfied (46.3% age-weighted). The samples, modes and populations reached are all different.

Caveats

Coverage bias: only the 58% of adults with a mobile number on record were invited, and the survey itself was online. Patients least able to use online booking are the least likely to appear, so dissatisfaction and phone preference are probably understated.
The response rate was low: 9.0% of those texted (886 of 9,840), or 5.2% of all registered adults. A further 160 started but did not finish, and those may skew towards the less digitally confident. Self-selection could push results either way.
Over-65s are under-represented: 9.8% of the sample against 24% of registered adults. The tables are unweighted. Age-weighted figures here are simple re-weights on age only, not on sex, disability or language.
Subgroup bases are small: 75 or over n=33 (n=23 at Q2), 65 to 74 Q5 n=18, 75 or over Q5 n=11, interpreter n=13. Treat their percentages as directional only.
Last year's 71% is not comparable. It came from a paper survey of people already in the waiting room, which favours patients who succeeded in getting an appointment, and it may have used different wording. No change in satisfaction can be measured from these two numbers.
Stated preference is not the same as need or behaviour. Some phone preference reflects habit, and some reflects dissatisfaction with slow replies that could be fixed inside the online route.
The survey was English-only and gave no language support, so it cannot assess patients with limited English.
The 'put off' estimate of about 4,400 adults assumes respondents represent non-respondents. It also covers only the last three months and relies on self-reported need.
Open-text counts show how often a view comes up among the 246 who wrote something. They are not population proportions. The phone count includes only answers that explicitly mention phoning or ringing; three further answers ask for 'a person' without naming a channel.
The disability satisfaction gap (40.0% against 52.3%) and the sex gap (47.5% against 55.3%) are both not significant at 95%. Neither should be reported as a proven difference.

Next steps

decisionLaunch the dedicated phone line and reception assistance for over-75s, disabled patients and patients with limited English. Make eligibility self-declared and simple, and flag it on the patient record.
decisionSet a response-time standard for online requests, for example acknowledgement within 2 working hours and a decision the same working day. Staff triage to meet it and publish performance.
experimentShorten and de-duplicate the request form, offer a quick route for simple requests, allow saving a part-finished request, and run a WCAG 2.2 AA accessibility audit with screen-reader and motor-impairment testing.
experimentRun a 3-month trial of a general phone booking option open to all patients. Calls go into the same triage queue, with staff entering the request, rather than an 8am first-come system. If possible, compare against a pre-trial baseline or run it on alternate weeks.
researchField a representative follow-up survey. Use postal and phone modes to reach patients without a mobile, offer translated and interpreter-assisted versions, weight results to age, sex and other known list characteristics, and repeat the same questions so it becomes a valid baseline.
researchAnalyse practice system data: booking and request volumes by age, sex and recorded interpreter need, before and after April. Also look at DNA rates, NHS 111 and A&E referrals back to the practice, and complaints.
researchInterview 10 to 15 patients who need an interpreter, are deaf or BSL users, have learning disabilities, or are carers, with interpreters present, to design the targeted support.
monitorTrack satisfaction, put-off rate, channel use, reply times and call waits monthly by age group. Report to the partners with margins of error and without comparing to the 2025 paper survey.
researchConsider offering continuity options, such as a named or usual GP for patients with long-term conditions, as part of the booking redesign.

Open-text themes it coded

Wants to book by phone / a phone booking line 94Phone booking should be for everyone, not only some groups 6Wants to speak to a person, channel not specified 3Slow replies to online requests 45Online form too long, repetitive or confusing 29Abandoned the form partway 3Happy with online form, no change wanted 46No suggestion / rarely uses the practice 32Continuity of care: same or named GP 15Disability-related access barriers or adjustments 21Language or interpreter needs (including BSL) 3Relies on or wants help from staff, family or a carer to book 9Wants in-person help at reception 3

The survey it planned

3 screening questions and 12 questions, as the model wrote them.

S1
Are you a patient registered at Ashgrove Medical Group?
One answer
- Yes
- No
- Not sure
Continues if Yes
S2
How old are you? If you are helping a patient fill this in, please give the patient's age.
One answer
- Under 18
- 18 to 44
- 45 to 64
- 65 to 74
- 75 or over
Continues if 18 to 44; 45 to 64; 65 to 74; 75 or over
S3
Who is filling in this survey?
One answer
- I am the patient, answering for myself
- I am helping a patient answer about their own experience (for example, as a family member or carer)
Continues if I am the patient, answering for myself; I am helping a patient answer about their own experience (for example, as a family member or carer)
Q1
Since April 2026, have you tried to get an appointment with Ashgrove Medical Group? This includes appointments with a GP, nurse or other member of staff.
One answer
- Yes
- No
- Not sure
Q2
The last time you tried to get an appointment, how did you ask for it?
One answer
- I filled in the online request form myself
- Someone else filled in the online request form for me
- I phoned the practice
- I went to reception in person
- Another way
- I can't remember
Routing Shown only if Q1 is: Yes
Q3
Overall, how satisfied or dissatisfied were you with the process of getting an appointment the last time you tried?
Scale

Scale 1-5: Very dissatisfied to Very satisfied

Routing Shown only if Q1 is: Yes
Q4
Since April 2026, has there been a time when you felt you needed an appointment but did not ask for one, or gave up before you got one?
One answer
- Yes
- No
- Not sure
Q5
Why did you not ask for an appointment, or give up? Please choose all that apply.
Any that apply
- I did not know how to ask for an appointment
- I found the online form hard to use
- I do not have internet access or a suitable device
- I could not get through on the phone
- When I phoned, I was told to use the online form
- The online form was closed or unavailable when I tried
- I was not sure my request would be seen in time
- I missed the practice's reply (text or call)
- Language difficulties
- A disability or health condition made it hard
- I thought the practice was too busy
- My problem got better
- Another reason
Routing Shown only if Q4 is: Yes
Q6
What did you do instead? Please choose all that apply.
Any that apply
- Went to a pharmacy
- Contacted NHS 111
- Went to A&E or an urgent treatment centre
- Paid for private care
- Looked after myself at home
- Waited and tried again later
- Did nothing
- Something else
Routing Shown only if Q4 is: Yes
Q7
If you needed an appointment, which way would you most like to ask for it?
One answer
- Fill in an online request form myself
- Fill in an online request form with help from someone
- Phone the practice
- Go to reception in person
- I have no preference
Q8
Which of these, if any, would make it easier for you to get an appointment? Please choose up to three.
Any that apply
- Keeping the online form open for longer each day
- A simpler online form
- The online form available in other languages
- A phone line open every morning for any patient to book
- A separate phone line only for patients who cannot easily go online
- Help in person at reception to make a request
- An interpreter when I contact the practice
- Being able to choose whether the practice replies by text or phone
- Being told when I will get a reply
- None of these, it is easy enough already
- Something else
Q9
Is there anything else you would like to tell us about getting an appointment at the practice? (Optional)
Open text
Q10
Do you have a physical or mental health condition, disability or illness that has lasted, or is expected to last, 12 months or more, and that reduces your ability to carry out day-to-day activities?
One answer
- Yes, a lot
- Yes, a little
- No
- Prefer not to say
Q11
Do you need an interpreter, or help with English, when you speak to the practice?
One answer
- Yes
- No
- Prefer not to say
Q12
How confident are you using the internet to fill in forms, for example, to order a prescription or fill in an online request?
One answer
- Very confident
- Fairly confident
- Not very confident
- Not at all confident
- I do not use the internet

Sample plan

WHO IS INVITED: Every registered adult patient with a mobile number on record gets one text with a link, plus one reminder text after 7 days. The text should come from the practice and be neutral, for example: 'Ashgrove Medical Group wants to hear about your experience of getting an appointment, good or bad. 5 minutes.' Proxy completion by a carer is allowed (S3). Please send no more than one invitation per number, so shared family numbers do not cause duplicates.

PRECISION, WHICH CANNOT BE MET AT 900: To get plus or minus 5 points at 95% confidence (worst case, p=0.5) you need about 385 completes per age band. Allowing for the finite size of each band at the practice (roughly 1,500 to 2,000 patients aged 75 or over and 65 to 74), you still need about 310 to 365 per band. That is about 1,350 completes in total. Nine hundred completes split evenly gives about 225 per band, which is roughly plus or minus 6 to 6.5 points. The partners need to choose one of two options. (a) Accept about plus or minus 6.5 points per band. (b) Keep the survey open past 900 until each band reaches about 330. Because every patient is texted anyway, option (b) costs only time. We recommend (b).

QUOTAS: These are minimum targets per age band (S2), not caps. At least 225 per band (option a) or about 330 per band (option b). Do not close the 18 to 44 and 45 to 64 bands early, because the total is weighted. Watch the 75 or over band daily. Response from this band is likely to be lowest.

COVERAGE BIAS, WHICH IS CRITICAL: A text link to an online survey misses the patients this decision is most about. These are patients without a mobile number, without internet access, or with limited English. If you survey only online, the results will lean towards online-first, and this would be built into the method. We strongly recommend three additions. (1) A paper version of the same 12 questions, posted to patients aged 75 or over with no mobile on record, and available at reception with a sealed return box. (2) Phone completion on request, using the same wording, done by someone who is not the person who handled the patient's bookings. (3) Translated versions in the practice's main non-English languages, with interpreter-supported completion. Without these, interpreter-need findings will be close to zero responses and must not be reported as representative. Record the completion mode for every response and report it.

WEIGHTING: Weight completes to the practice's registered adult profile by age band and sex. Report unweighted base sizes for every figure.

DATA QUALITY: Drop completions under 90 seconds and duplicate submissions.

NEUTRALITY: The practice manager hopes to show that online-first is working. The questionnaire therefore uses balanced scales, does not name the options under review, and lists all answers, including 'no preference' and 'none of these'. All results should go to the partners in full, whichever way they point. The manager should not filter the results or choose which ones to report.