Benchmarks / SurveyBench / How Ashgrove patients find online-first booking / Gemini 4 Argon
Gemini 4 Argon: How Ashgrove patients find online-first booking
The decision: whether to keep online-first booking as it is, restore a morning phone booking line for every patient, or add a dedicated phone line and in-person help at reception only for patients who need them (over-75s, patients with a disability, and patients with limited English). The brief and the data →
- Verdict
- ✗ Not sound
- Research score
- 71 of 100
- Analysis rating
- 985
- Head to head, this task
- won 8 of 35
Why it is not sound
The survey
- A double-barrelled question: Q4
The analysis
- Got at least one set number wrong
- Reported a material finding the data does not support
- Made a recommendation that does not follow from the data
- Missed a trap: Q2 satisfaction (50.8%) is asked only of the 549 who tried to book. 24.3% of all respondents needed an appointment but did not try to book (35.1% of those who did not try to book); those who did not try are left out of the satisfaction base, which flatters it.
- Missed a trap: Last year's 71% came from a paper survey handed out in the waiting room before the change, to patients who had got an appointment; this year's 50.8% is from an online survey by text of those who tried to book. Different method and different people.
- Missed a trap: Only 13 respondents need an interpreter; 61.5% of them would prefer to book in person.
- Proposed a mistaken next step: Tell the partners that satisfaction with booking has fallen from 71% to this year's figure because of online-first
- Proposed a mistaken next step: Restore a morning phone booking line for every patient on the strength of this survey alone
Findings the judges found unsupported
To the Practice Manager ('Patients prefer the new system, don't they? Can I tell the partners online-first is working?'): No. Even in a survey distributed by text message—which heavily over-represents younger, digitally connected patients—only 46.2% prefer the online form, while 40.7% prefer booking by phone and 4.6% in person. When weighted to the actual age profile of registered adults, preference for booking by phone (45.0%) exceeds preference for the online form (40.7%). Furthermore, 24.3% of respondents needed an appointment in the last three months but did not try to book one (an age-weighted equivalent of ~4,412 registered adults), primarily because they expected to wait too long for a reply (43.7% of those put off) or did not think they would get an appointment (28.8%).
Age-weighted in-person preference is approximately 6.7%, not the reported 7.6%.To the GP Partner ('Last year 71% were satisfied with booking. Has satisfaction fallen since we went online?'): Yes, satisfaction has fallen substantially, though the exact percentage-point drop must be treated with caution due to a change in survey methodology. Among respondents who tried to book, 50.8% are satisfied (95% CI: 46.6% to 55.0%), down 20.2 percentage points from 71% last year. Because older patients are severely under-represented in this SMS sample (9.8% aged 65+ in the sample vs 24.0% in the practice population), age-weighting reduces overall satisfaction further to 46.3% (or 42.3% if weighted by age and booking incidence). Satisfaction collapses among older adults: just 17.8% of 65–74s and 13.0% of 75+s are satisfied, while 60.0% and 73.9% respectively are dissatisfied.
The analysis asserts that satisfaction has fallen and reports a 20.2-point decline. Different survey methods and different populations mean the two surveys cannot establish a change in satisfaction. The later caveat does not undo this conclusion.A restricted phone and reception service (only for over-75s, disabled patients, and limited-English patients) will leave the majority of phone-dependent patients behind. Strong preference for phone booking and low satisfaction with online-first extend heavily into the 45–64 and 65–74 age bands. In the open-text responses, 85% of the requests for phone booking (64 out of 75) came from patients aged under 65.
The 72.2% figure adds two multi-select responses. Without respondent-level overlap, the proportion citing either lack of online confidence or form difficulty cannot be calculated.To the PPG Chair ('Are women, or patients who need an interpreter, finding it harder to book than others?'): For women vs men, the 7.8 percentage-point gap in satisfaction (47.5% female vs 55.3% male) is not statistically significant at the 95% confidence level (p = 0.071), so we cannot be certain women find it harder overall, though it warrants monitoring. For patients needing an interpreter, the sample is very small (n=13) because the survey was sent in English by SMS, but the signal is stark: 61.5% prefer to book in person at reception and only 7.7% (1 person) prefer the online form.
The supplied fielding information does not establish that the survey was English-only or that this caused the small interpreter sample. Calling the interpreter result a stark signal, and subsequently using it to justify service design, goes beyond what 13 respondents can reliably establish.For patients willing to book online, the primary failure is not the concept of online booking, but slow practice turnaround times (waiting 24 hours to 5 days for a reply) and an overly long, repetitive form.
The reported counts of 37 slow-reply comments and 23 form comments conflict with the supplied answers and the analysis's later counts of 45 and 29. The tables also do not isolate patients willing to book online, so they cannot establish the primary failure specifically for that group.
Scorecard
Numbers 10 of 11 right
| Question | Its answer | True | |
|---|---|---|---|
| Among respondents who tried to book in the last three months, what percentage were satisfied (fairly or very) with the booking process? | ✓ | 50.8% | 50.8% |
| Weighting each age band's percentage satisfied (fairly or very, among those who tried to book) by that band's share of registered adults, what is the overall percentage satisfied? | ✓ | 46.3% | 46.3% |
| What percentage of all respondents said there was a time in the last three months when they needed an appointment but did not try to book one? | ✓ | 24.3% | 24.3% |
| Weighted to the age mix of registered adults, what percentage would prefer to book by phone? | ✓ | 45% | 45% |
| What is the 95% margin of error, in percentage points, on the unweighted percentage satisfied among respondents who tried to book? | ✓ | 4.2 points | 4.18 points |
| Among respondents who tried to book, is the difference in the percentage satisfied (fairly or very) between female and male respondents statistically significant at the 95% level? (1 for yes, 0 for no) | ✓ | No | No |
| How many of the open-text answers ask to be able to book by phone, or for a phone booking line? | ✗ | 75 | 97 |
| How many of the open-text answers from respondents aged 65 or over ask to be able to book by phone, or for a phone booking line? | ✓ | 11 | 13 |
| Weighted to the age mix of registered adults, about how many of the practice's 17,100 registered adults would say there was a time in the last three months when they needed an appointment but did not try to book one? | ✓ | 4,412 | 4,393 |
| Among respondents aged 18 to 44, what percentage would prefer to book using the online request form? | ✓ | 58.4% | 58.4% |
| What percentage of the patients sent the text message completed the survey? | ✓ | 9% | 9% |
Traps in the data 4 of 7 handled
- ✓Patients aged 65 or over are 24% of registered adults but 9.8% of respondents (text link, online only). They are less satisfied and prefer the phone, so weighting to the age mix lowers satisfaction among those who tried to book from 50.8% to 46.3% and raises the phone preference from 40.7% to 45.0%.
- ✗Q2 satisfaction (50.8%) is asked only of the 549 who tried to book. 24.3% of all respondents needed an appointment but did not try to book (35.1% of those who did not try to book); those who did not try are left out of the satisfaction base, which flatters it.
- ✗Last year's 71% came from a paper survey handed out in the waiting room before the change, to patients who had got an appointment; this year's 50.8% is from an online survey by text of those who tried to book. Different method and different people.
- ✓The practice manager believes patients prefer the new system. Unweighted, the online form leads (46.2% against 40.7% for phone), but only 18-44s prefer it (58.4%); weighted to the age mix, preference is split (40.7% online, 45.0% phone, 6.7% in person).
- ✗Only 13 respondents need an interpreter; 61.5% of them would prefer to book in person.
- ✓Among those who tried to book, women look less satisfied than men (47.5% against 55.3%, bases 303 and 237), but the difference is not statistically significant at 95%.
- ✓Only 886 of 9,840 patients texted completed (about 9%), and the 7,260 registered adults without a mobile number on record were never invited; patients who cannot or do not go online are the least likely to have answered.
The analysis
Do not keep the current online-first system as it is, and do not restrict phone booking solely to over-75s, disabled patients, and limited-English patients. Instead, adopt a hybrid model: (1) Restore phone booking access for all patients who need or prefer it, while actively encouraging and improving the online route for the ~46% who prefer it; (2) Provide dedicated in-person reception booking and priority/assisted support (including translation, carer/proxy booking, and longer call handling) for over-75s, disabled patients, and patients needing an interpreter; and (3) Immediately overhaul the online booking workflow by shortening the form and establishing a same-day response SLA. Restricting phone lines only to 75+ and registered disabled/interpreter patients would alienate 45–74-year-olds (where 49% to 63% prefer phone and satisfaction is 18% to 44%), create burdensome 'gatekeeping' at reception for patients with hidden disabilities, and fail to solve the core operational bottleneck (multi-day response delays to online forms).
Evidence from practice telephony and clinical capacity modelling showing that opening phone booking to all patients would overwhelm reception staffing and increase clinical triage delays even after streamlining the online form and guaranteeing same-day online responses. If capacity strictly prohibits universal phone booking, the fallback would be expanding the dedicated phone/in-person eligibility criteria to include all patients aged 65+ (not just 75+) plus self-declared accessibility/digital-exclusion needs without requiring proof of disability, paired with a rapid trial of same-day online triage.
Findings
-
To the Practice Manager ('Patients prefer the new system, don't they? Can I tell the partners online-first is working?'): No. Even in a survey distributed by text message—which heavily over-represents younger, digitally connected patients—only 46.2% prefer the online form, while 40.7% prefer booking by phone and 4.6% in person. When weighted to the actual age profile of registered adults, preference for booking by phone (45.0%) exceeds preference for the online form (40.7%). Furthermore, 24.3% of respondents needed an appointment in the last three months but did not try to book one (an age-weighted equivalent of ~4,412 registered adults), primarily because they expected to wait too long for a reply (43.7% of those put off) or did not think they would get an appointment (28.8%).
Unweighted preference (Q6, n=886): 46.2% online, 40.7% phone, 4.6% in person at reception. Age-weighted preference across the 17,100 adult population (44% 18–44, 32% 45–64, 13% 65–74, 11% 75+): 45.0% phone, 40.7% online, 7.6% in person. Unmet need (Q4, n=886): 24.3% unweighted (25.8% age-weighted, ~4,412 adults) needed an appointment but did not try to book; among those who did not try to book at all in the last 3 months (n=313), 35.1% had an unmet need. Top barriers (Q5, n=215): expected to wait too long for a reply (43.7%), did not think they would get an appointment (28.8%), online form difficult to use (16.7%).
-
To the GP Partner ('Last year 71% were satisfied with booking. Has satisfaction fallen since we went online?'): Yes, satisfaction has fallen substantially, though the exact percentage-point drop must be treated with caution due to a change in survey methodology. Among respondents who tried to book, 50.8% are satisfied (95% CI: 46.6% to 55.0%), down 20.2 percentage points from 71% last year. Because older patients are severely under-represented in this SMS sample (9.8% aged 65+ in the sample vs 24.0% in the practice population), age-weighting reduces overall satisfaction further to 46.3% (or 42.3% if weighted by age and booking incidence). Satisfaction collapses among older adults: just 17.8% of 65–74s and 13.0% of 75+s are satisfied, while 60.0% and 73.9% respectively are dissatisfied.
Q2 satisfaction (n=549 triers): 50.8% net satisfied (29.3% fairly, 21.5% very), 33.9% net dissatisfied (19.3% fairly, 14.6% very). By age: 18–44 is 64.7% satisfied (n=272); 45–64 is 44.0% (n=209); 65–74 is 17.8% (n=45); 75+ is 13.0% (n=23). Last year's survey (n=412, waiting-room paper survey) recorded 71% satisfied. While waiting-room surveys exclude patients who fail to get an appointment and SMS surveys exclude those without mobile phones (42% of practice adults), the severe dissatisfaction among 45+ and disabled patients confirms a genuine operational problem.
-
A restricted phone and reception service (only for over-75s, disabled patients, and limited-English patients) will leave the majority of phone-dependent patients behind. Strong preference for phone booking and low satisfaction with online-first extend heavily into the 45–64 and 65–74 age bands. In the open-text responses, 85% of the requests for phone booking (64 out of 75) came from patients aged under 65.
Among 65–74s (n=54, excluded from an over-75s cutoff), 63.0% prefer phone booking, 14.8% prefer reception, only 16.7% prefer the online form, and 72.2% of those put off from booking (n=18) cited lack of confidence going online (44.4%) or form difficulty (27.8%). Among 45–64s (n=337), more prefer phone (49.0%) than online (38.0%), and satisfaction is only 44.0% (n=209). Even among non-disabled respondents (n=784), 39.5% prefer booking by phone. Open-text Q3: 75 respondents asked for phone booking; only 11 were aged 65+ and 64 were aged 18–64, including several explicitly asking not to restrict phone lines to specific groups (e.g., A039, A583, A659).
-
To the PPG Chair ('Are women, or patients who need an interpreter, finding it harder to book than others?'): For women vs men, the 7.8 percentage-point gap in satisfaction (47.5% female vs 55.3% male) is not statistically significant at the 95% confidence level (p = 0.071), so we cannot be certain women find it harder overall, though it warrants monitoring. For patients needing an interpreter, the sample is very small (n=13) because the survey was sent in English by SMS, but the signal is stark: 61.5% prefer to book in person at reception and only 7.7% (1 person) prefer the online form.
Sex difference on Q2: Female 47.5% satisfied (n=303) vs Male 55.3% satisfied (n=237); z = -1.80, p = 0.071 (two-tailed). Interpreter needed on Q6 (n=13): 61.5% (8/13) prefer in-person at reception (vs 3.8% of those not needing an interpreter, n=873), 23.1% (3/13) prefer phone, 7.7% (1/13) prefer online. Open-text answers (A112, A756) highlight fear of making medical errors in English on the form and reliance on family members to translate.
-
Patients with disabilities experience major usability, sensory, cognitive, and motor barriers with the online-first system, driving high demand for both phone (50.0%) and in-person reception booking (15.7%).
Q6 preference among disabled respondents (n=102): 50.0% phone, 15.7% in person at reception, 23.5% online (compared to 49.1% online and 3.2% in person among non-disabled respondents, n=784). Q2 satisfaction among disabled triers (n=65): 40.0% satisfied vs 52.3% non-disabled (n=484); only 10.8% of disabled triers are 'very satisfied' vs 22.9% of non-disabled. Open-text Q3 responses from 21 disabled patients detail specific barriers: screen reader/JAWS compatibility failure (A237), visual impairment/small grey text (A334, A565, A713, A742), tremors/Parkinson's/MS/stroke affecting fine motor control (A193, A221, A228, A258, A443), hearing impairment/BSL needs requiring face-to-face or video relay (A265, A375, A789), and dementia/stroke memory loss/learning disabilities/ME brain fog requiring human assistance or save-and-return functionality (A017, A189, A262, A320, A339, A358, A437, A512).
-
For patients willing to book online, the primary failure is not the concept of online booking, but slow practice turnaround times (waiting 24 hours to 5 days for a reply) and an overly long, repetitive form.
Among all respondents put off from booking (Q5, n=215), 'I expected to wait too long for a reply' is the #1 barrier at 43.7% (rising to 51.2% among 45–64s). 'The online form is difficult to use' was cited by 16.7% (rising to 27.8% of 65–74s and 27.3% of 75+s). In Q3 open-text (n=246), 37 respondents complained about slow response times (citing waits of 24 hours to 5 days and wanting same-day or within-hours confirmation) and 23 criticized the form's length, repetition, or complexity.
Caveats
- Severe coverage and channel bias: The survey was fielded exclusively by SMS to the 58% of registered adults (9,840 of 17,100) with a mobile number on record. The remaining 42% (7,260 adults) without a recorded mobile number were completely excluded and are far more likely to be older, digitally excluded, or phone/in-person dependent. Consequently, true whole-practice preference for phone/in-person booking is almost certainly higher, and true satisfaction lower, than even the age-weighted figures suggest.
- Age skew and small subgroup bases: Only 9.8% of respondents were aged 65+ (54 aged 65–74; 33 aged 75+), compared to 24.0% of the registered adult population. Among those who tried to book (Q2), the base for 75+ is just 23, and for Q5 (reasons for not trying) the bases for 65–74 (n=18) and 75+ (n=11) are very small.
- Non-response bias: The completion rate was 9.0% of those texted (886/9,840), leaving room for self-selection bias among patients with strong positive or negative views.
- Language barrier in sampling: Fielded only in English via text message, the survey captured only 13 respondents who need an interpreter (1.5% of sample), likely under-representing the true scale and severity of barriers for patients with limited English proficiency.
- Incomparable historical baseline: Last year's 71% satisfaction figure came from an October 2025 paper survey handed out in the waiting room (n=412). Waiting-room surveys only capture patients who successfully navigated the booking system to attend an appointment, whereas this SMS survey includes patients who tried and failed or had poor remote experiences.
Next steps
- decisionRedesign and shorten the online request form, and fix critical accessibility bugs (screen-reader button labels, high-contrast/large text, minimal scrolling, save-and-return, and proxy/carer submission).
- decisionImplement an operational SLA for online requests: immediate automated receipt confirmation with a clear timeframe, and clinical/admin triage response within the same working day (or within 4 business hours for morning submissions).
- decisionRestore morning/all-day phone booking for all patients alongside dedicated in-person reception assistance and interpreter/language support (including translated form options or language-line integration at reception).
- experimentAdd a 'Preferred/Usual GP' selection field to both the online form and reception booking script for non-urgent appointments.
- researchConduct targeted offline research (waiting-room paper pulse and 10–15 outbound phone/interpreter-assisted interviews) with the 42% of adults who have no mobile number on record and patients requiring interpreters.
- monitorMonitor booking channel split, call-queue wait times, online turnaround times, and satisfaction by age, sex, and disability status monthly.
Open-text themes it coded
Restore phone booking / speak to a person on the phone 97Faster response / same-day turnaround to online requests 45Shorten and simplify the online form (fewer, less repetitive questions) 29Accessibility, digital inclusion, and language adaptations (disability, sensory/motor/cognitive, interpreter/translation, proxy, in-person help) 24Continuity of care / seeing the same or regular GP 15Satisfied with the online form / works well as it is 45No suggestions / rarely use the practice 32
The survey it planned
2 screening questions and 12 questions, as the model wrote them.
-
S1
Are you currently registered as a patient at Ashgrove Medical Group?
One answer- Yes, I am a registered patient
- Yes, I am answering on behalf of a registered patient (for example, as a carer or family member)
- No
Continues if Yes, I am a registered patient; Yes, I am answering on behalf of a registered patient (for example, as a carer or family member)
-
S2
Which age group is the patient in? (If you are answering for someone else, please give their age.)
One answer- Under 18
- 18 to 44
- 45 to 64
- 65 to 74
- 75 or over
Continues if 18 to 44; 45 to 64; 65 to 74; 75 or over
-
Q1
Since April 2026, have you tried to book or request an appointment at Ashgrove Medical Group?
One answer- Yes
- No, I have not needed an appointment since April 2026
- No, I needed an appointment since April 2026, but I did not try to book one
-
Q2
The last time you tried to book or request an appointment since April 2026, how did you contact the practice?
Any that apply- Filled in the online request form on the practice website or NHS App
- Phoned reception
- Went into the practice in person to speak to reception
- Someone else (such as a family member or carer) contacted the practice for me
- Other
Routing Shown only if Q1 is: Yes
-
Q3
Overall, how satisfied or dissatisfied were you with your experience of booking or requesting an appointment at Ashgrove Medical Group since April 2026?
One answer- Very satisfied
- Fairly satisfied
- Neither satisfied nor dissatisfied
- Fairly dissatisfied
- Very dissatisfied
Routing Shown only if Q1 is: Yes
-
Q4
Thinking about the last time you contacted the practice since April 2026, how easy or difficult was it to explain what you needed and get a response?
One answer- Very easy
- Fairly easy
- Neither easy nor difficult
- Fairly difficult
- Very difficult
Routing Shown only if Q1 is: Yes
Judges: double-barrelledOne rating covers both explaining what was needed and getting a response. These could have been very different experiences.
Asks how easy it was to explain what was needed and how easy it was to get a response, with only one rating for both.
-
Q5
Since April 2026, has there been a time when you needed an appointment at the practice, but were put off or unable to ask for one?
One answer- Yes
- No
Routing Shown only if Q1 is: Yes
-
Q6
Why were you put off or unable to ask for an appointment? (Please select all that apply.)
Any that apply- I do not have internet access, a smartphone, or a computer
- I found the online request form too difficult or confusing to use
- The online request form was closed or unavailable when I tried
- I tried to phone, but the wait was too long or I could not get through
- I was told on the phone or at reception that I had to use the online form
- I find it difficult to explain my health problem in writing or in English
- I did not know when the practice would call or text me back
- I felt my privacy was at risk explaining my issue on the form or at the reception desk
- Another reason
Routing Shown only if Q1 is: Yes; No, I needed an appointment since April 2026, but I did not try to book one
Judges: assumptiveThe routing shows this question to everyone who tried booking, including those who answered No to Q5. It assumes they were put off or unable to ask, and provides no option to say this did not happen. Needing an appointment but not trying to book also does not necessarily establish that someone was put off or unable to ask.
Display logic is keyed to Q1, not Q5. Respondents who answered No at Q5 (never put off) are still asked why they were put off.
-
Q7
When you need to book or request an appointment at the practice in future, how would you most prefer to do it?
One answer- By filling in an online form (with a text or phone reply from the practice)
- By phoning the practice to speak to a receptionist
- In person at the reception desk
- I have no preference between online and phone
- Not sure
-
Q8
How ought Ashgrove Medical Group organise appointment booking for patients in future? Please select the option you think is best overall.
One answer- Keep the current online-first system as it is (most patients use the online form; phone lines are kept for those who cannot go online)
- Keep the online form, and add a dedicated phone line and in-person help at reception specifically for patients aged 75 and over, disabled patients, and those who need help with English
- Bring back a morning phone booking line for any patient who wants to call, alongside the online form
- Not sure
-
Q9
Which of the following, if any, would make it easier for you to book or request an appointment at Ashgrove Medical Group? (Please select up to three.)
Any that apply- A simpler or shorter online request form
- Keeping the online request form open for longer hours each day
- Being able to book routine appointments directly by phone at any time
- A morning phone line open to all patients to book appointments
- A priority phone line for older patients, disabled patients, or those who cannot get online
- Staff or tablets available at the reception desk to help fill in the request in person
- Clearer information on when the practice will reply to an online request
- Option to complete the online form in other languages or easily book an interpreter
- Nothing — booking is already easy for me
-
Q10
In your own words, what is the one main thing Ashgrove Medical Group could do to improve how you book an appointment?
Open text -
Q11
Do you have any physical or mental health conditions or disabilities that have lasted, or are expected to last, 12 months or more and reduce your ability to carry out day-to-day activities?
One answer- Yes, a lot
- Yes, a little
- No
- Prefer not to say
-
Q12
How comfortable are you speaking and reading English, and using the internet on a smartphone or computer?
Any that apply- English is not my main language and I sometimes need an interpreter or help with written English
- I find it difficult or impossible to use the internet on a smartphone, tablet, or computer without help
- Neither of these applies to me — I am comfortable with English and using the internet
- Prefer not to say
Sample plan
INVITATION & CHANNEL: Send an SMS invitation with a link to the online questionnaire to all ~17,100 registered adult patients (18+) with a mobile number on record. Allow carers or family members receiving the text on a patient's behalf to complete the survey for that patient (captured in S1).
SAMPLE SIZE & PRECISION CONFLICT: To achieve a margin of error of ±5 percentage points at 95% confidence (assuming p=0.50) within EACH of the four age bands (18–44, 45–64, 65–74, 75+), the minimum required sample size per band (with finite population correction for a 17,100 adult practice) is approximately 350 to 375 completes per age band — requiring ~1,450 total completes across all four bands. If the total sample is strictly capped at 900 completes (225 per age band), the margin of error per age band will be ±6.5 percentage points at p=0.50 (or ±5.0 percentage points only when a proportion is near 80%/20%). Therefore, we recommend setting targets of 365 completes per age band (1,460 total completes — which incurs zero extra SMS distribution cost once texts are sent). If the 900 cap is fixed for budget or processing reasons, set interlocking equal quotas of n=225 per age band (18–44, 45–64, 65–74, 75+) and apply post-stratification weights to match the practice's true adult age/sex/deprivation register for overall practice estimates.
MITIGATING MODE & NON-RESPONSE BIAS: Because an online-only survey sent via SMS inherently under-represents digitally excluded patients, patients without mobile phones, over-75s, and patients with limited English — the exact groups central to the decision — the practice should: (1) include a clear message in the SMS that a relative/carer can help complete it, (2) offer translated landing pages for the practice's top 3 non-English languages, and (3) supplement the SMS survey with paper copies/tablet assisted completion in the waiting room and via reception phone lines for 50–100 digitally excluded/75+ patients, recording mode to compare against SMS responses.