Benchmarks / SurveyBench / How Ashgrove patients find online-first booking / DeepSeek V4.1 Flash 1017

Measured by Spring Prompt

DeepSeek V4.1 Flash 1017: How Ashgrove patients find online-first booking

The decision: whether to keep online-first booking as it is, restore a morning phone booking line for every patient, or add a dedicated phone line and in-person help at reception only for patients who need them (over-75s, patients with a disability, and patients with limited English). The brief and the data →

Verdict
✓ Sound research
Research score
100 of 100
Analysis rating
1,221
Head to head, this task
won 11 of 13

Scorecard

Numbers 11 of 11 right

QuestionIts answerTrue
Among respondents who tried to book in the last three months, what percentage were satisfied (fairly or very) with the booking process?✓50.8%50.8%
Weighting each age band's percentage satisfied (fairly or very, among those who tried to book) by that band's share of registered adults, what is the overall percentage satisfied?✓46.3%46.3%
What percentage of all respondents said there was a time in the last three months when they needed an appointment but did not try to book one?✓24.3%24.3%
Weighted to the age mix of registered adults, what percentage would prefer to book by phone?✓45%45%
What is the 95% margin of error, in percentage points, on the unweighted percentage satisfied among respondents who tried to book?✓4.2 points4.18 points
Among respondents who tried to book, is the difference in the percentage satisfied (fairly or very) between female and male respondents statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
How many of the open-text answers ask to be able to book by phone, or for a phone booking line?✓9497
How many of the open-text answers from respondents aged 65 or over ask to be able to book by phone, or for a phone booking line?✓1113
Weighted to the age mix of registered adults, about how many of the practice's 17,100 registered adults would say there was a time in the last three months when they needed an appointment but did not try to book one?✓4,3924,393
Among respondents aged 18 to 44, what percentage would prefer to book using the online request form?✓58.4%58.4%
What percentage of the patients sent the text message completed the survey?✓9%9%

Traps in the data 7 of 7 handled

  • ✓Patients aged 65 or over are 24% of registered adults but 9.8% of respondents (text link, online only). They are less satisfied and prefer the phone, so weighting to the age mix lowers satisfaction among those who tried to book from 50.8% to 46.3% and raises the phone preference from 40.7% to 45.0%.
  • ✓Q2 satisfaction (50.8%) is asked only of the 549 who tried to book. 24.3% of all respondents needed an appointment but did not try to book (35.1% of those who did not try to book); those who did not try are left out of the satisfaction base, which flatters it.
  • ✓Last year's 71% came from a paper survey handed out in the waiting room before the change, to patients who had got an appointment; this year's 50.8% is from an online survey by text of those who tried to book. Different method and different people.
  • ✓The practice manager believes patients prefer the new system. Unweighted, the online form leads (46.2% against 40.7% for phone), but only 18-44s prefer it (58.4%); weighted to the age mix, preference is split (40.7% online, 45.0% phone, 6.7% in person).
  • ✓Only 13 respondents need an interpreter; 61.5% of them would prefer to book in person.
  • ✓Among those who tried to book, women look less satisfied than men (47.5% against 55.3%, bases 303 and 237), but the difference is not statistically significant at 95%.
  • ✓Only 886 of 9,840 patients texted completed (about 9%), and the 7,260 registered adults without a mobile number on record were never invited; patients who cannot or do not go online are the least likely to have answered.

The analysis

Recommendation

Reject keeping online-first unchanged. Implement the targeted phone line and in-person reception help now for over-75s, patients with a disability and patients with limited English, but do not make it hard to access: allow any patient who cannot complete the online form to use it. In parallel, shorten and simplify the form, add accessibility features (screen reader, large text, translation, save-and-return, carer/proxy booking) and set a same-day response standard. Because phone demand is broader than the three groups (age-weighted phone preference 45.0% vs online 40.7%; 94 explicit phone requests, many from younger adults), run a time-limited pilot of a morning phone line open to all or a universal callback, then decide whether to restore universal morning phone booking. Do not present 50.8%/46.3% satisfaction as evidence of a fall from 71%, or as proof that online-first is working for all patients.

What would change it

A representative mixed-mode survey including non-mobile and offline patients that shows online satisfaction above 70% and phone preference below 20% would support keeping online-first with only minimal targeted help. A pilot showing that a universal morning phone line cannot be staffed without damaging online response times and appointment capacity would also argue against universal restoration. Conversely, if the targeted line has low reach among older, disabled or interpreter patients, or if a universal morning line improves satisfaction without reducing online performance, the recommendation should shift to restoring universal morning phone booking.

Findings

  1. The practice manager cannot tell the partners that online-first is working across the patient population: satisfaction among people who tried to book is only about half, and age-weighted phone preference is higher than online preference.

    Medium confidenceBase 886

    Among 549 respondents who tried to book, 50.8% were fairly or very satisfied; weighting by the registered adult age mix gives 46.3%. Among all 886 respondents, 46.2% prefer online and 40.7% prefer phone; age-weighted phone preference is 45.0% versus 40.7% online. Open text includes 94 explicit phone/phone-line requests out of 246 comments.

  2. Older patients are much less satisfied and much more likely to prefer phone or in-person booking; this is the clearest equity signal in the survey.

    High confidenceBase 549

    Satisfied among triers: 18-44 64.7% (272), 45-64 44.0% (209), 65-74 17.8% (45), 75+ 13.0% (23). Prefer phone: 65-74 63.0%, 75+ 72.7%; prefer online: 16.7% and 6.1%. Q4 put-off yes is 33.3% in both 65-74 and 75+ groups.

  3. Disabled patients show a meaningful access gap; interpreter evidence is directionally similar but rests on only 13 respondents and cannot bear much weight alone.

    Medium confidenceBase 102

    Satisfied: disability yes 40.0% (65) vs no 52.3% (484). Prefer online 23.5% vs 49.1%; prefer phone 50.0% vs 39.5%; prefer in person 15.7% vs 3.2%. Interpreter yes n=13: 61.5% prefer in person, 23.1% phone, 7.7% online; satisfaction split not shown.

  4. Women are somewhat less satisfied than men, but the difference is not statistically significant at the 95% level; this survey cannot establish that women find booking harder.

    Medium confidenceBase 540

    Female satisfied 47.5% (303) vs male 55.3% (237), a gap of about 7.8 percentage points; a two-proportion test gives p about 0.07, so not significant at 95%. Q6 preference is not tabulated by sex.

  5. About a quarter of adults needed an appointment in the last three months but did not try to book; weighted to the practice list this is roughly 4,400 adults.

    Medium confidenceBase 886

    Q4 yes is 24.3% of 886; weighting by registered adult age mix gives 25.7%, so about 4,392 of 17,100 adults. Among 215 non-triers, reasons include expected wait too long 43.7%, did not think they would get an appointment 28.8%, got help elsewhere 22.8%, online form difficult 16.7%, not confident online 9.3% (44.4% of 65-74 and 36.4% of 75+).

  6. The 71% from last year cannot be compared with this survey; the apparent fall is not evidence of a real fall in satisfaction.

    High confidenceBase 886

    The 2025 figure is from a paper waiting-room questionnaire (n=412) before online-first; the 2026 survey is text-message only to mobile-number patients, with 886 completes and a 9.0% completion rate. The sample is 52.1% aged 18-44 vs 44% of registered adults and 3.7% aged 75+ vs 11%. Current satisfaction is 50.8% unweighted and 46.3% age-weighted.

  7. Open text confirms two operational fixes that can be acted on now: simplify the online form and respond faster. Phone access is not a niche request.

    Medium confidenceBase 246

    Of 246 comments, 94 explicitly ask for phone booking/a phone line (11 from respondents aged 65+); at least 31 mention form length/complexity and 42 mention response speed. Q5 also shows 43.7% of non-triers expected to wait too long for a reply.

Caveats

  • The survey was text-message only to the 9,840 registered adults with a mobile number on record (58% of adults), so patients without a mobile or who do not use text are excluded; these are likely to include some of the oldest and most digitally excluded patients.
  • Only 886 completed from 9,840 sent (9.0%), and respondents are self-selected. The sample is much younger than the registered adult list: 52.1% aged 18-44 vs 44%, and 3.7% aged 75+ vs 11%. Unweighted tables therefore over-represent younger patients.
  • Many subgroup bases are small: 65-74 n=54, 75+ n=33, disability yes n=102, interpreter yes n=13. Age-weighted estimates are provided, but weighting does not fix nonresponse bias and cannot correct for mobile-only exclusion.
  • Last year's 71% is not comparable: it came from a paper questionnaire handed out in the waiting room before online-first, not from a text-message survey after the change. It cannot be used to estimate a trend.
  • Q2 is asked only of people who tried to book, so it measures experience among users, not the whole patient population. Q4/Q5 rely on three-month recall and Q5 reasons are not mutually exclusive.
  • The survey does not measure actual booking mode used, appointment outcome, call waiting times, online form completion rates, or whether patients were offered help. These operational data are needed alongside patient-reported data.
  • Open-text comments are not representative and were not double-coded; counts are comments, not patients, and some respondents mention more than one theme. The phone count includes explicit phone/call/ring/telephone/phone-line requests, not every request to speak to a person.
  • The sex comparison is not statistically significant and Q6 preference is not tabulated by sex, so the chair's question about women cannot be answered fully. Interpreter satisfaction is not tabulated.

Next steps

  • decisionApprove targeted phone and in-person reception support for over-75s, disabled patients and limited-English patients, with a simple cannot-use-online exemption for anyone else.
  • experimentPilot a morning phone line open to all, or a universal callback option, for 3 months alongside the targeted offer.
  • decisionRedesign the online request form to remove repetition and unnecessary questions, and add accessibility and language options, with a same-day response standard.
  • researchRun qualitative interviews or focus groups with older, disabled, interpreter and offline patients, plus a translated and telephone/paper survey.
  • monitorSet up monthly equity monitoring by age, sex, disability, interpreter need and booking mode, and repeat a comparable mixed-mode survey with weights.

Open-text themes it coded

Wants phone booking or a dedicated phone line 94Shorter, simpler or less repetitive online form 26Faster response or same-day reply to online requests 42Continuity of care or same GP 15Accessibility, digital exclusion, assisted booking or language support 22Positive about current online form or no change needed 46No suggestion, rarely uses services or no opinion 31Wants in-person reception help or face-to-face booking 4Wants phone access for everyone, not only selected groups 5

The survey it planned

2 screening questions and 12 questions, as the model wrote them.

  1. S1

    Are you registered as a patient at Ashgrove Medical Group, and answering about your own care?

    One answer
    • Yes
    • No, I am answering on behalf of someone else
    • No, I am not registered at Ashgrove Medical Group

    Continues if Yes

  2. S2

    How old are you?

    One answer
    • 18 to 44
    • 45 to 64
    • 65 to 74
    • 75 or over
    • Under 18
    • Prefer not to say

    Continues if 18 to 44; 45 to 64; 65 to 74; 75 or over; Prefer not to say

  3. Q1

    Since April 2026, have you tried to book an appointment at Ashgrove Medical Group?

    One answer
    • Yes, more than once
    • Yes, once
    • No
    • Not sure or can't remember
  4. Q2

    Since April 2026, how have you asked for an appointment? Please choose all that apply.

    Any that apply
    • Using the online request form on the practice website
    • Phoning reception
    • Coming to reception in person
    • Someone else asked for me
    • Another way
    • I have not asked for an appointment since April 2026

    Routing Shown only if Q1 is: Yes, more than once; Yes, once

  5. Q3

    Overall, how satisfied or dissatisfied are you with the way you book appointments at Ashgrove Medical Group since April 2026?

    Scale

    Scale 1-5: Very dissatisfied to Very satisfied

    Routing Shown only if Q1 is: Yes, more than once; Yes, once

  6. Q4

    How easy or difficult did you find the online request form to use?

    Scale

    Scale 1-5: Very difficult to Very easy

    Routing Shown only if Q2 is: Using the online request form on the practice website

  7. Q5

    Since April 2026, have there been times when you needed to see a GP or nurse but did not contact the practice to ask for an appointment?

    One answer
    • Yes
    • No
    • Not sure or can't remember
  8. Q6

    What stopped you from contacting the practice? Please choose up to three reasons.

    Any that apply
    • I did not think my problem was serious enough
    • I thought I would not get an appointment
    • The online form was hard to use
    • I do not have internet access or a smartphone
    • I do not like using online forms
    • Reading or writing in English is difficult for me
    • I needed help from someone to fill in a form
    • I was worried about being asked to phone the practice back
    • I did not know how to ask for an appointment
    • It was not a good time or not convenient
    • Something else
    • Prefer not to say

    Routing Shown only if Q5 is: Yes

  9. Q7

    If you could choose, how would you prefer to ask for an appointment at Ashgrove Medical Group?

    One answer
    • Filling in the online request form
    • Phoning the practice
    • Coming to reception in person
    • No preference
  10. Q8

    Which of these should Ashgrove Medical Group offer? Please choose all that apply.

    Any that apply
    • The online request form, open to all patients, as it is now
    • A phone line open to all patients every morning
    • A separate phone line and in-person help at reception for patients who need extra help, for example people aged 75 or over, people with a disability, and people who need an interpreter
    • None of these
    • Don't know
  11. Q9

    What would make booking an appointment easier for you? Please choose all that apply.

    Any that apply
    • Being able to send a request online at any time
    • Being able to speak to someone on the phone
    • Being able to book at reception in person
    • Choosing a specific time for my appointment online
    • A shorter or simpler online form
    • Someone to help me fill in the form
    • Information in my language or an interpreter
    • Text message updates about my request
    • More appointments being available
    • Something else
    • Nothing, booking is fine as it is
  12. Q10

    How confident are you using the internet on a phone, tablet or computer?

    Scale

    Scale 1-5: Not at all confident to Very confident

  13. Q11

    Do you have a disability or health condition that makes it harder for you to contact the practice by phone or online, or to read or fill in forms?

    One answer
    • Yes
    • No
    • Prefer not to say
  14. Q12

    Do you need information in a language other than English, or an interpreter, when dealing with the practice?

    One answer
    • Yes
    • No
    • Prefer not to say
Sample plan

Population: all 17,100 registered adults. Invite every adult with a mobile number on record by text message with a unique survey link (practice estimate roughly 14,500 to 15,500 adults; confirm the exact count before launch). Fieldwork 3 weeks with one reminder text at day 7, no incentive, and a second targeted reminder at day 14 sent only to age bands still below quota (the practice can target by recorded age).

PRECISION FEASIBILITY - flag to partners before launch: plus or minus 5 percentage points at 95% confidence within each of four age bands needs about 384 completes per band (worst case p=0.5, finite population correction negligible at 21,000), i.e. about 1,540 completes in total. 900 completes gives about 225 per band and a margin of about plus or minus 6.5 points per band. Three options: (a) accept plus or minus 6.5 points and report it honestly; (b) extend fieldwork and reminder waves to reach about 1,540 completes to deliver plus or minus 5 points; (c) commission the full 900 now and report the two older bands (65-74, 75+) at plus or minus 5 points by giving them 384 each and the two younger bands at about plus or minus 7 points. Recommendation: (b) if the budget allows, otherwise (c), which protects the groups the decision is about.

Quotas: minimum 225 completes per age band as a floor, with 65-74 and 75+ actively boosted; cap 'Prefer not to say' on age at 5% of completes and report it as a separate unweighted cell (cannot contribute to band estimates). No other quotas. Weight final data to the practice's registered adult age-by-sex profile to correct differential response.

Coverage gap that must be acknowledged: a text-only invitation systematically misses adults with no mobile on record (likely 1,600 to 2,600 patients, concentrated in the 75+ and limited-English groups) - exactly the patients the decision is about. Recommend adding a short postal or telephone version for patients without a mobile number, or at minimum reporting results against the known age profile of that excluded group and caveating conclusions for it.

Expected response: 6-10% of invitees, i.e. roughly 900-1,500 completes over three weeks - sufficient for the 900 target and plausibly for the 1,540 target with the targeted reminder.