Benchmarks / SurveyBench / How Ashgrove patients find online-first booking / Qwen4 Max

Measured by Spring Prompt

Qwen4 Max: How Ashgrove patients find online-first booking

The decision: whether to keep online-first booking as it is, restore a morning phone booking line for every patient, or add a dedicated phone line and in-person help at reception only for patients who need them (over-75s, patients with a disability, and patients with limited English). The brief and the data →

Verdict
✗ Not sound
Research score
86 of 100
Analysis rating
1,373
Head to head, this task
won 10 of 14

Why it is not sound

The analysis

  • Got at least one set number wrong
  • Reported a material finding the data does not support
  • Made a recommendation that does not follow from the data
  • Proposed a mistaken next step: Restore a morning phone booking line for every patient on the strength of this survey alone

Findings the judges found unsupported

  • The practice manager cannot tell the partners that 'patients prefer the new system'. Preference is split, and once the sample is corrected for age, phone is the single most preferred route. Online-first is working well for a large, younger, digitally confident group and badly for everyone else — that is the finding to report, not a verdict of overall success.The statement that online-first works badly for everyone outside a younger, digitally confident group is too categorical. The tables show differences in satisfaction and preference, not uniform failure for all other patients, and digital confidence was not measured across the sample.
  • A phone line restricted to over-75s, disabled patients and interpreter users would miss most of the demand for phone booking. Phone preference is a mainstream preference, not only an access need — so the targeted option should be treated as a floor to build on, not the whole answer.The analysis establishes that most phone preferrers are under 65, not that most would be excluded by the targeted option. Disability and interpreter needs overlap with younger age groups, and the necessary joint breakdowns are absent. Stated phone preference also does not establish actual demand for a particular phone service.
  • Around 4,400 adults had a need they never brought to the practice in three months. Satisfaction questions cannot see these people at all, and they are concentrated among those who never even tried to book. This is the strongest equity argument for change and the metric to monitor afterwards.Satisfaction questions do not exclude everyone who needed an appointment but did not try on some occasion. Of the 215 reporting such an occasion, about 94 also tried to book on another occasion and therefore answered Q2. The excluded group is non-triers, not all patients reporting unmet need.
  • The survey systematically under-samples exactly the patients the targeted option is designed for, so it almost certainly understates the access problem. Treat every unweighted total as an optimistic upper bound.Coverage bias makes optimistic satisfaction estimates plausible, but it does not establish that every unweighted total is an optimistic upper bound. Nor are population disability or interpreter-need benchmarks supplied to establish under-representation of those groups.
  • Answer to the PPG chair, part 3: interpreter users cannot be quantified here (13 respondents), but the two open-text accounts and the preference pattern point to a specific, plausible barrier — an English-only clinical form. Act on the mechanism now and verify with targeted research; do not quote a percentage for this group.The claim that 1.5% is very likely a large undercount of interpreter need has no population benchmark. The two accounts support investigating language barriers, but not a conclusion about the magnitude of undercounting.
  • Older patients' problem is confidence and human contact, not form usability alone — a shortened form will not make online-first acceptable to them. Any digital-only future for over-75s is unsupported by these data.The evidence supports offering older patients human alternatives, but cannot establish that simplifying the form would not make online-first acceptable to them. That is an untested intervention claim resting partly on very small bases.

Scorecard

Numbers 10 of 11 right

QuestionIts answerTrue
Among respondents who tried to book in the last three months, what percentage were satisfied (fairly or very) with the booking process?✓50.8%50.8%
Weighting each age band's percentage satisfied (fairly or very, among those who tried to book) by that band's share of registered adults, what is the overall percentage satisfied?✓46.3%46.3%
What percentage of all respondents said there was a time in the last three months when they needed an appointment but did not try to book one?✓24.3%24.3%
Weighted to the age mix of registered adults, what percentage would prefer to book by phone?✓45%45%
What is the 95% margin of error, in percentage points, on the unweighted percentage satisfied among respondents who tried to book?✓4.2 points4.18 points
Among respondents who tried to book, is the difference in the percentage satisfied (fairly or very) between female and male respondents statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
How many of the open-text answers ask to be able to book by phone, or for a phone booking line?✓9497
How many of the open-text answers from respondents aged 65 or over ask to be able to book by phone, or for a phone booking line?✗1013
Weighted to the age mix of registered adults, about how many of the practice's 17,100 registered adults would say there was a time in the last three months when they needed an appointment but did not try to book one?✓4,3924,393
Among respondents aged 18 to 44, what percentage would prefer to book using the online request form?✓58.4%58.4%
What percentage of the patients sent the text message completed the survey?✓9%9%

Traps in the data 7 of 7 handled

  • ✓Patients aged 65 or over are 24% of registered adults but 9.8% of respondents (text link, online only). They are less satisfied and prefer the phone, so weighting to the age mix lowers satisfaction among those who tried to book from 50.8% to 46.3% and raises the phone preference from 40.7% to 45.0%.
  • ✓Q2 satisfaction (50.8%) is asked only of the 549 who tried to book. 24.3% of all respondents needed an appointment but did not try to book (35.1% of those who did not try to book); those who did not try are left out of the satisfaction base, which flatters it.
  • ✓Last year's 71% came from a paper survey handed out in the waiting room before the change, to patients who had got an appointment; this year's 50.8% is from an online survey by text of those who tried to book. Different method and different people.
  • ✓The practice manager believes patients prefer the new system. Unweighted, the online form leads (46.2% against 40.7% for phone), but only 18-44s prefer it (58.4%); weighted to the age mix, preference is split (40.7% online, 45.0% phone, 6.7% in person).
  • ✓Only 13 respondents need an interpreter; 61.5% of them would prefer to book in person.
  • ✓Among those who tried to book, women look less satisfied than men (47.5% against 55.3%, bases 303 and 237), but the difference is not statistically significant at 95%.
  • ✓Only 886 of 9,840 patients texted completed (about 9%), and the 7,260 registered adults without a mobile number on record were never invited; patients who cannot or do not go online are the least likely to have answered.

The analysis

Recommendation

Do not endorse online-first as 'working', and do not restore an untargeted 8am phone scramble either. Reject option 1 as it stands. Adopt option 3 as the floor and widen it into a genuine hybrid, then test capacity before making it permanent. Specifically: (a) keep the online form, but shorten it, fix its accessibility and set a published same-working-day response standard with immediate acknowledgement — this addresses the largest reported driver of unmet need (43.7% expected too long a reply) and the 45 open-text complaints about silence; (b) add a properly staffed phone booking line that any patient can use on request, without having to prove age, disability or language need — the age-weighted data show ~4,930 under-65 adults prefer phone versus ~2,770 aged 65+, so an eligibility-gated line would exclude most phone preferrers and would be experienced as gatekeeping (A583, A659, A746); (c) guarantee in-person help at reception and an interpreter route, proactively offered to over-75s, disabled patients and patients with limited English, since 93.9% of 75+ and 77.8% of 65-74 respondents prefer phone or reception, and 65.7% of disabled respondents prefer a non-online route. Report to partners in these terms: 'Booking works well for younger, digitally confident patients (64.7% satisfied) and fails older and disabled patients (16.2% of over-65 triers satisfied); about 4,400 adults had a need they never brought to us. We cannot say whether satisfaction has fallen since 2025 because the two surveys are not comparable.' Tell the PPG chair that the female/male gap is not statistically significant, that disabled patients' preference for non-digital routes is clear, and that interpreter users are too few to quantify but describe a specific English-only-form barrier we will fix and research. Staffing must be modelled, not assumed: a line that is not answered would simply convert today's silent waits into abandoned calls (10.2% already expect not to get through). Run the widened line as a time-boxed, measured pilot alongside the form and turnaround fixes, and commission a mixed-mode survey including patients with no mobile before any permanent decision to remove or restrict a channel.

What would change it

Four things would move me. (1) A mixed-mode survey (paper/postal/telephone plus SMS, sampled from the whole list including the 7,260 without mobiles, with oversamples of 75+, disabled and interpreter users) showing age-weighted phone preference well below ~30% and over-65 satisfaction comparable to under-65s — that would support keeping online-first with targeted help only. (2) Operational data or pilot results showing a universal-on-request line cannot be answered reliably (e.g. <90% answered within two minutes, high abandonment) without diverting clinical or reception capacity from urgent care — that would push me to a strictly targeted, guaranteed-answer line plus a callback guarantee for everyone else. (3) Evidence that the phone demand is really a turnaround complaint: if shortening the form and guaranteeing a same-working-day response lifts under-65 trier satisfaction from ~56% to 70%+ and collapses phone preference among under-65s in a repeat measure, then a targeted line plus fast online responses becomes sufficient and a universal line unnecessary. (4) Targeted research with interpreter users and disabled patients showing that a translated/easy-read/accessible form plus consented proxy booking removes the barrier without a separate human channel — that would reduce the in-person and phone requirement for those groups. Conversely, if the pilot shows large numbers of over-75s or patients with no mobile still not booking at all, the decision would shift from channel design to proactive outreach (named contact, care-home and carer routes, list cleanup).

Findings

  1. The practice manager cannot tell the partners that 'patients prefer the new system'. Preference is split, and once the sample is corrected for age, phone is the single most preferred route. Online-first is working well for a large, younger, digitally confident group and badly for everyone else — that is the finding to report, not a verdict of overall success.

    High confidenceBase 886

    Q6 unweighted (base 886): online 46.2%, phone 40.7%, in person 4.6%. Weighted to the registered adult age mix (44/32/13/11): phone 45.0%, online 40.7%, in person 6.7%, no preference 5.8%. Weighted phone preference leads online by ~4.3pp — inside the noise for a within-sample comparison (MoE on 40.7% at n=886 is ±3.2pp), so the honest statement is 'no online majority', not 'phone wins'.

  2. Satisfaction with booking is low in absolute terms and falls steeply with age. This is the most robust signal in the survey and is enough on its own to justify change, regardless of what last year's figure was.

    High confidenceBase 549

    Q2 (base 549 triers): 50.8% satisfied (±4.2pp), 33.9% dissatisfied. By age: 18-44 64.7% (272), 45-64 44.0% (209), 65-74 17.8% (45), 75+ 13.0% (23). Pooling 65+: 11 of 68 triers satisfied (16.2%) vs 268 of 481 under-65 (55.7%) — a ~39pp gap, far outside sampling error. Age-weighted overall satisfaction 46.3%. Small older bases mean the exact 75+ figure is fragile (±13.7pp) but the direction and size of the age gap are not in doubt.

  3. The GP partner's question cannot be answered from these data: the 20-point gap versus last year is not a measured fall in satisfaction. Do not report 'satisfaction fell from 71% to 51%'. Commission a like-for-like measure instead.

    High confidenceBase 549

    2025: paper, handed to patients in the waiting room, 412 returns, 71% satisfied — a sample of people who had already successfully reached the practice, older and more frequent attenders, collected face-to-face. 2026: SMS to the 58% of adults with a mobile, 9.0% completion, satisfaction asked only of the 549 who tried to book, self-completed on a phone. Mode, sampling frame, base and routing all changed at the same time as the booking system. The decline is plausible and consistent with the age gradient and open text, but its size is unquantifiable and could be anywhere from negligible to larger than 20pp.

  4. A phone line restricted to over-75s, disabled patients and interpreter users would miss most of the demand for phone booking. Phone preference is a mainstream preference, not only an access need — so the targeted option should be treated as a floor to build on, not the whole answer.

    High confidenceBase 886

    Q6 phone preference: 45-64 49.0% (base 337), 18-44 29.9% (base 462, ≈138 people), 65-74 63.0% (54), 75+ 72.7% (33). Weighted, ~28.8% of all adults prefer phone and are under 65 (≈4,930 of 17,100) versus ~16.2% who are 65+ (≈2,770). In the open text, 84 of the 94 phone requests come from under-65s; several explicitly reject eligibility gating (A583 'for everyone, not just those who can't go online', A659 'I shouldn't have to justify', A039, A201, A574, A332, A870, A746, A006).

  5. But the evidence does not support simply reopening the old 8am scramble either. The biggest self-reported driver of unmet need is slow replies and the belief that no appointment is available — capacity and turnaround problems that a phone line alone will not fix, and which a poorly staffed line would reproduce.

    Medium confidenceBase 215

    Q5 (base 215 who needed but did not try): expected to wait too long for a reply 43.7% (≈94 people), did not think they would get an appointment 28.8% (≈62), expected not to get through by phone 10.2% (≈22). Digital reasons are small overall: form difficult 16.7%, not confident online 9.3%. 45 of 246 open-text answers ask for faster replies to the online form (typical reported waits 24 hours to 5 days: A805, A728, A439, A208), and 29 ask for a shorter/less repetitive form. Some phone requests are explicitly a workaround for silence (A310, A544, A781, A195, A135).

  6. Around 4,400 adults had a need they never brought to the practice in three months. Satisfaction questions cannot see these people at all, and they are concentrated among those who never even tried to book. This is the strongest equity argument for change and the metric to monitor afterwards.

    Medium confidenceBase 886

    Q4 Yes 24.3% of 886 (215 people); age-weighted 25.7% → ~4,390 of 17,100 adults. Highest among respondents who did not try to book at all: 35.1% of 313, vs 17.1% of 549 triers. 65-74 33.3% (54) and 75+ 33.3% (33) vs 22.1% of 18-44 (462). Q2 was asked only of triers, so the 313 non-triers — the group with most unmet need — contribute nothing to the 50.8% satisfaction headline.

  7. The survey systematically under-samples exactly the patients the targeted option is designed for, so it almost certainly understates the access problem. Treat every unweighted total as an optimistic upper bound.

    High confidenceBase 886

    886 completes = 9.0% of the 9,840 adults texted and 5.2% of all 17,100 adults. 7,260 adults (42%) have no mobile on record and were never invited. Sample vs list: 18-44 52.1% vs 44%; 45-64 38.0% vs 32%; 65-74 6.1% (54) vs 13%; 75+ 3.7% (33) vs 11%. Only 13 interpreter users (1.5%) and 102 disabled respondents. Age weighting repairs the age mix only; within each band, people who answer a text survey are likely more digitally confident than those who do not, so even 46.3% weighted satisfaction is probably generous.

  8. Answer to the PPG chair, part 1: there is no statistically significant evidence that women find booking harder. Do not build a sex-targeted intervention on this survey; the apparent gap is within sampling error.

    Medium confidenceBase 540

    Q2 female 47.5% satisfied (base 303) vs male 55.3% (base 237): gap 7.8pp, SE ≈4.3pp, z ≈1.8, p ≈0.07, 95% CI roughly -0.7 to +16.3pp. Women are 54.2% of the sample and dominate the open-text phone requests, but that reflects who wrote in, not a measured barrier. The study is powered to detect only large sex differences; a real gap of 5-8pp could exist and be missed.

  9. Answer to the PPG chair, part 2: disabled patients do report worse booking experiences and clearly want non-digital routes — act on the preference evidence and the described mechanisms, not on the satisfaction gap, which is not significant at this base.

    Medium confidenceBase 102

    Q2 disability yes 40.0% satisfied (base 65, ±11.9pp) vs no 52.3% (484): 12.3pp gap, not significant. Q6 disability yes (base 102): phone 50.0% + in person 15.7% = 65.7% prefer a non-online route; only 23.5% prefer online vs 49.1% of non-disabled (784). Open text gives concrete, fixable mechanisms: screen-reader failures (A237), tremor/small targets (A193, A443, A742), one-handed use (A221), fatigue/save-and-return (A358, A512), easy-read and supported completion (A262, A320), hearing (A265, A375, A789), sight (A334, A713, A565), proxy booking with consent (A437).

  10. Answer to the PPG chair, part 3: interpreter users cannot be quantified here (13 respondents), but the two open-text accounts and the preference pattern point to a specific, plausible barrier — an English-only clinical form. Act on the mechanism now and verify with targeted research; do not quote a percentage for this group.

    Low confidenceBase 13

    Q6 interpreter yes: base 13 — 8 of 13 (61.5%) prefer in person at reception, 3 (23.1%) phone, 1 (7.7%) online. No satisfaction cross-tab is publishable at this base. A112: 'Form in Polish please… I am scared I write something wrong about my health.' A756: 'Someone who speaks Urdu… my daughter is not always home to help me.' 1.5% of a self-selected SMS sample is very likely a large undercount of the practice's true interpreter need.

  11. Older patients' problem is confidence and human contact, not form usability alone — a shortened form will not make online-first acceptable to them. Any digital-only future for over-75s is unsupported by these data.

    Medium confidenceBase 87

    Q5 among 65+ who did not try (base 29: 18 aged 65-74, 11 aged 75+): 'not confident going online' 44.4% and 36.4% (12 people combined) vs 5.9% of 18-44 and 2.4% of 45-64; 'form difficult' 27.8% and 27.3% (8 people). Q6: 75+ phone 72.7% + in person 21.2% = 93.9% of 33; 65-74 phone 63.0% + in person 14.8% = 77.8% of 54. 10 of the 12 phone/human-contact requests from over-65s in the open text are explicit (A006, A018, A110, A156, A320, A332, A395, A440, A678, A886; A114 and A644 ask for 'a person' without naming a channel). Bases of 11-33 mean individual percentages are indicative only.

  12. Two substantial open-text themes are outside this decision and should be routed elsewhere rather than allowed to inflate the case for either booking channel: response turnaround (a capacity/process issue) and continuity of care (a clinical model issue).

    Medium confidenceBase 246

    Of 246 open-text answers (27.8% of 886, self-selected): 94 ask for phone booking (38.2% of answers, 10.6% of all respondents), 45 ask for faster replies, 29 for a shorter/clearer form, 45 say the online form is already easy and need no change, 32 offer no suggestion (most saying they rarely book), 15 ask for a usual/named GP (e.g. A023, A032, A711, A723, A778, A881), 21 describe disability-related adjustments, 2 describe language needs. Open-text counts are volumes of feeling, not prevalence estimates.

Caveats

  • Coverage and response bias dominate everything else. 886 completes = 9.0% of the 9,840 adults texted and 5.2% of the 17,100 adult list. The 7,260 adults (42%) with no mobile on record had no chance of taking part, and lack of a mobile is likely correlated with older age, deprivation and limited English. The survey's design filters out the people most affected by online-first.
  • The sample is markedly younger than the list: 65+ is 9.8% of respondents (87) versus 24% of registered adults; 75+ is 33 people versus an expected ~188 at list proportions. All unweighted totals therefore flatter online-first.
  • Age weighting (46.3% satisfied; 45.0% phone; ~4,390 with unmet need) corrects the age mix only. It assumes respondents within a band resemble non-respondents in that band, which is unlikely — digitally confident people are more likely to answer an SMS survey. Treat weighted figures as indicative adjustments, not population estimates, and do not attach confidence intervals to them.
  • Small bases: 75+ satisfaction rests on 23 triers (13.0%, ±13.7pp), 65-74 on 45 (17.8%, ±11.2pp), disabled triers on 65 (40.0%, ±11.9pp), interpreter users on 13 in total. No interpreter satisfaction figure should be published; no significance test is meaningful for that group. Non-binary respondents (~2) cannot be reported at all.
  • The year-on-year comparison is invalid: 2025 was a paper waiting-room survey of 412 attenders (a group that had already succeeded in booking) with unknown routing; 2026 is an SMS survey asking satisfaction only of triers. Mode, frame, base and social-desirability pressure all changed simultaneously with the booking system. The 71%→50.8% difference cannot be attributed to online-first.
  • Q2 measures only the most recent attempt, and only among people who tried. The 313 respondents who did not try to book at all — 35.1% of whom report unmet need — are invisible in the satisfaction figure. Satisfaction among triers is a survivorship measure.
  • Q5 captures recalled, self-reported expectations about the most recent occasion, not observed behaviour, and is multi-coded. 'Expected not to get through by phone' (10.2%) and 'did not think I would get an appointment' (28.8%) are beliefs about the practice, possibly inherited from the pre-April system; they are not measurements of current phone or appointment performance.
  • Q6 is stated preference for a hypothetical future booking with no trade-offs (no queue time, no waiting for a callback, no cost). 'By phone' is ambiguous — patients may mean instant ring-and-book, or a staffed triage callback. The 45.0% weighted phone figure is an upper bound on demand for a live ring-and-book service.
  • Open text is 246 self-selected answers (27.8% of respondents); theme volumes are not prevalence estimates and skew toward the strongly motivated. Coding note: the phone count of 94 includes A044 ('in person on the phone', ambiguous) and excludes A114, A433 and A644, who ask for 'a person' without naming a channel; a broader 'human contact' reading would give 97, and 12 rather than 10 among over-65s.
  • No operational data were supplied: form submission-to-response times, form abandonment, phone answer/abandonment rates, appointment supply, or clinical outcomes. Self-reported waits of 24 hours to 5 days are unverified and are the single most important gap to close before committing capacity.
  • Disability (11.5%) and interpreter need (1.5%) are self-defined and almost certainly undercounted in this mode; the practice's own coded interpreter and accessibility flags should be used to size the targeted groups.
  • Fieldwork ran 7-21 September 2026, a single season; the change appears to date from April 2026 (A213). No pre-change baseline on the same instrument exists, and no data on patients who deregistered or on carers booking by proxy.

Next steps

  • decisionPartners agree the hybrid direction in principle: retain online, add a staffed phone route open to any patient on request, plus guaranteed reception help and interpreter access — and agree that no channel is withdrawn until the pilot and the mixed-mode survey report.
  • experimentRun a 10-12 week pilot of a staffed booking line (e.g. a defined morning window plus overflow staffing), open to all, with call answering targets, queue messaging and a callback fallback; log demand by age, disability and interpreter flag.
  • experimentShorten and de-duplicate the online request form (a short route for simple requests) and introduce an immediate acknowledgement plus a same-working-day response standard; measure before/after against the current baseline.
  • researchCommission a mixed-mode survey (SMS plus postal/paper and telephone, with an assisted option) sampled from the full adult list including the 7,260 patients with no mobile, with quotas or oversamples for 75+, disabled patients and interpreter users; keep Q2/Q4/Q6 wording identical and report weighted with confidence intervals and bases.
  • researchQualitative access work: 15-20 interviews or observed booking sessions with over-75s, disabled patients (sight, hearing, tremor, fatigue, learning disability) and interpreter users, conducted in their language and including people who have not tried to book; test translated, easy-read, large-text and consented-proxy routes.
  • decisionFix form accessibility to WCAG 2.2 AA (screen-reader labels, target size, contrast/large text, no fiddly drag/double-tap, save-and-return, easy-read version) and enable booking by a consented proxy/carer; add BSL/video-relay and interpreter booking routes.
  • monitorBuild an operational baseline dashboard now: submission-to-response times, form start/abandon rates, phone answer and abandonment rates, appointment supply versus demand, and contacts that end with no appointment.
  • researchAudit the register for missing mobile numbers, interpreter/language flags and accessibility needs; compare no-mobile patients' age profile with the list, and capture contact-route preferences at registration and at every contact.
  • experimentProactively contact over-75s, patients with recorded accessibility needs and patients needing an interpreter to tell them about the phone and reception routes and record their preferred channel; include care homes and known carers.
  • monitorRepeat the satisfaction and unmet-need measures at 6 months using identical wording and a mixed mode, reporting by age, disability and language with bases; explicitly do not benchmark against the 2025 paper figure.
  • decisionPass the continuity-of-care requests to the clinical model workstream and the faster-reply issue to the capacity/triage workstream, keeping them separate from the channel decision.

Open-text themes it coded

Wants to book by phone / speak to a person on the phone (explicit) 94Wants to speak to a person when booking but does not name a channel 3Phone booking should be open to everyone, not gated by age/disability/eligibility 9Faster response to online requests (waits of a day to five days) 45Online form too long, repetitive or confusing — shorten and simplify 29No change needed — online form already quick and easy 45No suggestion offered — mostly infrequent or non-users of the practice 32Disability-related adjustments: accessible form, easy read, save-and-return, proxy/carer booking, or a human alternative 21Wants in-person help at reception to make or complete a request 4Language barrier: translated form or a speaker of my language 2Continuity of care — a usual GP who knows my history (outside the booking-channel decision) 15

The survey it planned

3 screening questions and 12 questions, as the model wrote them.

  1. S1

    Are you registered as an NHS patient at Ashgrove Medical Group?

    One answer
    • Yes
    • No
    • I'm not sure

    Continues if Yes

  2. S2

    Which age group are you in?

    One answer
    • Under 18
    • 18 to 44
    • 45 to 64
    • 65 to 74
    • 75 or over
    • Prefer not to say

    Continues if 18 to 44; 45 to 64; 65 to 74; 75 or over

  3. S3

    Who is filling in this survey?

    One answer
    • Me, on my own
    • Me, with help from someone else
    • Someone else is filling it in for me, with my permission (for example a family member or carer)

    Continues if Me, on my own; Me, with help from someone else; Someone else is filling it in for me, with my permission (for example a family member or carer)

  4. Q1

    Since the start of April 2026, have you tried to get an appointment at Ashgrove Medical Group? This includes any appointment, for example with a GP, a nurse, a pharmacist or a health care assistant.

    One answer
    • Yes, once
    • Yes, more than once
    • No
    • I can't remember
  5. Q2

    Thinking about the most recent time you tried to get an appointment, how satisfied or dissatisfied were you with the way you asked for it?

    Scale

    Scale 1-5: Very dissatisfied to Very satisfied

    Routing Shown only if Q1 is: Yes, once; Yes, more than once

  6. Q3

    The most recent time you tried to get an appointment, what happened in the end?

    One answer
    • I got an appointment as soon as I needed one
    • I got an appointment, but I waited longer than I thought was safe for my health
    • I got help without an appointment (for example a phone call, a prescription or a referral)
    • I was told I could not have an appointment
    • I never heard back from the practice
    • I gave up before I finished asking
    • I am still waiting to hear
    • I can't remember

    Routing Shown only if Q1 is: Yes, once; Yes, more than once

  7. Q4

    The most recent time you tried to get an appointment, how did you first ask?

    One answer
    • I filled in the online request form on the practice website
    • I filled in an online request form in the NHS App
    • I phoned reception
    • I went into the practice and asked at reception
    • I sent a text message
    • A family member, friend or carer asked for me
    • Another way
    • I can't remember

    Routing Shown only if Q1 is: Yes, once; Yes, more than once

  8. Q5

    In the last 12 months, was there a time when you felt you needed to see or speak to someone at the practice, but you did not ask for an appointment?

    One answer
    • Yes, more than once
    • Yes, once
    • No
    • I can't remember
    • Prefer not to say
  9. Q6

    Which of these, if any, have made it harder for you to ask for an appointment in the last 12 months? Please choose all that apply.

    Any that apply
    • I don't use the internet
    • I don't have a device or enough mobile data to go online
    • Using a screen, typing or reading small text is difficult for me
    • The online form is too long or complicated
    • I find it hard to put my health problem into writing
    • I can't read or understand English well enough
    • I was worried about who would see my personal information
    • I didn't know what would happen after I sent the form
    • Waiting for a text or phone call back was difficult
    • Getting through to reception on the phone was difficult
    • Getting to the practice in person is difficult for me
    • I thought I wouldn't get an appointment anyway
    • I didn't want to make a fuss
    • Something else
    • Nothing has made it harder
    • Prefer not to say
  10. Q7

    If you needed an appointment next week, how would you most like to ask for it? Please choose the one you would prefer, even if it is not available now.

    One answer
    • Fill in an online form on the practice website
    • Fill in an online form in the NHS App
    • Phone reception and speak to someone there and then
    • Phone reception and ask them to call me back
    • Go into the practice and ask at reception
    • Send a text message
    • Ask a family member, friend or carer to ask for me
    • I have no preference
    • Don't know
  11. Q8

    The practice is deciding how patients should ask for appointments. Which one of these would you most like it to do?

    One answer
    • Keep things as they are now: ask online, and phone reception only if you cannot go online
    • Bring back a phone line every morning that any patient can call to book, alongside the online form
    • Keep asking online, and add a phone line and help at reception for patients who need it (for example patients aged 75 or over, disabled patients, and patients who need an interpreter)
    • Don't know
  12. Q9

    Which of these would make it easier for you to ask for an appointment? Please choose all that apply.

    Any that apply
    • A phone number where I can speak to a person
    • Someone at reception to fill in the online form with me
    • A shorter, simpler online form
    • The online form in a language other than English
    • An easy-read version of the form
    • A paper form I can hand in at reception
    • Phone lines open for longer
    • Help getting online (a device, mobile data, or someone to show me how)
    • A text telling me what is happening with my request
    • Being able to see what appointment times are available
    • Nothing — asking for an appointment is easy enough for me already
    • Something else
    • Prefer not to say
  13. Q10

    Do you have a physical or mental health condition or illness that lasts, or is likely to last, 12 months or more, which reduces your ability to do day-to-day activities?

    One answer
    • Yes, a lot
    • Yes, a little
    • No
    • Prefer not to say
  14. Q11

    When you contact the practice, do you need someone to interpret or translate for you?

    One answer
    • Yes, always
    • Yes, sometimes
    • No
    • Prefer not to say
  15. Q12

    Is there anything else you would like to tell us about asking for an appointment at Ashgrove Medical Group? You can skip this question. Please do not include names or anything that could identify you or another person.

    Open text
Sample plan

POPULATION AND FRAME: all 17,100 registered adults (18+). Main frame = every adult with a mobile number on record (expected ~15,300; confirm from the register before launch). Known coverage gap: the ~1,800 adults with no mobile on record are disproportionately 75+, disabled and speakers of other languages — exactly the groups the third decision option targets. An SMS-only survey would systematically under-represent them, so a second frame is added: postal invitation (with paper questionnaire, freepost return and a researcher phone number for completion by phone) to (a) every adult with no mobile on record and (b) all patients aged 75+ regardless of mobile, plus reception-offered assisted completion for walk-ins. Every respondent carries a mode flag (SMS-online / postal-paper / telephone / assisted) and an assistance flag (S3). PRECISION AND SAMPLE SIZE — THE 900 TARGET DOES NOT MEET THE BRIEF: ±5pp at 95% confidence for a proportion needs n=385 per age band (p=0.5). With 900 completes split four ways (225 each) the per-band margin is ±6.5pp unweighted, about ±7.2pp after weighting; only the overall figure (±3.3pp) would meet ±5pp. Recommendation to partners: raise the target to ~1,850 completes with a hard minimum quota of 460 per age band (18–44, 45–64, 65–74, 75+). 460 raw gives ±4.6pp unweighted and ±5.0pp allowing a weighting design effect of 1.2. Finite-population correction will reduce the requirement for the smaller bands (e.g. if 75+ numbers ~2,200 on the register, ~390 weighted-equivalent completes suffice); exact quotas will be set from register counts. If the budget is fixed at ~900, the partners must formally accept ±6.5–7pp per band (and wider for need subgroups) — this must be agreed in writing before fieldwork, not discovered at reporting. QUOTAS: hard quota 460 per age band (S2), monitored daily; soft targets of 250+ completes with a disability that limits activities 'a lot' (Q10) and 150+ needing an interpreter (Q11), achieved by boosting the postal/telephone route rather than by screening people out. Note the resulting precision for need subgroups: n=250 → ±6.2pp, n=150 → ±8pp; cells such as '75+ and disabled' will be indicative only and should be followed up qualitatively (10–15 interviews with 75+, disabled and interpreter-need patients, including non-responders and patients with no mobile). FEASIBILITY: SMS response of 8–12% on ~15,300 invites yields ~1,200–1,800 completes, but 75+ SMS response is typically far lower, so the 75+ quota is expected to be met only with the postal/telephone boost (dual invitation, paper return, researcher call-back). If 75+ still falls short, report the achieved margin honestly and do not present a boosted, self-selected reception sample as representative. FIELDWORK: 3 weeks, unique tokenised links (one response per patient), 2 SMS reminders (days 5 and 12) and 1 postal reminder, opt-out honoured, no incentive; sender shows the practice name and a researcher contact. ACCESSIBILITY: mobile-first, one question per screen, large text and high contrast, no grids or sliders, plain English (reading age ~10), save-and-return, translations into the practice's top three languages plus easy-read; telephone and paper versions use identical wording. Estimated completion time 4–5.5 minutes (12 questions, no matrices). DATA QUALITY: dedupe by token, drop completes under 90 seconds or with implausible patterns, report screen-outs (under-18s, non-patients, 'prefer not to say' age — who cannot be allocated to a quota) separately from completes. WEIGHTING: rim weight to register age band × sex, with a mode/no-mobile adjustment; report unweighted bases alongside weighted percentages and use design-effect-adjusted confidence intervals. NON-RESPONSE CHECKS: compare achieved sample to register on age, sex and area deprivation; early vs late responder comparison; 5-question follow-up with 100 random non-responders (postcard/phone) to estimate direction of bias. NEUTRALITY AND GOVERNANCE: the practice manager's stated wish that the survey show online-first is working is a bias risk and is recorded as such. Wording is neutral and balanced (Q6 and Q9 include 'nothing' options; Q8 describes all three options factually, in equal-length language, with the three policy options shown in random order; Q6, Q7 and Q9 option order randomised with 'nothing/something else/don't know/prefer not to say' anchored last). No question wording or intro text says or implies the new system is an improvement. Analysis specification, subgroup definitions and the primary outcome (% satisfied, top two of five on Q2) are pre-registered with the partners before launch; all findings are reported regardless of direction, including any that contradict the manager's view. The survey is anonymous, no identifiable clinical data is collected, respondents are told participation is voluntary and will not affect their care, and the practice's information governance lead signs off the invite text and DPIA. The 71% pre-change figure is treated as an indicative benchmark only: mode, wording and base may differ, so change is reported as a caveat-laden comparison, not a clean before/after measure.