Benchmarks / SurveyBench / How Ashgrove patients find online-first booking / GPT-6.2 Sol

Measured by Spring Prompt

GPT-6.2 Sol: How Ashgrove patients find online-first booking

The decision: whether to keep online-first booking as it is, restore a morning phone booking line for every patient, or add a dedicated phone line and in-person help at reception only for patients who need them (over-75s, patients with a disability, and patients with limited English). The brief and the data →

Verdict
✓ Sound research
Research score
100 of 100
Analysis rating
1,354
Head to head, this task
won 11 of 13

Scorecard

Numbers 11 of 11 right

QuestionIts answerTrue
Among respondents who tried to book in the last three months, what percentage were satisfied (fairly or very) with the booking process?✓50.8%50.8%
Weighting each age band's percentage satisfied (fairly or very, among those who tried to book) by that band's share of registered adults, what is the overall percentage satisfied?✓46.3%46.3%
What percentage of all respondents said there was a time in the last three months when they needed an appointment but did not try to book one?✓24.3%24.3%
Weighted to the age mix of registered adults, what percentage would prefer to book by phone?✓45%45%
What is the 95% margin of error, in percentage points, on the unweighted percentage satisfied among respondents who tried to book?✓4.2 points4.18 points
Among respondents who tried to book, is the difference in the percentage satisfied (fairly or very) between female and male respondents statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
How many of the open-text answers ask to be able to book by phone, or for a phone booking line?✓9597
How many of the open-text answers from respondents aged 65 or over ask to be able to book by phone, or for a phone booking line?✓1113
Weighted to the age mix of registered adults, about how many of the practice's 17,100 registered adults would say there was a time in the last three months when they needed an appointment but did not try to book one?✓4,3924,393
Among respondents aged 18 to 44, what percentage would prefer to book using the online request form?✓58.4%58.4%
What percentage of the patients sent the text message completed the survey?✓9%9%

Traps in the data 7 of 7 handled

  • ✓Patients aged 65 or over are 24% of registered adults but 9.8% of respondents (text link, online only). They are less satisfied and prefer the phone, so weighting to the age mix lowers satisfaction among those who tried to book from 50.8% to 46.3% and raises the phone preference from 40.7% to 45.0%.
  • ✓Q2 satisfaction (50.8%) is asked only of the 549 who tried to book. 24.3% of all respondents needed an appointment but did not try to book (35.1% of those who did not try to book); those who did not try are left out of the satisfaction base, which flatters it.
  • ✓Last year's 71% came from a paper survey handed out in the waiting room before the change, to patients who had got an appointment; this year's 50.8% is from an online survey by text of those who tried to book. Different method and different people.
  • ✓The practice manager believes patients prefer the new system. Unweighted, the online form leads (46.2% against 40.7% for phone), but only 18-44s prefer it (58.4%); weighted to the age mix, preference is split (40.7% online, 45.0% phone, 6.7% in person).
  • ✓Only 13 respondents need an interpreter; 61.5% of them would prefer to book in person.
  • ✓Among those who tried to book, women look less satisfied than men (47.5% against 55.3%, bases 303 and 237), but the difference is not statistically significant at 95%.
  • ✓Only 886 of 9,840 patients texted completed (about 9%), and the 7,260 registered adults without a mobile number on record were never invited; patients who cannot or do not go online are the least likely to have answered.

The analysis

Recommendation

Choose dedicated phone and reception support as the first change, while keeping the online form. Prioritise over-75s, disabled patients and people with language barriers, but let anyone who cannot use the form reliably ask for help without proving eligibility. Check current NHS access requirements before implementation; the dedicated service should add support, not remove any required phone or in-person access. Train reception staff to arrange interpreting and make reasonable adjustments. In parallel, shorten the form, fix accessibility problems and tell patients when to expect a meaningful reply. Do not restore a universal morning line on preference data alone: the survey does not tell us whether that would improve access or simply move the queue.

What would change it

Broaden phone booking if outreach and pilot data show that patients outside the priority groups are still abandoning or delaying requests because they cannot use the online route, or that the way support is offered is itself deterring patients. Test wider phone access before committing to a permanent morning line, and look for better booking completion and less reported deterrence without worse waits or unsustainable workload. If delays persist across all channels and are driven by appointment availability or triage capacity, address that constraint rather than adding another queue.

Findings

  1. The practice manager should not tell partners that patients prefer online-first or that it is working for everyone. Keep the online form for patients it serves well, but do not leave access unchanged.

    Medium confidenceBase 886

    Among 886 respondents, online was the most selected option but not a majority preference: 46.2% chose online, 40.7% phone and 4.6% reception. With adult population age weights, phone moves ahead of online, 45.0% versus 40.7%. Online was preferred by 58.4% of 462 respondents aged 18–44. Q6 measures preferred channel, not endorsement of an online-first policy.

  2. The GP partner can say that this year's survey recorded lower satisfaction, but not that satisfaction fell after, or because of, the move to online-first.

    High confidenceBase 549

    50.8% of 549 recent triers were fairly or very satisfied and 33.9% were dissatisfied. The age-standardised satisfaction figure is 46.3%. The unweighted result is 20.2 percentage points below last year's 71% from 412 waiting-room paper returns. The two surveys reached different people in different ways, and last year's denominator is not shown to be equivalent to this year's recent triers.

  3. The case for assisted access is strongest for older people and patients with disabilities. A strict age-75 cutoff would miss barriers already visible among 65–74-year-olds.

    Medium confidenceBase 886

    Satisfaction was 64.7% among triers aged 18–44 (n=272), 17.8% at 65–74 (n=45) and 13.0% at 75+ (n=23). Phone was preferred by 63.0% of 65–74s (n=54) and 72.7% of over-75s (n=33). Among triers with a disability, 40.0% were satisfied (n=65), versus 52.3% without (n=484). Of all 102 respondents with a disability, 50.0% preferred phone and 15.7% preferred reception. The comments show why neither channel alone will meet every need.

  4. For the patient participation group: female respondents reported lower satisfaction, but this survey does not establish that women find booking harder.

    Medium confidenceBase 540

    47.5% of female triers (n=303) and 55.3% of male triers (n=237) were satisfied. The 7.8-point gap is not significant in a two-sided comparison at the 95% level (p approximately 0.07). This is a satisfaction comparison, not a measure of booking success or of who was deterred.

  5. For the patient participation group: there is enough evidence to test language support now, but not enough to say how much harder booking is for patients who need an interpreter.

    Low confidenceBase 13

    Of 13 respondents who need an interpreter, 8 preferred reception, 3 phone and 1 online. Comments describe difficulty understanding the form and fear of describing symptoms incorrectly. No interpreter breakdown of satisfaction, booking success or deterred attempts is supplied.

  6. Changing booking channels alone will not address reported deterrence. Give patients a clear acknowledgement and a realistic timeframe for a meaningful reply.

    Medium confidenceBase 886

    215 of 886 respondents (24.3%) reported at least one occasion when they needed an appointment but did not try to book. Of those 215, 43.7% expected to wait too long for a reply and 28.8% thought they would not get an appointment, versus 16.7% citing form difficulty and 9.3% low online confidence. Multiple reasons were allowed. Age weighting gives 25.7%, equivalent to about 4,392 adults if the survey rates apply across the practice; that is an illustrative estimate, not a count of people who went without care.

Caveats

  • The SMS invitation reached only the 9,840 adults with a mobile number on record (58% of registered adults). Only 886 completed, 9.0% of those invited and 5.2% of all registered adults. People who are harder to reach digitally may also be harder to reach through this survey.
  • Respondents aged 65–74 were 6.1% of completes versus 13% of registered adults; those aged 75+ were 3.7% versus 11%. Age weighting corrects the age mix, not selection or nonresponse within age bands. The weighted satisfaction figure standardises triers' age-band results to the age mix of all adults, not to the unknown age mix of all adult triers.
  • The 4.2-point margin is a nominal 95% simple-random-sample calculation using 549 triers. It does not account for coverage or response bias. The female–male significance test likewise addresses sampling variation under its assumptions, not selection bias or differences in patient mix.
  • Satisfaction excludes people who did not try in the period. A respondent can have tried on one occasion and been deterred on another: 17.1% of recent triers reported a separate occasion when they needed an appointment but did not try. Some deterred respondents got help elsewhere, so this measure is not equivalent to unmet clinical need.
  • Interpreter need is not the same as limited English, and the interpreter base is only 13. The survey covers adults booking for themselves, so it does not directly measure booking for children or by carers on someone else's behalf.
  • Open comments are voluntary and can receive multiple codes. The phone count includes requests that identify phone or calling as a booking route, not comments that only ask to speak to a person without specifying how. Counts describe what commenters raised, not how common each need is across the practice.

Next steps

  • decisionApprove a time-limited pilot of staffed phone and reception help, with a simple request-for-help route, interpreting and clear arrangements for supported or proxy contact.
  • experimentTest a shorter accessible form and add an immediate acknowledgement with a realistic timeframe for the next meaningful reply.
  • researchRecruit beyond SMS using post, phone and supported in-person outreach. Include patients without a mobile on record, people who have not recently attended, older and disabled patients, and people with language or interpreting needs. Ask the same core questions again using consistent wording and recruitment.
  • monitorReview a baseline and weekly pilot dashboard by channel and, where sample sizes permit, patient group.

Open-text themes it coded

Wants to book by phone or have a phone booking line 95Wants to speak to a person when booking but does not specify a channel 2Wants in-person booking or help at the practice 4Wants a faster reply, confirmation or a clear response timeframe 45Wants a shorter, clearer or less repetitive form 27Wants specific digital accessibility features such as larger controls, screen-reader support, Easy Read or save-and-resume 7Wants translation, language support or BSL access 3Wants staff to adapt how they communicate, for example by allowing more time, speaking clearly or facing the patient 5Wants a carer or family member to be able to speak or book on their behalf 2Wants continuity with a familiar GP 15Finds the current online form quick, easy or convenient 45Offers no specific suggestion or reports too little recent use to comment 32

The survey it planned

2 screening questions and 11 questions, as the model wrote them.

  1. S1

    Are you aged 18 or over?

    One answer
    • Yes
    • No
    • Prefer not to say

    Continues if Yes

  2. S2

    Are you currently registered as a patient at Ashgrove Medical Group?

    One answer
    • Yes
    • No
    • Not sure

    Continues if Yes

  3. Q1

    Since 1 April 2026, have you or someone helping you tried to book an appointment for you at Ashgrove Medical Group? Please count attempts even if you did not get an appointment.

    One answer
    • Yes
    • No
    • Not sure
  4. Q2

    Overall, how satisfied or dissatisfied have you been with the way you book appointments at Ashgrove Medical Group since 1 April 2026? Please think about the booking process, not the care you received.

    One answer
    • Very satisfied
    • Fairly satisfied
    • Neither satisfied nor dissatisfied
    • Fairly dissatisfied
    • Very dissatisfied
    • Not sure

    Routing Shown only if Q1 is: Yes

  5. Q3

    The last time you or someone helping you tried to book an appointment for you, what did you try first?

    One answer
    • Use the practice's online request form
    • Phone the practice
    • Go to reception in person
    • Something else
    • I don't know or can't remember

    Routing Shown only if Q1 is: Yes

  6. Q4

    Since 1 April 2026, was there a time when you needed an appointment at Ashgrove Medical Group but waited longer than you wanted before trying to book, or did not try to book at all?

    One answer
    • Yes, I waited longer than I wanted before trying
    • Yes, I did not try to book
    • Yes, both happened on different occasions
    • No, I tried to book each time I needed an appointment without waiting longer than I wanted
    • I did not need an appointment
    • Not sure
  7. Q5

    Thinking about the most recent time this happened, what made you wait or decide not to try to book? Select all that apply.

    Any that apply
    • I was not sure how to ask for an appointment
    • I did not have suitable internet access or a device I could use
    • The online form was hard for me to use or complete
    • The online form was not available when I tried to use it
    • I needed language or interpreter support
    • I could not get through by phone or did not think I could book by phone
    • I was worried I would miss the practice's reply or did not know when it would come
    • I did not expect to get an appointment when I needed one
    • A previous experience with booking put me off
    • Work, caring responsibilities or other commitments made it difficult
    • I decided to manage the problem myself, wait to see if it improved, or get help elsewhere
    • Another reason
    • Not sure (select this on its own)

    Routing Shown only if Q4 is: Yes, I waited longer than I wanted before trying; Yes, I did not try to book; Yes, both happened on different occasions

  8. Q6

    If each of these ways of asking for an appointment were available to you, which would you prefer to use?

    One answer
    • Fill in an online form
    • Phone the practice
    • Ask at reception in person
    • Another way
    • No preference
    • Not sure
  9. Q7

    Which of these options would you prefer Ashgrove Medical Group to offer? The online request form would stay available under all three options.

    One answer
    • Keep the current system: patients use the online form to ask for an appointment, and patients who cannot use the form can phone reception.
    • Add a morning booking phone line that any patient can use.
    • Add a dedicated booking phone line and in-person help at reception for patients who need this support and are over 75, have a disability, or have difficulty using English.
    • No preference
    • Not sure
  10. Q8

    What, if anything, would make it easier for you to book an appointment at Ashgrove Medical Group? Select all that apply.

    Any that apply
    • Clearer instructions on how to ask for an appointment
    • An online form with fewer questions or simpler wording
    • An online form available at more times of day
    • An online form that works better with my accessibility needs
    • Help filling in the online form
    • Information in another language or access to an interpreter
    • A phone line I can use to ask for an appointment
    • Help at reception in person
    • Confirmation that the practice has received my request
    • A clearer idea of when and how the practice will reply
    • A way to choose whether the practice replies by text or phone
    • Something else
    • Nothing would make it easier for me (select this on its own)
    • Not sure (select this on its own)
  11. Q9

    What is your age?

    One answer
    • 18 to 44
    • 45 to 64
    • 65 to 74
    • 75 or over
    • Prefer not to say
  12. Q10

    Do you have a disability or a physical or mental health condition that has lasted, or is expected to last, at least 12 months and limits your day-to-day activities?

    One answer
    • Yes, it limits my activities a lot
    • Yes, it limits my activities a little
    • No
    • Not sure
    • Prefer not to say
  13. Q11

    When contacting the practice, do you need an interpreter for a language other than English?

    One answer
    • Yes, always or often
    • Yes, sometimes
    • No, but I find it difficult to communicate in English
    • No, and I do not find it difficult to communicate in English
    • Not sure
    • Prefer not to say
Sample plan

Send the SMS link to every registered adult with a mobile number on record. Invite one response per eligible patient. Plan for about 900 completes, use no hard quotas, and keep all age bands open. Track responses by age band and use reminders to improve participation, prioritising underrepresented bands where feasible. Test the questionnaire on mobile and with assistive technology, provide translated versions and help completing the same online questionnaire where feasible, and pilot it to verify that the longest route takes no more than six minutes. The precision requirement cannot be promised with 900 completes: under simple random sampling, with no finite-population correction and a proportion of 50%, ±5 percentage points at 95% confidence requires about 385 usable answers per age band, or 1,540 across the four bands. An even split of 900 gives 225 per band and roughly ±6.5 points before weighting, and the satisfaction question will have fewer eligible respondents. Agree before fieldwork whether to aim for at least 385 usable satisfaction answers per band or keep the 900-complete target and report wider intervals. Compare respondents with the invited and registered adult populations using available age-band counts, and report unweighted bases and effective bases if weighting is used. The SMS frame misses patients without a mobile number on record, and the online format and voluntary response may underrepresent people who struggle to book online. Neither weighting nor confidence intervals can remove those biases.