Sign inTest your prompt

Benchmarks / SurveyBench / How Ashgrove patients find online-first booking / GPT-6 Luna

Measured by Spring Prompt

GPT-6 Luna: How Ashgrove patients find online-first booking

The decision: whether to keep online-first booking as it is, restore a morning phone booking line for every patient, or add a dedicated phone line and in-person help at reception only for patients who need them (over-75s, patients with a disability, and patients with limited English). The brief and the data →

Verdict
✗ Not sound
Research score
91 of 100
Analysis rating
1,102
Head to head, this task
won 9 of 10

Why it is not sound

The analysis

  • Reported a material finding the data does not support
  • Missed a trap: Q2 satisfaction (50.8%) is asked only of the 549 who tried to book. 24.3% of all respondents needed an appointment but did not try to book (35.1% of those who did not try to book); those who did not try are left out of the satisfaction base, which flatters it.
  • Missed a trap: The practice manager believes patients prefer the new system. Unweighted, the online form leads (46.2% against 40.7% for phone), but only 18-44s prefer it (58.4%); weighted to the age mix, preference is split (40.7% online, 45.0% phone, 6.7% in person).

Findings the judges found unsupported

  • Do not tell the partners that patients prefer the new system overall. Among recent booking triers, satisfaction was only just over half; age-weighted satisfaction is lower still. The survey shows a divided experience, not an overall endorsement of online-first.The comparison mixes age-weighted phone preference (45.0%) with unweighted online preference (46.2%). Age-weighted online preference is 40.7%, so the comparison misrepresents the relative preferences after weighting.
  • Women and men show different channel preferences in this sample, but the evidence does not establish a sex-based access problem. Interpreter findings are especially uncertain because very few respondents said they needed one.The supplied tables do not report booking-channel preferences by sex, and the open-text responses do not establish that women and men have different channel preferences.

Scorecard

Numbers 11 of 11 right

QuestionIts answerTrue
Among respondents who tried to book in the last three months, what percentage were satisfied (fairly or very) with the booking process?✓50.8%50.8%
Weighting each age band's percentage satisfied (fairly or very, among those who tried to book) by that band's share of registered adults, what is the overall percentage satisfied?✓46.3%46.3%
What percentage of all respondents said there was a time in the last three months when they needed an appointment but did not try to book one?✓24.3%24.3%
Weighted to the age mix of registered adults, what percentage would prefer to book by phone?✓45%45%
What is the 95% margin of error, in percentage points, on the unweighted percentage satisfied among respondents who tried to book?✓4.2 points4.18 points
Among respondents who tried to book, is the difference in the percentage satisfied (fairly or very) between female and male respondents statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
How many of the open-text answers ask to be able to book by phone, or for a phone booking line?✓9297
How many of the open-text answers from respondents aged 65 or over ask to be able to book by phone, or for a phone booking line?✓1213
Weighted to the age mix of registered adults, about how many of the practice's 17,100 registered adults would say there was a time in the last three months when they needed an appointment but did not try to book one?✓4,3924,393
Among respondents aged 18 to 44, what percentage would prefer to book using the online request form?✓58.4%58.4%
What percentage of the patients sent the text message completed the survey?✓9%9%

Traps in the data 5 of 7 handled

  • ✓Patients aged 65 or over are 24% of registered adults but 9.8% of respondents (text link, online only). They are less satisfied and prefer the phone, so weighting to the age mix lowers satisfaction among those who tried to book from 50.8% to 46.3% and raises the phone preference from 40.7% to 45.0%.
  • ✗Q2 satisfaction (50.8%) is asked only of the 549 who tried to book. 24.3% of all respondents needed an appointment but did not try to book (35.1% of those who did not try to book); those who did not try are left out of the satisfaction base, which flatters it.
  • ✓Last year's 71% came from a paper survey handed out in the waiting room before the change, to patients who had got an appointment; this year's 50.8% is from an online survey by text of those who tried to book. Different method and different people.
  • ✗The practice manager believes patients prefer the new system. Unweighted, the online form leads (46.2% against 40.7% for phone), but only 18-44s prefer it (58.4%); weighted to the age mix, preference is split (40.7% online, 45.0% phone, 6.7% in person).
  • ✓Only 13 respondents need an interpreter; 61.5% of them would prefer to book in person.
  • ✓Among those who tried to book, women look less satisfied than men (47.5% against 55.3%, bases 303 and 237), but the difference is not statistically significant at 95%.
  • ✓Only 886 of 9,840 patients texted completed (about 9%), and the 7,260 registered adults without a mobile number on record were never invited; patients who cannot or do not go online are the least likely to have answered.

The analysis

Recommendation

Do not keep online-first unchanged, and do not claim that patients generally prefer it. Implement the targeted-access option as an immediate minimum: retain online booking, provide a dedicated phone route and practical in-person reception help for patients who need them, with simple self-identification and no burdensome proof requirement. Make the support available to anyone unable to use the online route, not only people who fit a narrow age, disability or interpreter category. The findings do not by themselves justify restoring the former morning phone line for every patient, but the strong phone preference warrants a time-limited test of broader phone access or callback capacity alongside the targeted offer. Set and publish a response-time standard for online requests, and review capacity and safety before changing channels.

What would change it

A representative, accessible follow-up showing that a substantial share of patients across age and access groups cannot or will not use online-first, or that a universal phone route materially reduces unmet need without unacceptable staffing and waiting-time impacts, would support restoring phone booking for everyone. Conversely, evidence from reliable booking records and a representative survey that unmet need is low, online response times are consistently prompt, and accessible alternatives work well for patients who need them would support retaining online-first with targeted support. The current tiny interpreter subgroup is not enough to decide language-access provision; targeted engagement with interpreter users could materially change the design.

Findings

  1. Do not tell the partners that patients prefer the new system overall. Among recent booking triers, satisfaction was only just over half; age-weighted satisfaction is lower still. The survey shows a divided experience, not an overall endorsement of online-first.

    Medium confidenceBase 549

    50.8% satisfied among 549 recent triers; weighting the age-band satisfaction rates to the registered adult age mix gives 46.3%. For comparison, 45.0% would prefer phone booking after age weighting, while 46.2% prefer online.

  2. The observed satisfaction is substantially below last year's result, but the difference is not a reliable estimate of change because the surveys used different methods and reached different people. Treat it as a warning signal, not proof that satisfaction fell by 20 points.

    Low confidenceBase 549

    Current satisfaction is 50.8% among 549 recent triers, versus 71% among 412 waiting-room paper respondents in October 2025: a raw difference of 20.2 percentage points. The current survey was texted to patients with a recorded mobile and had a 9.0% completion rate; the prior survey was handed out in the waiting room.

  3. Women were less satisfied in this sample, but the difference from men is not statistically significant at the 95% level. There is not enough evidence here to conclude women find booking harder.

    Medium confidenceBase 540

    Among triers, 47.5% of women (base 303) and 55.3% of men (base 237) were satisfied, a 7.8-point gap. A two-proportion comparison is not significant at 95% (approximately p=0.07).

  4. The survey raises an important access concern for older patients, but the older-age estimates are imprecise and should not be treated as population rates. Their reported satisfaction and preference patterns nonetheless justify accessible alternatives and closer monitoring.

    Low confidenceBase 54

    Among triers, satisfaction was 17.8% for ages 65–74 (base 45) and 13.0% for ages 75+ (base 23), versus 64.7% for ages 18–44 (base 272). Phone preference was 63.0% among ages 65–74 (base 54) and 72.7% among ages 75+ (base 33).

  5. A substantial group report needing care but not attempting to book. This is a material access warning, not just a preference issue; the age-weighted figure implies roughly 4,400 adults, but is an indicative estimate rather than a count of affected patients.

    Low confidenceBase 886

    24.3% of all respondents said this happened in the last three months (base 886). Weighting the age-band rates to the practice age mix gives 25.7%, or about 4,392 of 17,100 adults. The rate was 33.3% among respondents aged 65–74 (base 54) and 75+ (base 33).

  6. Among people who did not try to book despite needing an appointment, expected delay was the most commonly selected reason. Fixing response expectations and access to an appointment should accompany any channel change.

    Medium confidenceBase 215

    Among 215 respondents answering the multi-select question, 43.7% selected expected to wait too long for a reply, 28.8% expected not to get an appointment, and 16.7% said the online form was difficult to use.

  7. Phone access is a strong expressed preference across the sample, not just among older patients. But preference is not the same as demonstrated need or demand for a staffed universal phone line, so it supports testing broader access rather than automatically restoring the previous service unchanged.

    Medium confidenceBase 886

    Age-weighted phone preference is 45.0%; unweighted, 40.7% prefer phone. In open text, 92 of the 246 answers explicitly asked to be able to book by phone or for a phone booking line, including 12 answers from people aged 65+.

  8. Women and men show different channel preferences in this sample, but the evidence does not establish a sex-based access problem. Interpreter findings are especially uncertain because very few respondents said they needed one.

    Low confidenceBase 13

    For satisfaction, the female–male difference among triers is not significant (bases 303 and 237). Only 13 respondents said they needed an interpreter; among them, 61.5% preferred in-person booking and 23.1% phone (Q6 base 13).

Caveats

  • Only 886 of the 9,840 patients texted completed the survey (9.0%). Texts were sent only to adults with a mobile number on record; 42% of registered adults had no recorded mobile and were not invited.
  • The respondent profile differs from the practice population: ages 18–44 were 52.1% of respondents versus 44% of registered adults, while ages 75+ were 3.7% versus 11%. Age weighting addresses only this age-mix difference, not non-response or other selection bias.
  • The 4.2-point margin of error is the simple-random-sample calculation for the 50.8% satisfaction estimate. The survey is not a probability sample, so that margin does not capture selection and non-response error.
  • Last year's 71% result came from a waiting-room paper survey, while this survey was texted to mobile-number holders. Different modes, response patterns and respondent populations make a direct before/after claim unsafe.
  • Older-group bases are small (23–54 for satisfaction and 33–54 for preference); the interpreter group is only 13. Those subgroup percentages are unstable. The survey does not provide a reliable test of satisfaction among people needing an interpreter.
  • The survey measures reported recent experiences and stated preferences, not actual booking outcomes, clinical urgency, unmet need verified from records, or the staffing and cost implications of a phone line.
  • The 246 open-text answers are qualitative evidence, not a representative count of all patients. The number of mentions indicates the range and salience of views, not their population prevalence.

Next steps

  • decisionApprove targeted phone and in-person support as a minimum access provision, with eligibility based on patient-reported need and a route for any patient who cannot use online booking.
  • experimentPilot an expanded phone booking or callback option, comparing it with the targeted offer before deciding whether to restore a universal morning phone line.
  • decisionImprove online request acknowledgement and response times, and review form length, clarity and accessibility with patients who use assistive technology or support.
  • researchRun an accessible, mixed-mode follow-up that includes patients without recorded mobiles and oversamples older adults, disabled patients and people needing interpreters.
  • researchReview booking and contact records alongside patient feedback, including online request volumes, response times, appointment offers and abandoned or repeated contacts.
  • monitorMonitor access and equity after changes, with a review point agreed before implementation.

Open-text themes it coded

Want a phone booking option or phone booking line 92Quicker acknowledgement or response to online requests 42Shorter, clearer, less repetitive online form 25Accessibility, communication support, or help completing a request 22Continuity with a known GP 15Positive experience of the online form or no change wanted 44Specific language or interpreter support 3

The survey it planned

1 screening question and 10 questions, as the model wrote them.

  1. S1

    Are you aged 18 or over and currently registered as a patient at Ashgrove Medical Group?

    One answer
    • Yes
    • No

    Continues if Yes

  2. Q1

    Since Ashgrove Medical Group introduced online-first booking in April 2026, which best describes your most recent need for an appointment?

    One answer
    • I needed an appointment and made a request
    • I needed an appointment and tried to make a request, but could not
    • I needed an appointment but did not try to make a request
    • I have not needed an appointment
  3. Q2

    What were the reasons you did not try to make a request? Select all that apply.

    Any that apply
    • I expected it would be difficult to get through by phone
    • I found or expected the online form to be difficult to use
    • I did not have suitable access to a phone, device or the internet
    • A disability or health condition made it difficult to use the available ways to book
    • I needed language or interpreter support
    • I was unsure how to book or what would happen after I made a request
    • The available booking times did not suit me
    • I thought I would not be able to get an appointment
    • I decided my problem was not serious enough
    • I was worried about contacting the practice
    • Another reason
    • Prefer not to say

    Routing Shown only if Q1 is: I needed an appointment but did not try to make a request

  4. Q3

    How did you make your most recent attempt to request an appointment?

    One answer
    • I filled in the online request form
    • I phoned reception
    • I went to reception in person
    • Someone else made the request for me
    • Another way

    Routing Shown only if Q1 is: I needed an appointment and made a request; I needed an appointment and tried to make a request, but could not

  5. Q4

    What happened after your most recent attempt?

    One answer
    • I was offered an appointment that met my needs
    • I was offered an appointment, but it did not meet my needs
    • I received advice or another form of help instead of an appointment
    • I was not offered an appointment or other help
    • I am still waiting for a response
    • Something else happened

    Routing Shown only if Q1 is: I needed an appointment and made a request; I needed an appointment and tried to make a request, but could not

  6. Q5

    Overall, how satisfied or dissatisfied were you with the process of requesting an appointment on that most recent attempt?

    Scale

    Scale 1-5: Very dissatisfied to Very satisfied

    Routing Shown only if Q1 is: I needed an appointment and made a request; I needed an appointment and tried to make a request, but could not

  7. Q6

    Which approach to booking appointments would you prefer Ashgrove Medical Group to use?

    One answer
    • Keep online-first booking as it is, with phone help for patients who cannot go online
    • Keep the online form and restore a morning phone booking line that any patient can use
    • Keep online-first booking and add a dedicated phone line and in-person help for patients aged 75 or over, patients with a disability, and patients who need English-language support
    • Another approach
    • I have no preference
    • I am not sure
  8. Q7

    What, if anything, would make it easier for you to request an appointment? Select all that apply.

    Any that apply
    • A morning phone booking line that any patient can use
    • A dedicated phone line for patients who need extra help
    • Being able to get help in person at reception
    • A simpler online form
    • Help to fill in the online form
    • Interpreter or language support
    • Clearer information about what happens after I make a request
    • A quicker response or updates while I am waiting
    • Booking at times that better suit me
    • Nothing; booking is already easy for me
    • Something else
    • I am not sure
  9. Q8

    How old are you?

    One answer
    • 18 to 44
    • 45 to 64
    • 65 to 74
    • 75 or over
    • Prefer not to say
  10. Q9

    Do you have a disability or long-term health condition that affects how you contact the practice or use booking services?

    One answer
    • Yes
    • No
    • Prefer not to say
  11. Q10

    Do you need an interpreter to communicate with the practice in English?

    One answer
    • Yes
    • No
    • Prefer not to say
Sample plan

Send the survey link by text to every registered adult patient with a mobile number on record, as required. Aim for 900 completed surveys, monitoring completion by the four age bands and sending reminders to nonrespondents as needed. Aim for approximately 225 completes in each age band so that every band is represented; weight the overall results to the practice's adult age distribution, and report subgroup denominators and unweighted subgroup estimates. The requested precision cannot be achieved with 900 completes: 225 responses in a band give a maximum approximate 95% margin of error of ±6.5 percentage points under simple random sampling, before any effect of weighting or nonresponse. To achieve ±5 points in each band at 95% confidence requires at least 385 completes per band, or about 1,540 in total, assuming a 50% response distribution and simple random sampling. The 900-complete constraint and the requested age-band precision are therefore incompatible. Results will also represent adults with a mobile number who respond to an online survey, not necessarily all registered adults.