Benchmarks / SurveyBench / How Ashgrove patients find online-first booking / Gemini 4 Neon

Measured by Spring Prompt

Gemini 4 Neon: How Ashgrove patients find online-first booking

The decision: whether to keep online-first booking as it is, restore a morning phone booking line for every patient, or add a dedicated phone line and in-person help at reception only for patients who need them (over-75s, patients with a disability, and patients with limited English). The brief and the data →

Verdict
✗ Not sound
Research score
30 of 100
Analysis rating
629
Head to head, this task
won 0 of 6

Why it is not sound

The analysis

  • Produced no usable analysis

Scorecard

The analysis

This model produced no usable analysis for this task.

The survey it planned

2 screening questions and 12 questions, as the model wrote them.

  1. S1

    Are you currently registered as a patient at Ashgrove Medical Group?

    One answer
    • Yes
    • No
    • Not sure

    Continues if Yes

  2. S2

    Which age group are you in?

    One answer
    • Under 18
    • 18 to 44
    • 45 to 64
    • 65 to 74
    • 75 or over
    • Prefer not to say

    Continues if 18 to 44; 45 to 64; 65 to 74; 75 or over

  3. Q1

    Since April 2026, when Ashgrove Medical Group introduced the online appointment request form, have you tried to book an appointment at the practice?

    One answer
    • Yes, using the online request form only
    • Yes, by phone or in person at reception only
    • Yes, both online and by phone or in person
    • No, I have not tried to book an appointment since April 2026
  4. Q2

    Thinking about your most recent attempt to book an appointment since April 2026, how satisfied or dissatisfied were you with the booking process overall?

    Scale

    Scale 1-5: Very dissatisfied to Very satisfied

    Routing Shown only if Q1 is: Yes, using the online request form only; Yes, by phone or in person at reception only; Yes, both online and by phone or in person

  5. Q3

    Thinking about that same attempt, how easy or difficult was it to request an appointment?

    Scale

    Scale 1-5: Very difficult to Very easy

    Routing Shown only if Q1 is: Yes, using the online request form only; Yes, by phone or in person at reception only; Yes, both online and by phone or in person

  6. Q4

    Since April 2026, has there been a time when you felt you needed an appointment at Ashgrove Medical Group, but decided not to book or gave up trying to book because of the booking process?

    One answer
    • Yes, more than once
    • Yes, once
    • No, I have always been able to book when I needed to
    • Not applicable - I have not needed an appointment since April 2026
  7. Q5

    What put you off booking, or caused you to give up trying? (Select all that apply)

    Any that apply
    • The online form was difficult to use or took too long
    • I do not have reliable internet access or a suitable device
    • I prefer to speak to someone rather than explain my health issue in writing
    • I was unsure how to describe my symptoms in English
    • I did not want to wait for a text message or phone call back
    • I could not get through on the phone or was told to use the online form
    • I was worried about privacy or sharing personal details online
    • Another reason

    Routing Shown only if Q4 is: Yes, more than once; Yes, once

  8. Q6

    In future, how would you personally prefer to request a routine GP appointment?

    One answer
    • By filling in an online request form
    • By phoning reception
    • In person at the reception desk
    • No preference - any of these is fine
    • Another way
  9. Q7

    Ashgrove Medical Group is deciding how to run appointment booking in future. Which of the following options would you most support?

    One answer
    • Keep online-first booking as it is now (most requests online, with reception phones kept for patients who cannot go online)
    • Keep online booking for most patients, and add a dedicated phone line and in-person help at reception for patients who need extra support (such as patients aged 75 or over, disabled patients, and patients with limited English)
    • Bring back a morning phone booking line open to all patients, alongside the online request form
    • Not sure / no strong view
  10. Q8

    Which of the following changes would make booking an appointment easier for you? (Select up to 3)

    Any that apply
    • A shorter or simpler online request form
    • A morning phone booking line open to any patient
    • A dedicated phone line for patients who struggle to go online
    • In-person help at reception to complete booking requests
    • Online form and booking instructions available in other languages
    • Clearer information on when to expect a reply after submitting a request
    • Ability to state preferred appointment times or clinician on the form
    • None of these - booking is already easy for me
    • Something else
  11. Q9

    Is there anything else you would like to tell the practice about booking appointments, or one thing that would make it easier for you? (Optional)

    Open text
  12. Q10

    Do you have any physical or mental health conditions or illnesses lasting or expected to last 12 months or more that reduce your ability to carry out day-to-day activities?

    One answer
    • Yes, a lot
    • Yes, a little
    • No
    • Prefer not to say
  13. Q11

    Do you ever need an interpreter or help with English when contacting or speaking with the GP practice?

    One answer
    • Yes, always or most of the time
    • Yes, sometimes
    • No, never
    • Prefer not to say
  14. Q12

    How confident do you feel filling in online forms on a phone, tablet, or computer without help from someone else?

    One answer
    • Very confident
    • Fairly confident
    • Not very confident
    • Not at all confident / I cannot do this without help
    • Prefer not to say
Sample plan

INVITATION AND FIELDWORK: Under the stated constraint, send an SMS invitation with a unique online survey link to all registered adult patients (aged 18+, from the ~17,100 adult list) who have a mobile number on record and have not opted out of SMS communications. To avoid younger and more digitally active patients filling the sample immediately, release invites in age-stratified waves with targeted SMS reminders to under-filled older bands (65 to 74 and 75 or over), closing each age band once its quota is reached. QUOTAS AND PRECISION TRADE-OFF (CRITICAL FLAG): Achieving plus or minus 5 percentage points at 95% confidence (worst-case p=0.50) within EACH of the four age bands requires approximately 385 completes per band before finite population correction (or ~325 to 365 per band after finite population correction for a 17,100-adult register, totaling ~1,400 to 1,500 completes). With the stated cap of ~900 completes, the optimal allocation is equal quotas of 225 completes per age band (18 to 44, 45 to 64, 65 to 74, 75 or over), which yields a margin of error of approximately plus or minus 6.5 percentage points per band (about plus or minus 6.2pp in the smaller older bands with finite population correction) and wider margins on filtered questions such as Q2, Q3, and Q5. Recommend to partners that they either expand the target to ~1,450 completes (350 to 385 per band) to meet the plus or minus 5pp subgroup precision goal, or retain 900 completes (225 per band) and accept plus or minus 6.5pp subgroup precision. COVERAGE AND MODE BIAS WARNING (CRITICAL FLAG): An SMS-to-online-only survey systematically excludes or under-represents patients without a mobile number, smartphone, internet access, or English literacy, as well as many over-75s and disabled patients - the exact groups central to this decision - which would artificially inflate reported satisfaction and preference for online-first booking. Strongly recommend supplementing the SMS fieldwork with offline modes (postal/paper questionnaires at reception for patients without mobile numbers or aged 75+, brief telephone outreach to older non-responders, and translated/interpreter-assisted completion for top practice languages) and explicitly caveating any SMS-only findings. WEIGHTING AND DATA QUALITY: Apply post-stratification weights by age band (matching the true adult practice register proportions) for overall practice-level estimates, while keeping unweighted within-band analyses for subgroup comparisons. Use unique links to prevent duplicate entries and flag extreme speeders in cleaning rather than using trick attention-check questions that could unfairly exclude older patients or respondents with cognitive disabilities or limited English.