Benchmarks / SurveyBench / How Ashgrove patients find online-first booking / Mercury 3
Mercury 3: How Ashgrove patients find online-first booking
The decision: whether to keep online-first booking as it is, restore a morning phone booking line for every patient, or add a dedicated phone line and in-person help at reception only for patients who need them (over-75s, patients with a disability, and patients with limited English). The brief and the data →
- Verdict
- ✗ Not sound
- Research score
- 41 of 100
- Analysis rating
- 752
- Head to head, this task
- won 2 of 13
Why it is not sound
The survey
- A double-barrelled question: Q8
The analysis
- Got at least one set number wrong
- Made a recommendation that does not follow from the data
- Missed a trap: Patients aged 65 or over are 24% of registered adults but 9.8% of respondents (text link, online only). They are less satisfied and prefer the phone, so weighting to the age mix lowers satisfaction among those who tried to book from 50.8% to 46.3% and raises the phone preference from 40.7% to 45.0%.
- Missed a trap: Q2 satisfaction (50.8%) is asked only of the 549 who tried to book. 24.3% of all respondents needed an appointment but did not try to book (35.1% of those who did not try to book); those who did not try are left out of the satisfaction base, which flatters it.
- Missed a trap: Last year's 71% came from a paper survey handed out in the waiting room before the change, to patients who had got an appointment; this year's 50.8% is from an online survey by text of those who tried to book. Different method and different people.
- Missed a trap: The practice manager believes patients prefer the new system. Unweighted, the online form leads (46.2% against 40.7% for phone), but only 18-44s prefer it (58.4%); weighted to the age mix, preference is split (40.7% online, 45.0% phone, 6.7% in person).
- Missed a trap: Only 886 of 9,840 patients texted completed (about 9%), and the 7,260 registered adults without a mobile number on record were never invited; patients who cannot or do not go online are the least likely to have answered.
Scorecard
Numbers 0 of 11 right
| Question | Its answer | True | |
|---|---|---|---|
| Among respondents who tried to book in the last three months, what percentage were satisfied (fairly or very) with the booking process? | ✗ | not given | 50.8% |
| Weighting each age band's percentage satisfied (fairly or very, among those who tried to book) by that band's share of registered adults, what is the overall percentage satisfied? | ✗ | not given | 46.3% |
| What percentage of all respondents said there was a time in the last three months when they needed an appointment but did not try to book one? | ✗ | not given | 24.3% |
| Weighted to the age mix of registered adults, what percentage would prefer to book by phone? | ✗ | not given | 45% |
| What is the 95% margin of error, in percentage points, on the unweighted percentage satisfied among respondents who tried to book? | ✗ | not given | 4.18 points |
| Among respondents who tried to book, is the difference in the percentage satisfied (fairly or very) between female and male respondents statistically significant at the 95% level? (1 for yes, 0 for no) | ✗ | not given | No |
| How many of the open-text answers ask to be able to book by phone, or for a phone booking line? | ✗ | not given | 97 |
| How many of the open-text answers from respondents aged 65 or over ask to be able to book by phone, or for a phone booking line? | ✗ | not given | 13 |
| Weighted to the age mix of registered adults, about how many of the practice's 17,100 registered adults would say there was a time in the last three months when they needed an appointment but did not try to book one? | ✗ | not given | 4,393 |
| Among respondents aged 18 to 44, what percentage would prefer to book using the online request form? | ✗ | not given | 58.4% |
| What percentage of the patients sent the text message completed the survey? | ✗ | not given | 9% |
Traps in the data 2 of 7 handled
- ✗Patients aged 65 or over are 24% of registered adults but 9.8% of respondents (text link, online only). They are less satisfied and prefer the phone, so weighting to the age mix lowers satisfaction among those who tried to book from 50.8% to 46.3% and raises the phone preference from 40.7% to 45.0%.
- ✗Q2 satisfaction (50.8%) is asked only of the 549 who tried to book. 24.3% of all respondents needed an appointment but did not try to book (35.1% of those who did not try to book); those who did not try are left out of the satisfaction base, which flatters it.
- ✗Last year's 71% came from a paper survey handed out in the waiting room before the change, to patients who had got an appointment; this year's 50.8% is from an online survey by text of those who tried to book. Different method and different people.
- ✗The practice manager believes patients prefer the new system. Unweighted, the online form leads (46.2% against 40.7% for phone), but only 18-44s prefer it (58.4%); weighted to the age mix, preference is split (40.7% online, 45.0% phone, 6.7% in person).
- ✓Only 13 respondents need an interpreter; 61.5% of them would prefer to book in person.
- ✓Among those who tried to book, women look less satisfied than men (47.5% against 55.3%, bases 303 and 237), but the difference is not statistically significant at 95%.
- ✗Only 886 of 9,840 patients texted completed (about 9%), and the 7,260 registered adults without a mobile number on record were never invited; patients who cannot or do not go online are the least likely to have answered.
The analysis
Findings
Open-text themes it coded
<theme in the open-text answers> 1... 0
The survey it planned
2 screening questions and 10 questions, as the model wrote them.
-
S1
Are you aged 18 or over and registered as a patient at Ashgrove Medical Group?
One answer- Yes
- No
Continues if Yes
-
S2
What is your age group?
One answer- 18-44
- 45-64
- 65-74
- 75 or over
Routing Shown only if S1 is: Yes
Continues if 18-44; 45-64; 65-74; 75 or over
-
Q1
Since April 2026, have you needed an appointment at Ashgrove Medical Group for yourself or someone you help care for?
One answer- Yes, and I booked an appointment
- Yes, but I did not book an appointment
- No, I have not needed an appointment
-
Q2
Since April 2026, have you put off booking an appointment (delayed or avoided booking) because the booking process was difficult?
One answer- Yes
- No
Routing Shown only if Q1 is: Yes, and I booked an appointment; Yes, but I did not book an appointment
-
Q3
What stopped you from booking or made it difficult to book? Select all that apply.
Any that apply- The online request form was difficult to find, use, or complete
- I don't have internet access or I am not confident using online forms
- I could not get through by phone
- The phone line was not available when I called
- I did not know how to book
- There were no appointments that suited me
- My health problem got better or I decided I no longer needed an appointment
- Something else
Routing Shown only if Q2 is: Yes
-
Q4
Thinking about the most recent appointment you booked at Ashgrove since April 2026, how satisfied are you with the way you booked it?
ScaleScale 1-5: Very dissatisfied to Very satisfied
Routing Shown only if Q1 is: Yes, and I booked an appointment
-
Q5
If you could choose, which way would you most prefer to book an appointment at Ashgrove? Please answer even if you have not needed an appointment since April 2026.
One answer- Online request form on the website
- Telephone call to reception
- In person at reception
- No preference
- Another way (please tell us in the final question)
-
Q6
Which of the following approaches to appointment booking would you prefer Ashgrove to use?
One answer- Keep the current online-first system as it is
- Bring back a morning phone booking line for every patient
- Add a dedicated phone line and in-person help at reception only for patients who need them (for example, over-75s, patients with a disability, or patients who need an interpreter)
- Not sure / no preference
-
Q7
What would make booking an appointment easier for you? Select all that apply.
Any that apply- A simpler online request form
- More appointments available
- A phone line available at set times (for example, in the morning)
- Help at reception to book in person
- Help over the phone to fill in the online form
- Reminders or instructions about how to book
- Nothing – I find booking easy already
- Other (please tell us in the final question)
-
Q8
Do you have a disability or long-term health condition?
One answer- Yes
- No
- Prefer not to say
Judges: double-barrelledOne answer combines disability and long-term health conditions. A respondent can have a long-term condition without a disability, so a Yes cannot identify the disability subgroup required by O3.
Asks about disability or long-term health condition in a single question.
-
Q9
When you contact Ashgrove, do you need help communicating in English (for example, an interpreter, a family member, or translation support)?
One answer- Yes
- No
- Prefer not to say
-
Q10
Is there anything else that would make booking an appointment easier for you? If you selected 'Another way' or 'Other' above, please tell us here.
Open text
Sample plan
Invite all registered adult patients with a valid mobile number on record (approximately 16,000-17,000) via SMS with a unique link. Target 1,540 completed surveys - not 900 - because plus/minus 5 percentage points at 95% confidence for each of four age bands requires at least 385 completes per band. Set quotas of 385 completes in each age band (18-44, 45-64, 65-74, 75+), use targeted reminder texts to under-filled age bands, and close the survey for an age band only after its quota is reached. Make the survey mobile-friendly and accessible; provide translated versions and assisted completion where possible. If the 900-complete cap is fixed, the precision target cannot be met (roughly 225 per age band gives plus/minus 6.5 percentage points).