Benchmarks / SurveyBench / Why Riverside Food Network is losing volunteers / GPT-6.2 Astra
GPT-6.2 Astra: Why Riverside Food Network is losing volunteers
The decision: how to spend next year's £45,000 for volunteer support: reimburse volunteers' travel costs, fund a part-time volunteer coordinator with online shift booking, or do both at a smaller scale. The brief and the data →
- Verdict
- ✗ Not sound
- Research score
- 86 of 100
- Analysis rating
- 1,467
- Head to head, this task
- won 6 of 10
Why it is not sound
The survey
- Answer bands that overlap or leave gaps
The analysis
- Missed a trap: Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
Scorecard
Numbers 12 of 12 right
| Question | Its answer | True | |
|---|---|---|---|
| What percentage of active volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2? | ✓ | 37.2% | 37.2% |
| What percentage of lapsed volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2? | ✓ | 40.2% | 40.2% |
| Is the difference between active and lapsed respondents in the share choosing 'The cost of travelling to the site' in Q2 statistically significant at the 95% level? (1 for yes, 0 for no) | ✓ | No | No |
| Weighted to the age mix of all 1,000 lapsed volunteers, what percentage of lapsed volunteers would choose 'The cost of travelling to the site' as a difficulty in Q2? | ✓ | 47.4% | 47.4% |
| What percentage of lapsed respondents gave a scheduling reason as their main reason for stopping in Q3 (either of its first two options: shifts changed at short notice, or hard to arrange shifts)? | ✓ | 42.9% | 42.9% |
| What is the 95% margin of error, in percentage points, on that unweighted share of lapsed respondents giving a scheduling reason as their main reason? | ✓ | 9.2 points | 9.17 points |
| On average, how many difficulties did each respondent choose in Q2 (counting every option except 'None of these', which counts as zero), across all respondents? | ✓ | 1.55 | 1.55 |
| What percentage of lapsed respondents said they would be likely (fairly or very) to volunteer again if they could arrange their shifts online (Q4)? | ✓ | 40.2% | 40.2% |
| How many of the open-text answers mention shift scheduling (booking or swapping shifts, or last-minute changes or cancellations)? | ✓ | 29 | 34 |
| What percentage of the active volunteers invited completed the survey? | ✓ | 52.4% | 52.4% |
| What percentage of the lapsed volunteers invited completed the survey? | ✓ | 11.2% | 11.2% |
| If the lapsed respondents were representative, about how many of the 1,000 lapsed volunteers would give a scheduling reason (either of the first two Q3 options) as their main reason, using the unweighted share? | ✓ | 429 | 429 |
Traps in the data 7 of 8 handled
- ✗Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
- ✓The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.
- ✓Q2 is multi-select: respondents chose 1.6 difficulties on average, so columns add up to more than 100%.
- ✓52% of active volunteers responded but only 11% of lapsed ones, so the lapsed base is small (112) and those who replied may differ from the 89% who did not (for example, still feeling some attachment).
- ✓Volunteers aged 65 or over are 43% of respondents but about 26% of volunteers in the database; younger volunteers cite travel cost more. Weighted to the lapsed age mix, travel cost among lapsed volunteers rises from 40% to 47%; a scheduling main reason goes from 43% to 41%, still well ahead of travel cost (21%).
- ✓Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
- ✓The travel-cost gap between lapsed and active respondents (3.0 points) is not significant at 95%.
- ✓40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.
The analysis
Choose both at a smaller scale, with rota reliability as the retention priority and travel support as an access priority. Give a part-time coordinator responsibility for publishing dependable shifts, reducing avoidable changes and sending direct alerts, alongside simple online booking and swaps. Keep phone or message booking available. Pilot a capped, easy-to-claim travel contribution open to volunteers facing cost barriers at any age. Do not promise a citywide entitlement, a precise budget split or a number of returners before costing the offers and observing attendance.
Put more of the budget into travel if nonrespondent follow-up confirms wider financial exclusion and a travel offer reduces out-of-pocket burden or enables sustained attendance at an affordable cost. Put more into coordination if sites reduce late changes and gain more sustained shifts per pound than with travel support. If the coordinator cannot influence cancellations, do not scale booking software on stated interest alone. If a combined pilot cannot deliver either offer adequately within £45,000, stage the tests rather than underfund both.
Findings
-
Operations lead: no. Travel cost was the most commonly selected difficulty overall. That makes it a legitimate volunteer-support expense, but does not show it is the best use of all £45,000.
Travel cost was selected by 125/327 respondents: 80/215 active (37.2%) and 45/112 lapsed (40.2%). The 3.0-point gap is not statistically significant (two-sided proportion test, p≈0.60). Q2 allowed multiple difficulties, averaging 1.55 per respondent; it did not ask which remedy would keep someone volunteering. Among lapsed respondents, short-notice changes were more common than travel cost (51.8% versus 40.2%).
-
For retention, prioritise a rota people can rely on, not just a website where they can book. Preventing an avoidable cancellation and promptly telling someone about an unavoidable one are different jobs.
Of 112 lapsed respondents, 33 named short-notice changes and 15 named difficulty arranging shifts as their main reason for leaving: 48/112 (42.9%, nominal margin ±9.2 points). A further 14 named late communication, outside that scheduling definition. By comparison, 13 named travel cost and 26 a change in personal circumstances. V058 described an 8am cancellation for a 9:30 start; V122 wanted both self-service swaps and a text when a shift changed.
-
Chair: the number who will come back because of online booking is unknown. There is enough interest to test it, especially among people who left over scheduling.
45/112 (40.2%) said return was fairly or very likely with online booking, versus 30/112 (26.8%) with travel paid. Among the 48 who left over scheduling, 29 (60.4%) said likely with booking; among the 14 who left over travel cost or journey length, 8 (57.1%) said likely with reimbursement. The projection of 429 per 1,000 is about reasons for leaving if respondents were representative, not returns. Neither question measured actual attendance or how many would return anyway.
-
Volunteer manager: include young volunteers in a travel pilot, but do not make under-25 status the eligibility rule on this evidence.
Of just 11 lapsed under-25 respondents, 9 reported travel cost as a difficulty and 8 named it as their main reason for stopping. One answer moves either share by 9.1 points. The signal matters, but cost also affected 17/36 lapsed respondents aged 45–64. V234, an active volunteer living on a pension, described the weekly burden of a return fare.
-
Do not use the raw lapsed result as a ceiling on travel need. Correcting the age mix raises the estimated share experiencing cost difficulty, though it still does not establish demand for a particular scheme.
Applying the database age shares (22%, 30%, 23%, 25%) to the four lapsed age groups' travel-cost rates (81.8%, 42.9%, 47.2%, 22.7%; bases 11, 21, 36, 44) gives 47.4%, versus 40.2% unweighted. This estimates difficulty, not the proportion who left because of cost or who would claim.
Caveats
- All volunteers on the two invitation lists were invited, but only 215/410 active and 112/1,000 lapsed completed. Active volunteers account for 66% of completes but only 29% of invitees, so the overall difficulty ranking is not a population ranking. Database status uses shift dates; survey status uses self-description, which may not match.
- Respondents skew older in both groups: 39.3% of lapsed respondents were 65+ versus 25% on file, and 45.1% of active respondents versus 30%. Age weighting cannot correct differences between responders and nonresponders within age groups. All nine sites are represented, but site coverage does not remove this bias.
- The ±9.2 points is the conventional unweighted 95% normal-approximation margin, 1.96 × √[p(1−p)/112], expressed in percentage points. It and the significance test describe sampling variability under random-response assumptions, not total uncertainty. A non-significant travel gap is not proof the groups are identical.
- Q3 records one recalled main reason; it does not prove causation or rule out secondary difficulties. Q4 and Q5 are separate hypothetical ratings by the same people, not a choice between funded programmes. Their likely shares cannot be added. Neither the coordinator role nor a combined offer was tested.
- The 109 optional texts illustrate mechanisms, not prevalence among all volunteers. The 29 scheduling mentions include requests to book or swap and requests for more stable shifts; a request solely to be told of a change is coded as communication, not scheduling. Reimbursement addresses price, not necessarily a long journey or lack of parking.
Next steps
- decisionCost a viable pilot before releasing the budget: coordinator hours and cover, booking and messaging costs, likely claims per shift, administration and a travel cap.
- researchContact a stratified sample of lapsed nonrespondents by phone as well as text, oversampling under-25s; interview site leads about why shifts change.
- experimentStagger the rota and travel offers across comparable sites or periods, using a common invitation to lapsed volunteers and recording which offer each person receives. Retain not-yet-treated comparisons where feasible.
- monitorReview rota performance and travel access monthly with volunteers and site leads.
Open-text themes it coded
Shift scheduling: booking, swapping, recurring slots, or keeping shifts stable rather than changing or cancelling them at short notice 29Timely, direct notice of changes rather than late messages, word of mouth or finding out on arrival 14Travel affordability: fares, fuel, paid parking, passes, reimbursement or sharing costs 22Journey time, distance, poor connections or wanting a nearer site 9Physical parking or drop-off space, distinct from parking price 2Training or induction 1Thanks or personal recognition 2No change suggested or content as things stand 38
The survey it planned
1 screening question and 10 questions, as the model wrote them.
-
S1
When did you last complete a volunteer shift with Riverside Food Network?
One answer- Within the last 3 months
- More than 3 months ago, but within the last 12 months
- More than 12 months ago
- I have never completed a shift with Riverside Food Network
- I’m not sure
Continues if Within the last 3 months; More than 3 months ago, but within the last 12 months
-
Q1
Which best describes your volunteering with Riverside at the moment?
One answer- I still volunteer, even if my shifts are far apart
- I’m taking a break
- I’ve stopped volunteering with Riverside
- I’m not sure
-
Q2
What was the main reason you stopped volunteering with Riverside or took a break? A few words are enough. You can leave this blank if you prefer.
Open textRouting Shown only if Q1 is: I’m taking a break; I’ve stopped volunteering with Riverside
-
Q3
Think about the last 3 months in which you volunteered with Riverside, or the whole time if you volunteered for less than 3 months. Which of these, if any, made it difficult for you to do shifts? Select all that apply.
Any that apply- Paid work or study commitments
- Caring responsibilities
- My health or access needs
- The shift times available
- Finding out which shifts were available
- Arranging or changing a shift
- What I had to pay to get to and from the site
- The time it took to get to and from the site
- Not having reliable transport
- The tasks I was asked to do
- Not getting the support I needed at the site
- Not feeling welcome or appreciated
- Something else
- None of these — select this on its own
- I can’t remember — select this on its own
-
Q4
About how much did you usually pay out of your own pocket to travel to a shift and back during that period? Include fares, fuel and parking. Your best estimate is fine.
One answer- £0
- More than £0 but less than £2
- £2 to less than £5
- £5 to less than £10
- £10 or more
- It varied too much to estimate
- I don’t know or prefer not to say
-
Q5
If Riverside made no changes to travel costs or how shifts are arranged, how likely would you be to do at least one shift in the next 3 months?
One answer- Very unlikely
- Fairly unlikely
- Neither likely nor unlikely
- Fairly likely
- Very likely
- I’m not sure
Routing Shown only if S1 is: More than 3 months ago, but within the last 12 months
-
Q6
Suppose Riverside paid you back for your usual travel costs for every shift. You would still arrange shifts as you do now. How likely would you be to do at least one shift in the next 3 months?
One answer- Very unlikely
- Fairly unlikely
- Neither likely nor unlikely
- Fairly likely
- Very likely
- I’m not sure
Routing Shown only if S1 is: More than 3 months ago, but within the last 12 months
-
Q7
Suppose you could see, book and change shifts online, with a part-time volunteer coordinator to help. You would still pay your own travel costs. How likely would you be to do at least one shift in the next 3 months?
One answer- Very unlikely
- Fairly unlikely
- Neither likely nor unlikely
- Fairly likely
- Very likely
- I’m not sure
Routing Shown only if S1 is: More than 3 months ago, but within the last 12 months
-
Q8
Suppose Riverside paid back half your usual travel costs for each shift. You could also see, book and change shifts online, but the volunteer coordinator would have half as much time available to help as under the online-booking-only option. How likely would you be to do at least one shift in the next 3 months?
One answer- Very unlikely
- Fairly unlikely
- Neither likely nor unlikely
- Fairly likely
- Very likely
- I’m not sure
Routing Shown only if S1 is: More than 3 months ago, but within the last 12 months
-
Q9
At which Riverside site did you do the most shifts in the last 12 months? If two sites were equal, give the one you volunteered at more recently. The site name, street or neighbourhood is enough.
Open text -
Q10
What is your age group?
One answer- Under 25
- 25–34
- 35–44
- 45–54
- 55–64
- 65 or over
- Prefer not to say
Sample plan
Invite everyone recorded as having completed a shift in the last 12 months (410 active and about 1,000 lapsed), using email and text where available and the same one-use mobile-friendly link per person. Use a neutral invitation about volunteers’ experiences, not an appeal for travel reimbursement. Pilot the longest route on phones to verify completion within 6 minutes. Aim for about 330 completes; no hard quotas or closing of site cells. Monitor soft targets of roughly 96 active and 234 lapsed overall, track coverage at all nine sites, and prioritise reminders to nonrespondents in underrepresented groups. The requested site precision cannot be promised: even equal allocation gives only about 37 responses per site. For illustration, for a site with 157 eligible volunteers, a simple random sample of 37 gives a worst-case 95% margin of about ±14 points after finite-population correction; about 88 responses would be needed for ±7. Calculate requirements separately using actual site counts. An invitation census with voluntary response is not a random sample: more responses improve coverage but do not guarantee unbiased site estimates. Agree before fieldwork to treat site results as indicative or resource substantially more follow-up if precise site estimates are essential.