Benchmarks / SurveyBench / Why Riverside Food Network is losing volunteers / Muse Ember

Measured by Spring Prompt

Muse Ember: Why Riverside Food Network is losing volunteers

The decision: how to spend next year's £45,000 for volunteer support: reimburse volunteers' travel costs, fund a part-time volunteer coordinator with online shift booking, or do both at a smaller scale. The brief and the data →

Verdict: ✗ Not sound
Research score: 85 of 100
Analysis rating: 1,392
Head to head, this task: won 10 of 14

Why it is not sound

The survey

Answer bands that overlap or leave gaps
A double-barrelled question: Q2
A double-barrelled question: Q5

Scorecard

Numbers 12 of 12 right

Question		Its answer	True
What percentage of active volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?	✓	37.2%	37.2%
What percentage of lapsed volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?	✓	40.2%	40.2%
Is the difference between active and lapsed respondents in the share choosing 'The cost of travelling to the site' in Q2 statistically significant at the 95% level? (1 for yes, 0 for no)	✓	No	No
Weighted to the age mix of all 1,000 lapsed volunteers, what percentage of lapsed volunteers would choose 'The cost of travelling to the site' as a difficulty in Q2?	✓	47.4%	47.4%
What percentage of lapsed respondents gave a scheduling reason as their main reason for stopping in Q3 (either of its first two options: shifts changed at short notice, or hard to arrange shifts)?	✓	42.9%	42.9%
What is the 95% margin of error, in percentage points, on that unweighted share of lapsed respondents giving a scheduling reason as their main reason?	✓	9.2 points	9.17 points
On average, how many difficulties did each respondent choose in Q2 (counting every option except 'None of these', which counts as zero), across all respondents?	✓	1.55	1.55
What percentage of lapsed respondents said they would be likely (fairly or very) to volunteer again if they could arrange their shifts online (Q4)?	✓	40.2%	40.2%
How many of the open-text answers mention shift scheduling (booking or swapping shifts, or last-minute changes or cancellations)?	✓	37	34
What percentage of the active volunteers invited completed the survey?	✓	52.4%	52.4%
What percentage of the lapsed volunteers invited completed the survey?	✓	11.2%	11.2%
If the lapsed respondents were representative, about how many of the 1,000 lapsed volunteers would give a scheduling reason (either of the first two Q3 options) as their main reason, using the unweighted share?	✓	429	429

Traps in the data 8 of 8 handled

✓Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
✓The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.
✓Q2 is multi-select: respondents chose 1.6 difficulties on average, so columns add up to more than 100%.
✓52% of active volunteers responded but only 11% of lapsed ones, so the lapsed base is small (112) and those who replied may differ from the 89% who did not (for example, still feeling some attachment).
✓Volunteers aged 65 or over are 43% of respondents but about 26% of volunteers in the database; younger volunteers cite travel cost more. Weighted to the lapsed age mix, travel cost among lapsed volunteers rises from 40% to 47%; a scheduling main reason goes from 43% to 41%, still well ahead of travel cost (21%).
✓Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
✓The travel-cost gap between lapsed and active respondents (3.0 points) is not significant at 95%.
✓40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.

The analysis

Recommendation

Spend most of the 45,000 pounds on a part-time volunteer coordinator plus an online booking system, and make the coordinator's job explicitly to publish rotas well ahead, keep shifts stable, send proactive email/text alerts when anything changes, and offer phone-assisted booking for volunteers who cannot or do not want to book online. Do not spend the whole budget on universal travel reimbursement, and do not launch an under-25-only travel scheme yet. Reserve a capped minority of the budget, once coordinator and platform costs are known, for a time-limited travel-assistance pilot (for example bus passes or a capped per-shift contribution, allocated by travel burden rather than by age alone, with under-25 uptake and return tracked separately) to test whether travel support produces incremental shifts and returns per pound, and release any scale-up funding only if pilot and follow-up evidence meet pre-set thresholds.

What would change it

The recommendation would change if a representative, age-weighted follow-up with adequate under-25 numbers showed travel cost, not scheduling, as the main reason for leaving, or showed stated return from paid travel exceeding return from online booking; if itemised costing showed the coordinator and platform cannot be delivered within the 45,000 pounds or that a capped travel payment is cheap enough to cover all active and returning volunteers without crowding out the coordinator; if pilots showed online booking does not increase bookings, swaps or lapsed returns in practice, or disproportionately excludes older volunteers, while the travel pilot shows high uptake, low deadweight (recipients would not have volunteered anyway) and strong incremental return per pound; or if site leads cannot commit to a rota-stability and advance-notice protocol, since much of the scheduling benefit in Q2 and open text depends on fixing last-minute changes and notification, not just adding a booking page.

Findings

Scheduling, not travel cost, is the main driver of leaving: prioritise the coordinator and online booking over universal travel reimbursement.

High confidenceBase 112

Q3 main reason among 112 lapsed respondents: scheduling 42.9% (48/112; shifts changed at short notice 29.5% plus hard to arrange 13.4%) versus travel cost 11.6% (13/112) and circumstances changed 23.2% (26/112). Q2 shows the same pattern: lapsed respondents cite shifts changed at short notice 51.8%, no easy way to arrange shifts 42.0%, and not being told in good time 41.1%, compared with active respondents at 14.4%, 11.6% and 7.4%.
Answer to the operations lead: No. Travel cost being the most-mentioned difficulty in Q2 does not show the whole 45,000 pounds should go on travel reimbursement.

High confidenceBase 327

Q2 is multi-select: travel cost is highest single mention at 38.2% of 327 respondents (125 people), but respondents chose on average 1.55 difficulties, so a mention is not a main reason or a cause of leaving. Travel is cited almost as often by active volunteers who stayed (37.2% of 215) as by lapsed volunteers (40.2% of 112); the 3.0 percentage point gap is not statistically significant (about 0.53 standard errors). Only 11.6% of lapsed respondents give travel cost as their main reason for stopping, and only 26.8% of lapsed respondents (30/112) say they would be likely to return if travel were paid (Q5), versus 40.2% for online booking (Q4).
Answer to the chair: Online booking is the more promising route back, but 40.2% likely to return is a stated intention, not a forecast of actual returns. Projected naively that is about 402 of the 1,000 lapsed volunteers, with very wide uncertainty.

Low confidenceBase 112

Q4: 40.2% of 112 lapsed respondents (45 people) are fairly or very likely to return if they could arrange shifts online. Among the 48 whose main reason was scheduling, 60.4% (29/48) say likely, versus 14.3% of the 14 travel-main-reason respondents and 28.0% of the 50 other-main-reason respondents. The 95% margin on 40.2% with n=112 is about plus or minus 9 percentage points, i.e. roughly 310 to 490 of 1,000 before allowing for the 11.2% lapsed response rate, age bias, and the intention-behaviour gap. Online booking also does not by itself promise fixed shifts or advance notice, which many respondents ask for.
Answer to the volunteer manager: Do not start an under-25-only travel scheme on this evidence alone. The young travel signal is striking in direction but rests on 11 lapsed respondents.

Low confidenceBase 11

Among lapsed under-25 respondents, 72.7% (8/11) give travel cost as main reason in Q3 and 81.8% (9/11) mention travel cost in Q2 (lapsed by age table). With n=11 the 95% margin is roughly plus or minus 26 percentage points. The sample also under-represents young lapsed volunteers: 9.8% of lapsed respondents are under 25 versus 22% of the 1,000 lapsed population. Travel cost is also cited by older lapsed respondents (25 to 44: 42.9%, 45 to 64: 47.2%, 65 or over: 22.7%), and open-text travel mentions include pensioners and drivers, so an age-only scheme risks inequity and misses need.
The scheduling problem is unpredictability and late notice, not just the absence of a booking website. A coordinator who only adds self-service booking without stabilising shifts and notifying people will not capture the stated benefit.

Medium confidenceBase 112

Q2 lapsed: shifts changed at short notice 51.8%, not being told about changes in good time 41.1%, and no easy way to arrange shifts 42.0% - three overlapping but distinct problems. In open text, 37 of 109 answers mention scheduling in the defined sense (booking or swapping, last-minute changes or cancellations, including notice of them), with many asking for a week's notice, fixed shifts, confirmation, or a text/email when a shift moves. Q2 'None of these' is 28.4% for active but only 2.7% (3/112) for lapsed respondents.
The lapsed results cannot be generalised confidently to all 1,000 lapsed volunteers, and unweighted travel burden is understated. Any population projection must be treated as indicative until non-response is addressed.

High confidenceBase 327

Completion was 215 of 410 active volunteers invited (52.4%) but only 112 of 1,000 lapsed volunteers (11.2%). Lapsed respondents are 39.3% aged 65 or over versus 25% in the lapsed population, and 9.8% under 25 versus 22%. Age-weighting Q2 travel cost for lapsed respondents to the population age mix (22%, 30%, 23%, 25%) raises it from 40.2% unweighted to 47.4%, showing sensitivity to the skewed sample while the under-25 cell (n=11) makes that weight unstable. Q3 scheduling projection of 429 of 1,000 uses the unweighted 42.9%.
Universal travel reimbursement would largely subsidise people who already volunteer, so its case must be made on retention and fairness, not mainly on reactivating lapsed volunteers.

Medium confidenceBase 215

37.2% of 215 active respondents (about 80 people) cite travel cost in Q2, essentially the same as lapsed respondents at 40.2%. Open text includes 22 travel-cost mentions across active and lapsed respondents (for example bus fares, a 3.80 pound tram fare, 5.20 pound day ticket, 12 pound fuel trip, 9 pound train fare), but the survey collected no mode, distance, frequency or spend data with which to cost universal reimbursement against the 45,000 pounds. Even among the 14 lapsed respondents whose main reason was travel, only 57.1% (8/14) say they would likely return if travel were paid.
There is no clear single-site problem in these data; the scheduling and communication failure appears system-wide, supporting a central coordinator model rather than fixing one site.

Medium confidenceBase 327

Q6 respondents are spread across all nine sites, from 6.7% (Eastfield) to 13.1% (Bridge End) of 327, consistent with the roughly even spread in the population stated in the brief, with 6.4% volunteering at more than one site about equally. Circumstances changed (23.2% of lapsed respondents' main reason) also set a ceiling: about a quarter left for work, study, health or moving reasons that neither intervention directly removes.
Open text corroborates a divided volunteer body: about a third say nothing would help, while the rest cluster on scheduling and travel, so a single universal offer will miss many needs.

Medium confidenceBase 109

109 open-text answers: 38 say nothing/nothing would help, 37 mention scheduling, 22 mention travel cost, 9 mention distance/journey time or a nearer site, with only 1 training and 2 feeling-thanked mentions as primary themes (see themes). Only 109 of 327 respondents answered Q1, so these are self-selected counts, not population rates.

Caveats

Lapsed response was only 11.2% (112 of 1,000) versus 52.4% for active volunteers (215 of 410), and lapsed respondents are strongly age-skewed (65 or over 39.3% versus 25% in population; under 25 9.8% versus 22%), so unweighted lapsed estimates for Q3, Q4 and Q5 are likely biased and age-weighting is unstable because the youngest lapsed cell has n=11.
Respondents are classed by their own answer to S1, not by the database, and S2 only continues lapsed respondents whose last shift was less than 12 months ago, so some lapsed volunteers may have been screened out or misclassified; the number screened out is not reported.
Q2 is multi-select and measures whether a difficulty was ever experienced, not its severity, frequency, cost or whether it caused leaving; the 1.55 average number of difficulties per respondent confirms multiple overlapping issues.
Q3 forces a single 'main reason', which simplifies multi-causal leaving and separates scheduling from circumstances even though the two may interact (for example last-minute changes clashing with work or childcare).
Q4 and Q5 are hypothetical stated intentions with no time frame, no trade-off between the two interventions, and no commitment; they will generally overstate actual return, and Q4's online-booking wording does not guarantee that shifts will not change at short notice.
Key subgroups are very small: lapsed under 25 n=11, lapsed 25 to 44 n=21, and the Q4/Q5 'travel main reason' group n=14, so subgroup percentages have margins of roughly 20 to 26 percentage points and should not drive an age-restricted scheme.
The survey contains no data on the cost of a part-time coordinator, platform licence, per-volunteer travel spend, travel mode, distance, income, car ownership, disability or digital access, so the affordability of the 45,000 pound options and the equity or digital-exclusion risks of online-only booking (in a volunteer body where 65 or over are 30% of active volunteers in the population and 45.1% of active respondents) cannot be assessed from these tables.
Open text was answered by only 109 of 327 respondents (33.3%), is self-selected, and coding answers about notice versus booking versus last-minute change involves judgement; the count of 37 scheduling mentions depends on treating mentions of a shift being moved, dropped or cancelled, and requests for advance notice, as scheduling.
There is no experimental or longitudinal evidence in the survey: the association between scheduling difficulties and having lapsed could partly reflect recall or attribution differences, and site-lead practices are not measured, so causal effects of either intervention remain untested.

Next steps

researchObtain itemised quotes and a specification for the part-time coordinator and online booking platform, including licence, setup, alerts, accessibility and assisted-booking arrangements, and set out how the 45,000 pounds would be split between coordinator, platform and a capped travel pilot.
experimentPilot online booking plus a rota-stability and notification protocol (publish ahead, freeze period, automatic alert on any change) in a subset of sites, with phone-assisted booking available, and compare against non-pilot sites over about three months.
experimentRun a small, time-limited travel-assistance pilot allocated by travel burden (distance, mode, fare) rather than age alone, with under-25s tracked as a separate cohort, and record redemption, attendance and whether recipients would have volunteered without it.
researchRecontact non-responding lapsed volunteers with a booster survey that oversamples under-25 and 25 to 44 year olds, applies age and site weights, and asks about travel mode, distance, spend per shift, digital access, and a forced trade-off between online booking and travel payment.
monitorPut in place a monthly dashboard by site and age group covering rota lead time, percentage of shifts changed inside the freeze period, notification lead time, online versus assisted bookings, and volunteer returns and retention.
decisionHold a stage-gate decision after the pilots and booster survey to reallocate the remaining budget between coordinator capacity, platform and travel support, or to stop an element that shows no incremental benefit.

Open-text themes it coded

Nothing would make it easier / satisfied as is 38Shift scheduling: booking, swapping, last-minute changes or cancellations, and notice of changes 37Travel cost (bus, train, fuel, parking charges, mileage) 22Distance, journey time and nearer site 9Parking availability and physical access 2Training 1Feeling thanked or valued 2

The survey it planned

2 screening questions and 10 questions, as the model wrote them.

S1
Have you volunteered with Riverside Food Network in the last 12 months (done at least one shift)?
One answer
- Yes
- No
Continues if Yes
S2
When did you last do a shift with Riverside Food Network?
One answer
- In the last 3 months
- More than 3 months ago but within the last 12 months
- More than 12 months ago
- I have never volunteered
Routing Shown only if S1 is: Yes

Continues if In the last 3 months; More than 3 months ago but within the last 12 months
Q1
At which Riverside site did you mainly volunteer in the last 12 months?
One answer
- Site 1
- Site 2
- Site 3
- Site 4
- Site 5
- Site 6
- Site 7
- Site 8
- Site 9
- I volunteered at more than one site equally
- Don't know
Q2
When you were volunteering in the last 12 months, which of the following made volunteering difficult for you? Please tick all that apply.
Any that apply
- Booking or swapping shifts by phone or message
- Shift days or times not fitting around my other commitments
- Work commitments
- Family or caring commitments
- Health or physical demands
- Communication with the site team, or not feeling valued
- Time or distance to get to the site
- Cost of bus fares or parking to get to the site
- Something else
- None of these
Judges: double-barrelled
The option 'Communication with the site team, or not feeling valued' combines distinct difficulties in one response, so selecting it does not establish which difficulty occurred.
The option 'Communication with the site team, or not feeling valued' combines two distinct issues, making it double-barrelled.
Q3
When you were volunteering in the last 12 months, about how much did you usually spend on bus fares or parking for one shift, including the return journey? Please enter 0 if you did not pay for travel.
Number
Q4
When you were volunteering in the last 12 months, how much of a burden was the cost of travelling to the site for you?
Scale

Scale 1-5: Not at all a burden to A very great burden
Q5
What was the main reason you stopped volunteering with Riverside Food Network?
One answer
- Shift days or times not fitting around my other commitments
- Work commitments
- Family or caring commitments
- Health or physical demands
- Moved house or area
- Booking or swapping shifts by phone or message
- Lost interest, or did not feel valued
- No suitable shifts or role available
- Cost of travelling to the site
- Something else
Routing Shown only if S2 is: More than 3 months ago but within the last 12 months

Judges: double-barrelled
Having no shift in the last three months does not establish that someone has stopped volunteering; respondents taking a temporary break have no applicable response. 'Lost interest, or did not feel valued' also combines distinct reasons in one answer. Without an actual stopped/still-volunteering measure, O1 and O2 cannot reliably distinguish those populations.
The option 'Lost interest, or did not feel valued' combines two distinct reasons, making it double-barrelled.
Q6
If you could book and swap shifts online yourself, without phoning or messaging the site lead, how likely would you be to volunteer again at least once a month?
Scale

Scale 1-5: Very unlikely to Very likely

Routing Shown only if S2 is: More than 3 months ago but within the last 12 months
Q7
If Riverside paid back your bus fares or parking costs for each shift, how likely would you be to volunteer again at least once a month?
Scale

Scale 1-5: Very unlikely to Very likely

Routing Shown only if S2 is: More than 3 months ago but within the last 12 months
Q8
If both online booking and swapping, and your bus fares or parking costs paid back, were in place, how likely would you be to volunteer again at least once a month?
Scale

Scale 1-5: Very unlikely to Very likely

Routing Shown only if S2 is: More than 3 months ago but within the last 12 months
Q9
Which age group are you in?
One answer
- Under 25
- 25-34
- 35-49
- 50-64
- 65-74
- 75 or over
- Prefer not to say
Q10
Is there anything else about volunteering with Riverside that you would like to tell us, including anything that would help you keep volunteering or return to volunteering?
Open text

Sample plan

Invite all 1,410 people on the database with at least one shift in the last 12 months (410 active: last shift in last 3 months; about 1,000 lapsed: last shift 3-12 months ago) by email and text, using neutral invitation wording that does not mention the travel-cost hypothesis or favour either intervention. Target about 330 completes (about 23% of invitees). Monitor completes weekly by database active/lapsed status and site; send a reminder at day 7 to all non-respondents and a day-14 reminder targeted to active volunteers (aiming for at least 110 active completes as the smaller comparison group) and to sites with low response. Set no hard age quota, as the age distribution is not assumed, but monitor age and site (minimum 30 completes per site for separate reporting, otherwise pool). Randomise the order of Q6 and Q7. Precision warning: plus/minus 7 percentage points per site at 95% confidence needs 196 per site ignoring finite population (1,764 total), or about 87 per site if each site population is about 157 (about 783 total, 55% of all invitees) - far above the expected 330. With about 37 per site the margin is about plus/minus 16 percentage points (about 14 with finite-population correction), versus about plus/minus 5.4 points overall, so site results must be reported with confidence intervals as indicative only and trustees should not make per-site funding decisions from them unless fieldwork is extended or incentivised to reach roughly 780 completes.