Benchmarks / SurveyBench / Why Riverside Food Network is losing volunteers / Muse Wonder

Measured by Spring Prompt

Muse Wonder: Why Riverside Food Network is losing volunteers

The decision: how to spend next year's £45,000 for volunteer support: reimburse volunteers' travel costs, fund a part-time volunteer coordinator with online shift booking, or do both at a smaller scale. The brief and the data →

Verdict
✗ Not sound
Research score
89 of 100
Analysis rating
1,460
Head to head, this task
won 10 of 14

Why it is not sound

The survey

  • Answer bands that overlap or leave gaps

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What percentage of active volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓37.2%37.2%
What percentage of lapsed volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓40.2%40.2%
Is the difference between active and lapsed respondents in the share choosing 'The cost of travelling to the site' in Q2 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Weighted to the age mix of all 1,000 lapsed volunteers, what percentage of lapsed volunteers would choose 'The cost of travelling to the site' as a difficulty in Q2?✓47.4%47.4%
What percentage of lapsed respondents gave a scheduling reason as their main reason for stopping in Q3 (either of its first two options: shifts changed at short notice, or hard to arrange shifts)?✓42.9%42.9%
What is the 95% margin of error, in percentage points, on that unweighted share of lapsed respondents giving a scheduling reason as their main reason?✓9.2 points9.17 points
On average, how many difficulties did each respondent choose in Q2 (counting every option except 'None of these', which counts as zero), across all respondents?✓1.551.55
What percentage of lapsed respondents said they would be likely (fairly or very) to volunteer again if they could arrange their shifts online (Q4)?✓40.2%40.2%
How many of the open-text answers mention shift scheduling (booking or swapping shifts, or last-minute changes or cancellations)?✓3734
What percentage of the active volunteers invited completed the survey?✓52.4%52.4%
What percentage of the lapsed volunteers invited completed the survey?✓11.2%11.2%
If the lapsed respondents were representative, about how many of the 1,000 lapsed volunteers would give a scheduling reason (either of the first two Q3 options) as their main reason, using the unweighted share?✓429429

Traps in the data 8 of 8 handled

  • ✓Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
  • ✓The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.
  • ✓Q2 is multi-select: respondents chose 1.6 difficulties on average, so columns add up to more than 100%.
  • ✓52% of active volunteers responded but only 11% of lapsed ones, so the lapsed base is small (112) and those who replied may differ from the 89% who did not (for example, still feeling some attachment).
  • ✓Volunteers aged 65 or over are 43% of respondents but about 26% of volunteers in the database; younger volunteers cite travel cost more. Weighted to the lapsed age mix, travel cost among lapsed volunteers rises from 40% to 47%; a scheduling main reason goes from 43% to 41%, still well ahead of travel cost (21%).
  • ✓Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
  • ✓The travel-cost gap between lapsed and active respondents (3.0 points) is not significant at 95%.
  • ✓40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.

The analysis

Recommendation

Prioritise the coordinator-plus-online-booking option, and do both at smaller scale rather than travel-only: spend the majority of the £45,000 on a part-time volunteer coordinator who owns a single online rota for booking/swapping across sites, mandatory change alerts (text/email) and a minimum-notice standard for shift changes, retaining a phone route via site leads for digitally excluded volunteers. Ring-fence a smaller, capped portion for a time-limited, targeted travel reimbursement pilot (e.g. claimable bus/tram fares and parking, prioritised by need/distance rather than an under-25-only rule, but actively promoted to under-25s and low-income volunteers). Do not spend the full £45,000 on universal travel reimbursement: it addresses a real friction for current volunteers but only 12.5% of lapsed cite travel as their main reason for leaving, whereas 42.9% cite scheduling. Evaluate both elements on actual behaviour before committing recurrent funding.

What would change it

The recommendation would change if: (1) a real-offer pilot showed online booking produced few actual re-bookings while a travel offer produced materially more returned shifts per pound; (2) database/operational audit showed most short-notice changes are unavoidable and a coordinator could not reduce them, making software-only change cosmetic; (3) take-up data showed travel reimbursement strongly retained active volunteers at risk of leaving (not just lapsed return), shifting the objective from reactivation to retention; or (4) properly sampled evidence (adequate under-25 base) showed travel is the binding constraint for a large, reachable segment.

Findings

  1. Travel cost is a widespread friction, but it is not the main reason lapsed volunteers stopped, and it does not differentiate lapsed from active volunteers

    High confidenceBase 327

    Q2 travel cost: 37.2% of 215 active vs 40.2% of 112 lapsed, a 3.0 percentage-point gap well within sampling error (not significant at 95%). In Q3 only 11.6% of lapsed named travel cost as the main reason for stopping, plus 0.9% travel time. Answering the operations lead directly: being the largest single Q2 item (38.2% overall) shows prevalence, not causation or priority for retention spending.

  2. Scheduling as a system - short-notice changes, no easy way to book/swap, and poor notification - is the dominant stated cause of lapsing

    High confidenceBase 112

    Among lapsed, Q2: 51.8% shifts changed at short notice, 42.0% no easy way to arrange shifts, 41.1% not told in good time, vs 14.4%, 11.6% and 7.4% respectively among active. In Q3, 42.9% of lapsed (48 of 112; 29.5% short-notice changes + 13.4% hard to arrange shifts) gave a scheduling main reason, vs 23.2% changed circumstances and 12.5% travel cost/distance combined. If representative, that is about 429 of 1,000 lapsed volunteers, +/-9.2 percentage points on the share.

  3. Online booking has materially stronger stated return potential than travel reimbursement overall, especially for scheduling leavers, but it cannot be read as a forecast of returners

    Medium confidenceBase 112

    Q4 net likely to return with online booking: 40.2% of 112 lapsed (60.4% of the 48 scheduling leavers). Q5 net likely with travel costs repaid: 26.8% overall (57.1% of only 14 travel leavers, 27.1% of scheduling leavers). Answering the chair directly: 40.2% is stated intention in a hypothetical, not 402 returners; neutrals (24.1%), non-response and the intention-behaviour gap mean actual returns will be substantially lower and must be measured in a pilot.

  4. The case for an under-25-only travel scheme rests on 8-9 people and should not drive targeting on its own

    Low confidenceBase 11

    Q3: 72.7% of lapsed under-25s named travel cost as main reason, but that is 8 of a base of 11; Q2 is 81.8%, 9 of 11. Answering the volunteer manager directly: the direction is plausible, but the base is far too small for an age-specific scheme. Note also Q2 travel difficulty is high for active under-25s too (74.1% of 27), suggesting hardship/retention relevance, not only reactivation.

  5. Respondents understate the lapsed population's likely travel difficulty because younger lapsed volunteers are severely under-represented

    Medium confidenceBase 112

    Lapsed response rate was 11.2% (112/1,000) vs 52.4% active (215/410). Lapsed respondents are 9.8% under-25 vs 22% in the lapsed population, and 71.4% aged 45+ vs 48% in the population. Reweighting lapsed Q2 travel cost by the true age mix raises it from 40.2% to 47.4%. Unweighted total-sample comparisons therefore underplay travel for the lapsed population as a whole, even though it still does not explain most lapsing.

  6. Open-text answers corroborate scheduling as an operational problem of notice and control, not just absence of software

    High confidenceBase 109

    37 of 109 Q1 answers mention booking/swapping or last-minute changes/cancellations and late notification - repeatedly describing turning up to locked doors, hearing changes second-hand, and being unable to swap without phoning. An online tool without enforced notice periods, change alerts and site-lead discipline will not fix the underlying complaint.

  7. Many current volunteers need nothing new, so universal untargeted spending would be inefficient

    Medium confidenceBase 327

    38 of 109 Q1 answers say everything is fine/nothing needed, and 28.4% of active volunteers chose 'None of these' in Q2 vs 2.7% of lapsed. Respondents averaged only 1.55 difficulties each in Q2. Support should be targeted at pain points and at-risk groups rather than spread thinly as a universal benefit.

Caveats

  • Non-response bias is the biggest threat: only 11.2% of lapsed volunteers completed the survey vs 52.4% of active. Lapsed respondents who replied may be more engaged, more aggrieved about scheduling, and older (9.8% under-25 vs 22% in the lapsed population; only 11 lapsed under-25s). All lapsed estimates, especially by age, should be treated as indicative, not population-precise.
  • Self-classification and screening: active/lapsed status comes from S1 self-report, not database verification, and lapsed volunteers whose last shift was 12+ months ago were screened out, so longer-term leavers - who may be hardest to win back - are absent.
  • Q2 is multi-select prevalence, not importance: choosing a difficulty does not mean it caused leaving. Q3 forces a single main reason and may simplify multi-causal exits; 23.2% mainly left because circumstances changed, whom neither option will bring back.
  • Q4 and Q5 are hypothetical stated intentions with no cost, timing or eligibility detail. 'Likely' will overstate actual re-registration and shift-taking; do not budget on 40.2% of 1,000 returning.
  • Small bases: lapsed under-25 n=11, travel-main-reason subgroup n=14, scheduling subgroup n=48. Percentages on these bases move by 7-9 points per person. Margin of error on the headline scheduling share is +/-9.2 points even before non-response bias.
  • Open-text Q1 was answered by only 109 of 327 completers and is self-selecting; theme counts show what engaged respondents volunteered, not population prevalence. A few answers also appear internally inconsistent (e.g. stated age/site vs content), so verbatims illustrate mechanisms rather than segment truths.
  • No cost, feasibility or site-level operational data were collected: the survey does not show cost per volunteer of reimbursement, coordinator capacity across 9 sites, digital exclusion risk among older volunteers (43.1% of respondents are 65+), or whether short-notice changes are avoidable given donation flows.

Next steps

  • decisionDecide the split now: fund part-time coordinator + online booking as the core, with a capped targeted travel pilot, and set a 6-month review gate before any recurrent commitment
  • experimentPilot online booking with real invitations to lapsed scheduling leavers at 2-3 sites, with phone booking retained, and track actual re-registration and shifts completed - not clicks or stated intent
  • experimentRun the travel reimbursement as a capped pilot with simple claiming, promoted to under-25s, low-income and long-distance volunteers, and compare returned/retained shifts per pound against the booking pilot
  • researchAudit rota practice at all 9 sites for the last 3 months: frequency and notice of changes/cancellations, and how volunteers were notified, before the system launches
  • researchDo targeted follow-up research with under-represented groups: lapsed under-25s and 25-44s and non-responders, using database sampling and short interviews, oversampling to get usable bases
  • monitorMonitor active-volunteer retention, shift fill rates, no-shows and volunteer-reported difficulties quarterly, split by age and site

Open-text themes it coded

Shift scheduling: booking, swapping, last-minute changes/cancellations and notice of changes 37Travel cost: fares, fuel, mileage and parking charges/reimbursement 21Travel time, distance and proximity to a site 9Parking and physical access to site 6Training, recognition and feeling valued 3Nothing needed - fine as it is 38

The survey it planned

3 screening questions and 10 questions, as the model wrote them.

  1. S1

    Have you done at least one volunteer shift for Riverside Food Network in the last 12 months?

    One answer
    • Yes
    • No

    Continues if Yes

  2. S2

    When was your most recent volunteer shift for Riverside Food Network?

    One answer
    • Within the last 3 months
    • More than 3 months ago, but within the last 12 months
    • More than 12 months ago
    • I have never done a volunteer shift for Riverside Food Network

    Continues if Within the last 3 months; More than 3 months ago, but within the last 12 months

  3. S3

    At which Riverside Food Network distribution site have you volunteered most often in the last 12 months?

    One answer
    • [Distribution site 1 - insert site name]
    • [Distribution site 2 - insert site name]
    • [Distribution site 3 - insert site name]
    • [Distribution site 4 - insert site name]
    • [Distribution site 5 - insert site name]
    • [Distribution site 6 - insert site name]
    • [Distribution site 7 - insert site name]
    • [Distribution site 8 - insert site name]
    • [Distribution site 9 - insert site name]

    Continues if [Distribution site 1 - insert site name]; [Distribution site 2 - insert site name]; [Distribution site 3 - insert site name]; [Distribution site 4 - insert site name]; [Distribution site 5 - insert site name]; [Distribution site 6 - insert site name]; [Distribution site 7 - insert site name]; [Distribution site 8 - insert site name]; [Distribution site 9 - insert site name]

  4. Q1

    In your own words, what was the main reason you stopped volunteering for Riverside Food Network?

    Open text

    Routing Shown only if S2 is: More than 3 months ago, but within the last 12 months

    Judges: assumptive

    Having no shift in the last three months does not establish that someone has stopped volunteering. This question assumes that everyone shown it has stopped, while excluding people who stopped after a shift within the last three months. The survey therefore cannot reliably identify the stopped and continuing groups required for O1–O3.

    Shown to everyone whose last shift was more than 3 months ago. It assumes they have stopped volunteering, but some will only be taking a gap (seasonal, occasional or planned breaks) and still see themselves as volunteers. There is no option to say they have not stopped.

  5. Q2

    Thinking about the last 12 months, which of the following, if any, made it difficult for you to volunteer for Riverside Food Network? Please select all that apply.

    Any that apply
    • A change in my health
    • A change in my work, study or caring responsibilities
    • Difficulty booking or swapping shifts by phone or message
    • I moved away from the area
    • Not feeling supported or appreciated
    • Shifts were not available at times I could do
    • The cost of travelling to the site
    • The physical demands of the work
    • The time or distance involved in travelling to the site
    • Volunteering sessions felt poorly organised
    • Something else made it difficult
    • None of these - I had no difficulties
  6. Q3

    Which one of the difficulties you selected was the biggest difficulty for you?

    One answer
    • A change in my health
    • A change in my work, study or caring responsibilities
    • Difficulty booking or swapping shifts by phone or message
    • I moved away from the area
    • Not feeling supported or appreciated
    • Shifts were not available at times I could do
    • The cost of travelling to the site
    • The physical demands of the work
    • The time or distance involved in travelling to the site
    • Volunteering sessions felt poorly organised
    • Something else made it difficult

    Routing Shown only if Q2 is: A change in my health; A change in my work, study or caring responsibilities; Difficulty booking or swapping shifts by phone or message; I moved away from the area; Not feeling supported or appreciated; Shifts were not available at times I could do; The cost of travelling to the site; The physical demands of the work; The time or distance involved in travelling to the site; Volunteering sessions felt poorly organised; Something else made it difficult

  7. Q4

    About how much did you usually spend out of your own pocket travelling to and from one volunteer shift, in pounds? Please enter 0 if you usually spent nothing. If you are unsure, please give your best estimate.

    Number
  8. Q5

    If Riverside Food Network paid back in full the cost of travelling to and from each shift, with payment made within one week of the shift, how likely would you be to volunteer again in the next 3 months?

    Scale

    Scale 1-5: Very unlikely to Very likely

    Routing Shown only if S2 is: More than 3 months ago, but within the last 12 months

  9. Q6

    If you could view, book and swap volunteer shifts yourself online, instead of phoning or messaging a site lead, how likely would you be to volunteer again in the next 3 months?

    Scale

    Scale 1-5: Very unlikely to Very likely

    Routing Shown only if S2 is: More than 3 months ago, but within the last 12 months

  10. Q7

    Riverside Food Network has £45,000 next year for volunteer support. Which one of the following would you most prefer it to spend the money on?

    One answer
    • Paying back volunteers' travel costs only
    • Funding a part-time volunteer coordinator and online shift booking only
    • Both travel-cost payments and a coordinator with online booking, each at a smaller scale
    • Neither of these - spend it on other volunteer support
  11. Q8

    To show that you are reading the questions carefully, please select 'Disagree' below.

    One answer
    • Strongly agree
    • Agree
    • Neither agree nor disagree
    • Disagree
    • Strongly disagree
  12. Q9

    Which age group are you in?

    One answer
    • 16-24
    • 25-34
    • 35-44
    • 45-54
    • 55-64
    • 65-74
    • 75 or over
    • Prefer not to say
  13. Q10

    Is there anything else Riverside Food Network could do that would help you to volunteer, or to volunteer again? Please tell us in your own words.

    Open text
Sample plan

Invite everyone eligible on the volunteer database by email and text message: all approximately 1,410 people who did at least one shift in the last 12 months (410 with a shift in the last 3 months and approximately 1,000 with no shift in the last 3 months), with reminders, aiming for about 330 valid completes after removing failures of Q8. Use S2 to classify current volunteers ('Within the last 3 months') versus lapsed volunteers ('More than 3 months ago, but within the last 12 months'). Monitor completes by S2 status and S3 site during fieldwork and target reminders to under-represented strata, seeking at least 110 current and 220 lapsed completes and at least 25 completes per site if possible; weight final results back to the database proportions by status and site for overall estimates. The trustees' requested precision cannot be met: +/-7 percentage points at 95% confidence for a proportion near 50% requires about 196 completes per site (1.96^2 x 0.25 / 0.07^2), or about 1,764 across 9 sites, which exceeds both the ~330-complete target and the entire eligible population of ~1,410 before non-response. With 330 completes spread evenly (~37 per site), a site estimate near 50% has a margin of error of about +/-16 percentage points. Site-level results must therefore be reported as indicative only, with base sizes and confidence intervals shown, and the trustees must either accept wider site-level intervals, pool sites, or substantially increase the sample; increasing invitations alone cannot achieve +/-7 points per site because the population is a census frame of ~1,410 and all of it is already being invited.