Benchmarks / SurveyBench / Why Riverside Food Network is losing volunteers / Mercury 3

Measured by Spring Prompt

Mercury 3: Why Riverside Food Network is losing volunteers

The decision: how to spend next year's £45,000 for volunteer support: reimburse volunteers' travel costs, fund a part-time volunteer coordinator with online shift booking, or do both at a smaller scale. The brief and the data →

Verdict
✗ Not sound
Research score
90 of 100
Analysis rating
752
Head to head, this task
won 11 of 15

Why it is not sound

The survey

  • A double-barrelled question: Q3
  • A double-barrelled question: Q5

The analysis

  • Proposed a mistaken next step: Set up a travel scheme for under-25s on the strength of the lapsed under-25 result

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What percentage of active volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓37.2%37.2%
What percentage of lapsed volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓40.2%40.2%
Is the difference between active and lapsed respondents in the share choosing 'The cost of travelling to the site' in Q2 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Weighted to the age mix of all 1,000 lapsed volunteers, what percentage of lapsed volunteers would choose 'The cost of travelling to the site' as a difficulty in Q2?✓47.4%47.4%
What percentage of lapsed respondents gave a scheduling reason as their main reason for stopping in Q3 (either of its first two options: shifts changed at short notice, or hard to arrange shifts)?✓42.9%42.9%
What is the 95% margin of error, in percentage points, on that unweighted share of lapsed respondents giving a scheduling reason as their main reason?✓9.2 points9.17 points
On average, how many difficulties did each respondent choose in Q2 (counting every option except 'None of these', which counts as zero), across all respondents?✓1.551.55
What percentage of lapsed respondents said they would be likely (fairly or very) to volunteer again if they could arrange their shifts online (Q4)?✓40.2%40.2%
How many of the open-text answers mention shift scheduling (booking or swapping shifts, or last-minute changes or cancellations)?✓3734
What percentage of the active volunteers invited completed the survey?✓52.4%52.4%
What percentage of the lapsed volunteers invited completed the survey?✓11.2%11.2%
If the lapsed respondents were representative, about how many of the 1,000 lapsed volunteers would give a scheduling reason (either of the first two Q3 options) as their main reason, using the unweighted share?✓429429

Traps in the data 8 of 8 handled

  • ✓Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
  • ✓The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.
  • ✓Q2 is multi-select: respondents chose 1.6 difficulties on average, so columns add up to more than 100%.
  • ✓52% of active volunteers responded but only 11% of lapsed ones, so the lapsed base is small (112) and those who replied may differ from the 89% who did not (for example, still feeling some attachment).
  • ✓Volunteers aged 65 or over are 43% of respondents but about 26% of volunteers in the database; younger volunteers cite travel cost more. Weighted to the lapsed age mix, travel cost among lapsed volunteers rises from 40% to 47%; a scheduling main reason goes from 43% to 41%, still well ahead of travel cost (21%).
  • ✓Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
  • ✓The travel-cost gap between lapsed and active respondents (3.0 points) is not significant at 95%.
  • ✓40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.

The analysis

Recommendation

Spend the majority of the £45,000 on a part-time volunteer coordinator plus a simple online shift-booking/rota system with automatic text/email notifications. This directly targets the biggest driver of lapsing: scheduling and communication. The operations lead should not read the travel figure as a mandate to spend everything on travel: travel is a real friction, especially for current volunteers and younger leavers, but it is not the main reason people lapse. The chair should treat the 40.2% stated likelihood of returning with online booking as an upper bound, rising to 60.4% among those who left for scheduling reasons, and use a pilot to measure actual re-enrolment. The volunteer manager should pilot a small, targeted travel/parking scheme for under-25s rather than assuming travel support applies equally to all ages. Keep a smaller pot for that pilot and expand it only if it proves cost-effective per retained volunteer.

What would change it

If a pilot of online booking shows low adoption or converts few lapsed scheduling-leavers into actual returning volunteers, or if the travel reimbursement pilot proves much cheaper per retained volunteer and demonstrably brings back under-25s, shift more of the £45,000 to travel support. If active volunteer attrition rises because of travel costs, a broader travel scheme would become more urgent.

Findings

  1. Scheduling and rota problems are the biggest identifiable driver of lapsing, not travel.

    High confidenceBase 112

    Among 112 lapsed respondents, 29.5% gave 'shifts changed at short notice' as their main reason for stopping and 13.4% gave 'hard to arrange shifts', a combined 42.9% (48/112). Only 11.6% gave travel cost and 0.9% travel time as main reasons. In Q2, lapsed volunteers also reported short-notice changes (51.8%), no easy way to arrange shifts (42.0%), and not being told about changes (41.1%) more often than travel cost (40.2%).

  2. Online booking is the option most likely to bring lapsed volunteers back, especially those who left for scheduling reasons.

    Medium confidenceBase 112

    40.2% of lapsed respondents said they would be likely/very likely to return if they could arrange shifts online, versus 26.8% if travel costs were paid back. Among lapsed respondents whose main reason was scheduling (n=48), 60.4% said they would be likely to return with online booking, versus 27.1% with travel reimbursement.

  3. Travel cost is the most commonly selected difficulty among active volunteers, but it does not by itself explain lapsing.

    Medium confidenceBase 215

    In Q2, 37.2% of active volunteers selected travel cost, the highest single difficulty for that group. Lapsed volunteers selected it at a similar level (40.2%), and the difference is not statistically significant at 95% (z≈0.5). Active volunteers were far less likely to report scheduling difficulties: 14.4% cited short-notice changes versus 51.8% of lapsed volunteers.

  4. Under-25 lapsed volunteers look like a distinct travel-cost segment, but the sample is very small.

    Low confidenceBase 11

    Among the 11 lapsed under-25 respondents, 81.8% cited travel cost in Q2 and 72.7% gave it as the main reason for stopping. In the full under-25 sample (n=27), 74.1% cited travel cost. After weighting lapsed travel responses to the age mix of all 1,000 lapsed volunteers, the estimated lapsed travel-cost share rises to 47.4%, because younger lapsed volunteers are underrepresented in the respondent sample.

  5. Open-text answers reinforce the quantitative pattern: volunteers want online rota/booking/swapping and timely communication about changes.

    Medium confidenceBase 109

    37 of 109 open-text answers mention shift scheduling, booking/swapping, or last-minute changes/cancellations. Repeated asks include an app or website for booking, a swap board, text/email alerts, and more notice before shifts change.

  6. The lapsed survey sample is not representative of the lapsed population by age, so lapsed estimates need caution.

    High confidenceBase 112

    Lapsed respondents are 9.8% under 25 versus 22% of the lapsed population, and 39.3% aged 65+ versus 25% of the lapsed population. The lapsed completion rate was low: 112 completes from 1,000 invited (11.2%), compared with 215 of 410 active volunteers (52.4%).

Caveats

  • The lapsed response rate was low: 112 completes from 1,000 invited (11.2%). Lapsed respondents are older than the lapsed population, so unweighted aggregates misstate younger volunteers' views; an age-weighted travel estimate is provided.
  • Under-25 lapsed cells are very small (n=11). The 72.7% travel-related main reason for this group has a wide margin of error and should be treated as directional, not precise.
  • Q4 and Q5 ask about hypothetical likelihood to volunteer again. Stated intentions typically overstate actual behaviour, so they should be treated as upper bounds, not forecasts.
  • Q2 allowed multiple selections, so percentages are not mutually exclusive; they do not sum to 100 and are best compared across rows rather than added.
  • Open-text coding is qualitative; counts depend on interpretation and are used to illustrate themes, not as precise measures.
  • The survey reached volunteers in the database. It did not capture people who stopped more than 12 months ago or never volunteered, so it cannot explain recruitment losses.

Next steps

  • decisionSet the budget: allocate most of the £45,000 to a coordinator plus online rota/booking system, and reserve a pilot pot for travel/parking support.
  • experimentPilot online shift booking/rota with text/email notifications and a named coordinator.
  • experimentPilot a targeted travel/parking reimbursement for under-25s and others who identify cost as a barrier.
  • monitorMonitor operational metrics before and after: shift cancellations/changes with less than 48 hours notice, number of active volunteers by age, and lapsed volunteers who return.
  • researchConduct short follow-up interviews with lapsed respondents, especially under-25s and those who said they would return, to understand barriers and refine the offer.

Open-text themes it coded

Travel cost / reimbursement 17Journey distance / time / closer site 9Parking / physical access 6Shift scheduling, booking and swapping / online rota 20Last-minute changes and timely communication 21Recognition / feeling valued 2Training / induction 1Car sharing / lift-sharing 2No change needed / happy as is 38

The survey it planned

1 screening question and 10 questions, as the model wrote them.

  1. S1

    When was the last time you did a volunteer shift at Riverside Food Network?

    One answer
    • Within the last 3 months
    • Between 3 and 12 months ago
    • More than 12 months ago

    Continues if Within the last 3 months; Between 3 and 12 months ago

  2. Q1

    At which Riverside distribution site do you usually volunteer? If you have volunteered at more than one site, please choose the site where you have done the most shifts in the last 12 months.

    One answer
    • Site 1
    • Site 2
    • Site 3
    • Site 4
    • Site 5
    • Site 6
    • Site 7
    • Site 8
    • Site 9
    • I'm not sure
  3. Q2

    What is your age group?

    One answer
    • Under 18
    • 18-24
    • 25-34
    • 35-44
    • 45-54
    • 55-64
    • 65-74
    • 75 or over
  4. Q3

    Which of the following difficulties, if any, have you experienced as a Riverside volunteer in the last 12 months? Tick all that apply. If you have not experienced any of these, tick 'None of these' only.

    Any that apply
    • Shift times don't suit me
    • Not enough shifts are available
    • Shifts are sometimes cancelled or changed at short notice
    • Booking or swapping shifts is difficult
    • Contacting my site lead is difficult
    • Travel to my site is too expensive
    • Travel to my site takes too long or is inconvenient
    • Parking at or near my site is difficult or expensive
    • My travel costs are not reimbursed
    • I don't feel supported or valued
    • The work is physically difficult
    • Health problems or a disability have limited me
    • Work, study, or family commitments have limited me
    • None of these – I have not experienced any of these difficulties
    Judges: double-barrelled

    Some options combine distinct difficulties into one response. In particular, 'Parking at or near my site is difficult or expensive' does not distinguish parking access from parking cost, which matters to the reimbursement decision.

    Several options combine distinct concepts, such as 'takes too long or is inconvenient', 'difficult or expensive', and 'Work, study, or family commitments'.

  5. Q4

    Approximately how much do you spend on travel to and from a Riverside shift? If you have stopped volunteering, please think about your last period of volunteering. Enter 0 if you do not usually pay anything. Amount in pounds (for example, 4.50).

    Number
  6. Q5

    You told us you have not volunteered in the last 3 months. Which of the following contributed to you stopping? Tick all that apply. If your main reason is not listed, tick 'Other' and tell us.

    Any that apply
    • Shift times no longer suited me
    • There were not enough shifts available
    • Shifts were cancelled or changed too often
    • Booking or swapping shifts was too difficult
    • I didn't feel supported or valued
    • Travel costs were too high
    • Travel took too long or was difficult
    • Parking was too difficult or expensive
    • I was not reimbursed for travel expenses
    • My health made it difficult
    • Work, study, or caring commitments
    • I moved away from the area
    • I found another volunteering opportunity
    • I didn't deliberately stop; I just haven't been back
    • Other

    Routing Shown only if S1 is: Between 3 and 12 months ago

    Judges: double-barrelled

    'Parking was too difficult or expensive' combines parking access and parking cost as reasons for stopping, so a selected answer cannot distinguish these decision-relevant causes.

    Several options combine distinct concepts (e.g., 'too long or was difficult', 'difficult or expensive').

  7. Q6

    You selected 'Other' as a reason for stopping. Please tell us more:

    Open text

    Routing Shown only if Q5 is: Other

  8. Q7

    If Riverside started reimbursing volunteers' travel costs, would that make you more likely to volunteer again?

    One answer
    • Yes, definitely
    • Yes, probably
    • No, probably not
    • No, definitely not
    • Not sure

    Routing Shown only if S1 is: Between 3 and 12 months ago

  9. Q8

    If Riverside introduced an online system for booking and swapping shifts, would that make you more likely to volunteer again?

    One answer
    • Yes, definitely
    • Yes, probably
    • No, probably not
    • No, definitely not
    • Not sure

    Routing Shown only if S1 is: Between 3 and 12 months ago

  10. Q9

    If Riverside could afford only ONE of the following changes, which would most encourage you to volunteer again?

    One answer
    • Reimbursing travel costs
    • Introducing online shift booking
    • Doing both, even at a smaller scale
    • Neither would make a difference
    • Not sure

    Routing Shown only if S1 is: Between 3 and 12 months ago

    Judges: ambiguous

    'Doing both, even at a smaller scale' leaves the actual offer undefined. Respondents could assume materially different reimbursement amounts, eligibility or booking provision, making preferences for this package difficult to interpret.

    The question says Riverside could afford only ONE change, yet one option is 'Doing both, even at a smaller scale'. This contradicts the premise, so respondents may read the choice in different ways.

  11. Q10

    Is there anything else you would like to tell us about volunteering at Riverside Food Network?

    Open text
Sample plan

Invite everyone on the volunteer database who did at least one shift in the last 12 months: 410 active volunteers and about 1,000 lapsed volunteers (total about 1,410). Send personalised email and text-message invitations with unique links, followed by two reminders. Target 330 completed surveys (about 23% response rate). Quotas: aim for at least 100 completes from active volunteers and 200 from lapsed volunteers; if lapsed response is low, send an extra reminder to lapsed volunteers. Note: the requested plus or minus 7 percentage point precision for each of the 9 sites would require about 196 completed surveys per site, or about 1,764 completed surveys in total, which is not achievable with 330 completes. Site-level results will therefore be reported with their actual confidence intervals and flagged where wider than plus or minus 7 points; if site-level precision is non-negotiable, the sample size and budget must be substantially increased.