Benchmarks / SurveyBench / Why Riverside Food Network is losing volunteers / Gemini 4 Hydrogen

Measured by Spring Prompt

Gemini 4 Hydrogen: Why Riverside Food Network is losing volunteers

The decision: how to spend next year's £45,000 for volunteer support: reimburse volunteers' travel costs, fund a part-time volunteer coordinator with online shift booking, or do both at a smaller scale. The brief and the data →

Verdict
✗ Not sound
Research score
96 of 100
Analysis rating
1,088
Head to head, this task
won 6 of 9

Why it is not sound

The survey

  • A double-barrelled question: Q7

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What percentage of active volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓37.2%37.2%
What percentage of lapsed volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓40.2%40.2%
Is the difference between active and lapsed respondents in the share choosing 'The cost of travelling to the site' in Q2 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Weighted to the age mix of all 1,000 lapsed volunteers, what percentage of lapsed volunteers would choose 'The cost of travelling to the site' as a difficulty in Q2?✓47.4%47.4%
What percentage of lapsed respondents gave a scheduling reason as their main reason for stopping in Q3 (either of its first two options: shifts changed at short notice, or hard to arrange shifts)?✓42.9%42.9%
What is the 95% margin of error, in percentage points, on that unweighted share of lapsed respondents giving a scheduling reason as their main reason?✓9.2 points9.17 points
On average, how many difficulties did each respondent choose in Q2 (counting every option except 'None of these', which counts as zero), across all respondents?✓1.551.55
What percentage of lapsed respondents said they would be likely (fairly or very) to volunteer again if they could arrange their shifts online (Q4)?✓40.2%40.2%
How many of the open-text answers mention shift scheduling (booking or swapping shifts, or last-minute changes or cancellations)?✓3634
What percentage of the active volunteers invited completed the survey?✓52.4%52.4%
What percentage of the lapsed volunteers invited completed the survey?✓11.2%11.2%
If the lapsed respondents were representative, about how many of the 1,000 lapsed volunteers would give a scheduling reason (either of the first two Q3 options) as their main reason, using the unweighted share?✓429429

Traps in the data 8 of 8 handled

  • ✓Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
  • ✓The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.
  • ✓Q2 is multi-select: respondents chose 1.6 difficulties on average, so columns add up to more than 100%.
  • ✓52% of active volunteers responded but only 11% of lapsed ones, so the lapsed base is small (112) and those who replied may differ from the 89% who did not (for example, still feeling some attachment).
  • ✓Volunteers aged 65 or over are 43% of respondents but about 26% of volunteers in the database; younger volunteers cite travel cost more. Weighted to the lapsed age mix, travel cost among lapsed volunteers rises from 40% to 47%; a scheduling main reason goes from 43% to 41%, still well ahead of travel cost (21%).
  • ✓Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
  • ✓The travel-cost gap between lapsed and active respondents (3.0 points) is not significant at 95%.
  • ✓40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.

The analysis

Recommendation

Choose the blended 'both at a smaller scale' option, anchored primarily on funding a part-time volunteer coordinator plus a low-cost online shift booking and automated SMS/email alert system, while reserving a ring-fenced, capped travel-support pilot (rather than an open-ended or under-25-only reimbursement scheme). Spending all £45,000 on travel reimbursement would miss the primary driver of churn (scheduling and short-notice communication failures cited by 55.4% of lapsed respondents as their main exit reason) and likely exceed £45,000/year if widely claimed. Conversely, buying booking software without coordinator hours will not fix last-minute shift cancellations (29.5% of exits) or late change alerts (12.5%), and ignoring travel completely leaves a real barrier for under-25s (74%–82% difficulty) and low-income commuters.

What would change it

Evidence that (1) a full-year part-time coordinator plus software consumes almost the entire £45,000 (leaving <£5,000 for travel) would shift the decision to coordinator + booking only, supplemented by zero-cost travel fixes (lift-share board, site-matching closer to home, negotiated council parking permits); OR (2) follow-up outreach with non-responding lapsed under-25s/25–44s and a costed travel-claim audit showing travel reimbursement for high-need volunteers costs <£15,000/year and doubles shift retention over a 3-month trial would justify shifting more of the £45,000 toward travel expenses.

Findings

  1. Operations lead question: Although travel cost is the single most-checked difficulty overall (38.2%), it does not explain volunteer churn; scheduling and communication failures separate lapsed volunteers from active ones by 30 to 37 percentage points.

    High confidenceBase 327

    In Q2 (n=327), travel cost is virtually identical between active (37.2%, n=215) and lapsed (40.2%, n=112) volunteers—a statistically non-significant 3.0 pp difference (z=0.52). By contrast, lapsed volunteers are vastly more likely than active volunteers to cite short-notice shift changes (51.8% vs 14.4%), difficulty arranging shifts (42.0% vs 11.6%), and not being told about changes in good time (41.1% vs 7.4%).

  2. Scheduling problems are the leading main reason volunteers stopped (42.9%, rising to 55.4% when including late notice of changes), nearly four times the share who left mainly over travel costs (11.6%).

    Medium confidenceBase 112

    In Q3 (n=112 lapsed volunteers), 29.5% left mainly due to short-notice shift changes and 13.4% due to difficulty arranging/swapping shifts (42.9% combined, 95% MoE ±9.2 pp; age-weighted 41.5%), plus 12.5% due to not being told about changes in good time. Only 11.6% named travel cost (12.5% including journey time) as their main exit reason, and 23.2% cited changed personal circumstances.

  3. Chair of trustees question: Online booking outperforms travel reimbursement on stated return intent (40.2% vs 26.8% likely), but 40.2% (~400 of 1,000 lapsed volunteers) is an optimistic upper ceiling, not a forecast, and booking software alone will not stop last-minute cancellations.

    Low confidenceBase 112

    In Q4 (n=112), 40.2% say they are likely to return with online booking (17.0% very likely, 23.2% fairly likely), rising to 60.4% among the 48 scheduling-leavers, versus 26.8% overall (9.8% very likely) for travel reimbursement in Q5. However, 88.8% of invited lapsed volunteers did not respond, stated intent typically overstates actual return rates, and 29.5% in Q3 left over short-notice changes—an operational discipline issue requiring coordinator oversight, not just software.

  4. Volunteer manager question: Under-25 volunteers report the sharpest travel-cost barrier (74.1% active; 81.8% lapsed), and 72.7% of lapsed under-25s blame travel cost for leaving, but the sample is far too small and skewed (n=11, 5% response rate) to justify an exclusive under-25 travel scheme on its own.

    Low confidenceBase 11

    In Q2/Q3, 9 of 11 lapsed under-25s cite travel cost as a difficulty and 8 of 11 (72.7%, 95% MoE ±26.3 pp) name it their main reason for stopping (vs 2.3% of 65+ lapsed). Active under-25s also report 74.1% travel-cost difficulty (20 of 27). Age-weighting lapsed Q2 raises travel-cost difficulty from 40.2% to 47.4% (and Q3 travel main reason to ~21%), but only 11 of ~220 lapsed under-25s responded.

  5. Open-text feedback shows that self-service booking, schedule stability, and proactive change alerts must go together, while travel pain is concentrated in specific fare and parking pinch points (£4–£7 parking, £3.80–£9 transit, £12 fuel).

    High confidenceBase 109

    Of 109 Q1 answers, 38 say nothing needs changing, 36 raise shift booking/swapping or last-minute changes/cancellations, 18 highlight poor communication (turning up to locked doors or moved sessions, lost WhatsApp messages), 22 cite specific travel costs (parking near Northgate/Eastfield/Hollins Park, bus/tram/train fares, petrol), and 9 cite journey length/site proximity.

Caveats

  • Severe non-response differential between active (52.4%) and lapsed (11.2%) volunteers: the 1,000 lapsed volunteers are 71% of the database pool but only 34% of survey completes, so unweighted 'Total' figures heavily reflect active volunteers.
  • Strong age skew in responses: 65+ volunteers are 30% of active and 25% of lapsed populations, yet make up 45.1% of active and 39.3% of lapsed completes. Under-25s (18% active / 22% lapsed in population) are only 7.4% / 9.8% of completes, causing unweighted lapsed totals to understate travel-cost difficulty (40.2% unweighted vs 47.4% age-weighted).
  • Lapsed under-25 findings rest on just 11 respondents (~5% response rate among ~220 invited), giving a 95% margin of error of ±26 percentage points and high risk of non-response bias.
  • Hypothetical return-intent questions (Q4 and Q5) suffer from intention-behaviour gap and selection bias among the 11.2% who engaged with the survey; actual reactivation will be substantially lower than 40.2%.
  • The survey did not collect actual out-of-pocket travel spend, reimbursement threshold needed, or site-level scheduling cancellation logs, so budget sizing for travel reimbursement cannot be calculated from these responses alone.
  • Eastfield is slightly underrepresented in completes (6.7% vs ~11.1% even share across 9 sites), and 6.4% of respondents split shifts across multiple sites.

Next steps

  • decisionCommit the core of the £45,000 budget to a part-time volunteer coordinator and a shared online rota tool with self-service booking, shift-swap board, and automated SMS/email change alerts, alongside a capped travel-expense pilot pot.
  • experimentRun a 3-month opt-in travel-expense pilot at 2–3 high-cost sites (or targeted via a simple financial-need/distance request, capped per shift, plus parking validation at Northgate/Eastfield/Hollins Park and a lift-share board) before scaling city-wide or by age.
  • researchConduct brief phone/SMS follow-up interviews (15–20 respondents) with non-responding lapsed volunteers—especially under-25s and 25–44s—and audit 6 months of site rota logs to measure true cancellation rates and typical travel spend per shift.
  • monitorTrack actual reactivation of the 1,000 lapsed volunteers (especially the scheduling-leaver segment) and active churn by age band monthly for 6 months after launching online booking.

Open-text themes it coded

No changes needed / satisfied 38Online self-service shift booking and swapping 20Short-notice shift changes, cancellations, and schedule stability 18Poor communication and lack of timely updates 18Travel costs (public transport fares, fuel, and parking charges) 22Journey time, distance, and site proximity 9Parking availability and physical site access 2Volunteer recognition and feeling valued 2Onboarding training 1

The survey it planned

1 screening question and 10 questions, as the model wrote them.

  1. S1

    When did you last complete a volunteer shift with Riverside Food Network?

    One answer
    • Within the last 3 months
    • Between 3 and 12 months ago
    • More than 12 months ago
    • I have never volunteered with Riverside Food Network

    Continues if Within the last 3 months; Between 3 and 12 months ago

  2. Q1

    Which Riverside Food Network distribution site have you volunteered at most often in the last 12 months?

    One answer
    • Site 1
    • Site 2
    • Site 3
    • Site 4
    • Site 5
    • Site 6
    • Site 7
    • Site 8
    • Site 9
  3. Q2

    Which, if any, of the following difficulties have you experienced while volunteering with Riverside Food Network in the last 12 months? Please select all that apply.

    Any that apply
    • Paying for bus fares, fuel, or parking to get to the site
    • The time or convenience of travelling to the site
    • Booking or swapping shifts by phoning or messaging the site lead
    • Not knowing in advance which shifts need cover
    • Shifts clashing with paid work, study, or caring responsibilities
    • Short-notice cancellations or changes to site opening hours
    • Feeling unclear about tasks or unsupported during a shift
    • Physical demands of sorting donations or handing out food parcels
    • None of these - I have not experienced difficulties
  4. Q3

    Thinking about your experience over the last 12 months, how much of a barrier was or is the cost of travelling to your distribution site (such as bus fares, fuel, or parking)?

    Scale

    Scale 1-5: Not a barrier at all to A major barrier

  5. Q4

    Thinking about your experience over the last 12 months, how easy or difficult did you find booking or swapping shifts by phoning or messaging your site lead?

    Scale

    Scale 1-5: Very easy to Very difficult

    Judges: assumptive

    Booking and swapping shifts may have different levels of difficulty, but receive one rating. The question also assumes respondents have used phoning or messaging to arrange shifts; there is no option for those who have not.

    Assumes every respondent has booked or swapped shifts by phoning or messaging a site lead. Volunteers on fixed regular shifts may never have done so, and there is no 'not applicable' option, so they are forced onto the ease scale.

  6. Q5

    You mentioned you have not completed a shift in the last 3 months. Which of the following reasons explain why you stopped volunteering? Please select all that apply.

    Any that apply
    • Changes in my paid work, study, or caring commitments
    • Health or personal reasons
    • I moved away from the area
    • The cost of travelling to the site (bus fares, fuel, or parking)
    • Difficulty getting to the site (such as transport links or travel time)
    • Difficulty booking, confirming, or swapping shifts
    • Poor communication or lack of volunteer coordination
    • Shifts being cancelled or sites closing due to low volunteer numbers
    • I did not enjoy the tasks or atmosphere at the site
    • I only intended to volunteer temporarily or for a short period
    • Another reason

    Routing Shown only if S1 is: Between 3 and 12 months ago

    Judges: assumptive

    Having no shift in the last three months does not establish that someone has stopped volunteering. Infrequent volunteers or those taking a temporary break must nevertheless select a reason for stopping.

    Treats everyone with no shift in the last 3 months as having 'stopped volunteering'. Some of them are only on a temporary gap and still see themselves as volunteers, yet they must give reasons for stopping.

  7. Q6

    How likely or unlikely would you be to start volunteering with Riverside Food Network again if your travel costs (such as bus fares, fuel, or parking) were reimbursed?

    Scale

    Scale 1-5: Very unlikely to Very likely

    Routing Shown only if S1 is: Between 3 and 12 months ago

  8. Q7

    How likely or unlikely would you be to start volunteering with Riverside Food Network again if Riverside Food Network introduced an online shift booking system and a part-time volunteer coordinator?

    Scale

    Scale 1-5: Very unlikely to Very likely

    Routing Shown only if S1 is: Between 3 and 12 months ago

    Judges: double-barrelled

    The routing does not establish that respondents have stopped volunteering, although the question assumes they would need to start again. The coordinator and booking system constitute the combined intervention specified in the brief.

    Bundles two separate interventions, online shift booking and a part-time coordinator, into one likelihood rating. The effect of online booking on its own (O3) cannot be separated out.

  9. Q8

    Next year, Riverside Food Network has a fixed budget for volunteer support. Which of the following options would make the biggest positive difference to your ability to volunteer regularly (or return to volunteering)?

    One answer
    • Reimbursing reasonable travel costs (bus fares, fuel, or parking)
    • Funding a part-time volunteer coordinator with online shift booking
    • Doing both at a smaller scale (for example, partial travel reimbursement and simpler online shift booking)
    • None of these would affect my volunteering
    • Not sure
  10. Q9

    In your own words, what is the single most important change Riverside Food Network could make to help you keep volunteering or return to volunteering?

    Open text
  11. Q10

    Which age group do you belong to?

    One answer
    • Under 25
    • 25 to 34
    • 35 to 44
    • 45 to 54
    • 55 to 64
    • 65 or over
    • Prefer not to say
Sample plan

Census invitation via email and SMS to all ~1,410 volunteers on the database with at least one shift in the last 12 months (~410 active in the last 3 months and ~1,000 lapsed with no shift in the last 3 months), with up to two targeted reminders (prioritising lapsed volunteers and lower-responding sites). Target ~330 completes overall, with soft quotas aiming for ~150-165 active and ~165-180 lapsed completes and roughly ~36-37 completes per site (adjusted proportionally if site rosters differ markedly); apply post-stratification weights by active/lapsed status and site if response skews toward active volunteers. CRITICAL PRECISION NOTE FOR TRUSTEES: Achieving +/-7 percentage points at 95% confidence for each of the 9 sites requires n=196 per site without finite population correction (FPC), or ~88 completes per site (~792 total completes across ~1,410 volunteers, averaging ~157 per site) with FPC. At the planned ~330 completes (~37 per site on average), site-level margins of error will be roughly +/-14 percentage points after FPC (+/-16 pp without FPC), while charity-wide estimates (n=330) will be +/-4.7 pp and active vs. lapsed subgroups (~165 each) will be +/-6.7 pp. Trustees should either treat site-level results as directional or expand fieldwork/incentives to reach ~790 completes if +/-7 pp per site is strictly required.