Sign inTest your prompt

Benchmarks / SurveyBench / Why Riverside Food Network is losing volunteers / Gemini 4 Carbon

Measured by Spring Prompt

Gemini 4 Carbon: Why Riverside Food Network is losing volunteers

The decision: how to spend next year's £45,000 for volunteer support: reimburse volunteers' travel costs, fund a part-time volunteer coordinator with online shift booking, or do both at a smaller scale. The brief and the data →

Verdict
✗ Not sound
Research score
82 of 100
Analysis rating
1,076
Head to head, this task
won 2 of 9

Why it is not sound

The analysis

  • Reported a material finding the data does not support
  • Made a recommendation that does not follow from the data
  • Proposed a mistaken next step: Forecast how many lapsed volunteers will return from the likely-to-return answers

Findings the judges found unsupported

  • Online booking software alone will fail unless paired with a coordinator and site-lead standards to stop last-minute cancellations and uncommunicated changes.The evidence supports pairing booking software with better coordination and notice, but does not establish that software alone will fail. No intervention or comparison of implementation models was tested.
  • Answer to the Chair of Trustees: Online booking has stronger reactivation pull than travel reimbursement, but realistic return volume is roughly 35 to 100 lapsed volunteers—not the ~402 implied by a naive reading of Q4—and requires fixing short-notice changes alongside booking.The forecast of 35 to 100 returning volunteers is unsupported. Neither the assumed twofold-to-fourfold intention inflation nor non-respondents' actual engagement or return probability was measured. Acknowledging hypothetical-intention bias does not justify replacing the naive forecast with another numerical forecast.
  • Answer to the Volunteer Manager: Travel costs hit under-25s hardest, making travel support worthwhile, but the under-25 lapsed base is too small for high precision and older low-income volunteers also struggle—so reimbursement should be opt-in and receipted for any volunteer who needs it rather than age-restricted to under-25s.The survey does not establish that an opt-in, receipted reimbursement scheme open to everyone would naturally concentrate spending among younger and lower-income volunteers. Income, likely claims and scheme uptake were not measured.
  • Churn spans all nine distribution sites and is aggravated by cross-site inconsistency and unaddressed parking/car-share friction.Lapsed respondents appearing at every site and a higher multi-site share among lapsed respondents do not establish that cross-site inconsistency aggravates churn. The comments establish parking and car-sharing concerns, but not their effect on leaving or that a coordinator could resolve them cheaply.

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What percentage of active volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓37.2%37.2%
What percentage of lapsed volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓40.2%40.2%
Is the difference between active and lapsed respondents in the share choosing 'The cost of travelling to the site' in Q2 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Weighted to the age mix of all 1,000 lapsed volunteers, what percentage of lapsed volunteers would choose 'The cost of travelling to the site' as a difficulty in Q2?✓47.4%47.4%
What percentage of lapsed respondents gave a scheduling reason as their main reason for stopping in Q3 (either of its first two options: shifts changed at short notice, or hard to arrange shifts)?✓42.9%42.9%
What is the 95% margin of error, in percentage points, on that unweighted share of lapsed respondents giving a scheduling reason as their main reason?✓9.2 points9.17 points
On average, how many difficulties did each respondent choose in Q2 (counting every option except 'None of these', which counts as zero), across all respondents?✓1.551.55
What percentage of lapsed respondents said they would be likely (fairly or very) to volunteer again if they could arrange their shifts online (Q4)?✓40.2%40.2%
How many of the open-text answers mention shift scheduling (booking or swapping shifts, or last-minute changes or cancellations)?✓3034
What percentage of the active volunteers invited completed the survey?✓52.4%52.4%
What percentage of the lapsed volunteers invited completed the survey?✓11.2%11.2%
If the lapsed respondents were representative, about how many of the 1,000 lapsed volunteers would give a scheduling reason (either of the first two Q3 options) as their main reason, using the unweighted share?✓429429

Traps in the data 8 of 8 handled

  • ✓Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
  • ✓The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.
  • ✓Q2 is multi-select: respondents chose 1.6 difficulties on average, so columns add up to more than 100%.
  • ✓52% of active volunteers responded but only 11% of lapsed ones, so the lapsed base is small (112) and those who replied may differ from the 89% who did not (for example, still feeling some attachment).
  • ✓Volunteers aged 65 or over are 43% of respondents but about 26% of volunteers in the database; younger volunteers cite travel cost more. Weighted to the lapsed age mix, travel cost among lapsed volunteers rises from 40% to 47%; a scheduling main reason goes from 43% to 41%, still well ahead of travel cost (21%).
  • ✓Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
  • ✓The travel-cost gap between lapsed and active respondents (3.0 points) is not significant at 95%.
  • ✓40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.

The analysis

Recommendation

Choose the split option ('do both at a smaller scale'), weighting roughly 55-60% of the £45,000 (~£25,000) to a part-time volunteer coordinator, low-cost online booking/swap software with automated SMS/email alerts, and mandatory rota-stability standards, and the remaining 40-45% (~£20,000) to an opt-in, receipted travel and parking reimbursement fund capped per shift and open to all ages. Directly answering stakeholders: (1) To the Operations Lead—do not spend all £45,000 on travel, because active and lapsed volunteers cite travel costs equally while scheduling and uncommunicated short-notice changes are the primary avoidable cause of churn, and uncapped universal travel payouts would absorb the budget mostly on stayers; (2) To the Chair of Trustees—expect roughly 35 to 100 lapsed volunteers to return (not ~400), and only if online booking is paired with coordinator-enforced notice rules to stop last-minute cancellations; (3) To the Volunteer Manager—do support under-25s' travel, where attrition from fares is sharpest, but make the capped expense scheme opt-in for any volunteer rather than age-gated so pensioners and other low-income adults paying fares or council parking are not excluded.

What would change it

Two pieces of evidence would shift this split: (1) If software quotes and local labour market testing show a capable part-time coordinator plus SMS-enabled rota software cannot be delivered for under ~£35,000 (leaving too little for a meaningful travel pot), prioritise the coordinator, online booking, council parking permits, and volunteer lift-share matching first, as scheduling/notice drives >2x as much attrition as travel even after age-weighting; conversely, (2) if rapid follow-up checks with non-responding under-25 and 25-44 leavers plus shift-frequency financial modelling show travel costs drive far higher churn across the 888 non-respondents while site leads already can adopt a free/low-cost booking tool without dedicated coordinator hours, shift a larger share of the £45,000 into the capped travel reimbursement fund.

Findings

  1. Answer to the Operations Lead: Travel cost is an ongoing friction endured equally by stayers and leavers, whereas rota instability and scheduling friction are what actually drive volunteers out—so spending all £45,000 on travel reimbursement would misdirect the budget.

    High confidenceBase 327

    In Q2, active (37.2%, n=215) and lapsed (40.2%, n=112) respondents report travel cost difficulty at statistically indistinguishable rates (+3.0pp gap, z=0.52, p=0.60). By contrast, lapsed volunteers are 3.6x as likely to report shifts changed at short notice (51.8% vs 14.4% active), 3.6x as likely to report no easy way to book or swap shifts (42.0% vs 11.6%), and 5.6x as likely to report not being told about changes in good time (41.1% vs 7.4%). In Q3, 42.9% (±9.2pp) of lapsed respondents stopped mainly for scheduling reasons—rising to 55.4% when including poor notice of changes—versus just 11.6% for travel cost (and even after reweighting Q3 to the database age mix, scheduling at 41.5%—52.2% with change notice—remains at least double travel cost at 20.7%).

  2. Online booking software alone will fail unless paired with a coordinator and site-lead standards to stop last-minute cancellations and uncommunicated changes.

    High confidenceBase 112

    Among the 48 lapsed volunteers who left for scheduling reasons in Q3, more than twice as many quit over shifts changing at short notice (29.5% of all lapsed, n=33) as over difficulty booking or swapping shifts (13.4%, n=15), and a further 12.5% (n=14) quit mainly because changes were not communicated in time. Across 109 open-text answers, 20 request online self-service booking/swapping, 10 cite last-minute shift changes or cancellations, and 13 cite learning of changes late or second-hand (including turning up to locked doors).

  3. Answer to the Chair of Trustees: Online booking has stronger reactivation pull than travel reimbursement, but realistic return volume is roughly 35 to 100 lapsed volunteers—not the ~402 implied by a naive reading of Q4—and requires fixing short-notice changes alongside booking.

    Medium confidenceBase 112

    In Q4, 40.2% of lapsed respondents say they are fairly (23.2%) or very (17.0%) likely to return with online shift booking (rising to 60.4% among the n=48 who left due to scheduling), outperforming travel reimbursement in Q5 (26.8% likely overall; 9.8% very likely). However, only 11.2% of the 1,000 lapsed volunteers completed the survey: 40.2% of the 112 respondents is 45 individuals (19 'very likely'). Because the 888 non-respondents are likely much less engaged and stated likelihood routinely overstates actual return by 2x to 4x, decision-makers should plan for tens of reactivated volunteers rather than hundreds.

  4. Answer to the Volunteer Manager: Travel costs hit under-25s hardest, making travel support worthwhile, but the under-25 lapsed base is too small for high precision and older low-income volunteers also struggle—so reimbursement should be opt-in and receipted for any volunteer who needs it rather than age-restricted to under-25s.

    Low confidenceBase 27

    Among under-25s, 74.1% overall (20/27) and 81.8% of lapsed respondents (9/11) cite travel cost as a difficulty in Q2 (lifting the age-weighted lapsed rate from 40.2% to 47.4%), and 72.7% of lapsed under-25s (8/11) name travel cost as their main reason for stopping in Q3. However, n=11 represents only 5.0% of the 220 lapsed under-25s in the database (±26.3pp 95% MoE on Q3), and open-text responses (22 on travel/parking cost) show 25-64 and 65+ volunteers on pensions or low incomes also struggling with £3.80-£9 fares, fuel, and £4-£7 council parking fees. An opt-in capped expense claim naturally concentrates uptake among younger and lower-income volunteers without creating unfairness or age-discrimination risk.

  5. Churn spans all nine distribution sites and is aggravated by cross-site inconsistency and unaddressed parking/car-share friction.

    Medium confidenceBase 327

    Lapsed respondents are spread across all nine sites (7.1% to 13.4% per site in Q6, n=112), with multi-site volunteers more than twice as prevalent among lapsed as active respondents (10.7% vs 4.2%). Open-text comments highlight volunteers paying £4-£7 at council car parks (V034, V060, V095, V158), struggling to park near sites when carrying heavy donations (V118, V289), or wanting lift-share matching (V047, V138)—issues a coordinator can alleviate cheaply.

Caveats

  • Severe non-response and age skew among lapsed volunteers: only 11.2% of lapsed volunteers responded (112/1,000) versus 52.4% of active volunteers (215/410), and under-45s make up just 28.6% of the lapsed sample versus 52.0% of the lapsed database (with only n=11 lapsed under-25s out of 220, a 5.0% response rate). Unweighted totals over-represent older, active volunteers.
  • Total-column Distortion: In the unweighted Total column (n=327), active volunteers make up 65.7% of respondents despite being only 29.1% of the 1,410-person volunteer database, inflating 'None of these' and masking how acute scheduling issues are across the wider cohort.
  • Self-reported status vs database classification: Respondents were grouped as active or lapsed via S1 self-report rather than database flags, so some occasional volunteers with no shift in the last 3 months may have self-classified as active (or vice versa).
  • Stated-intent inflation on Q4 and Q5: Hypothetical 1-5 likelihood scales overstate real-world behaviour, especially because Q4 asks only about 'arranging shifts online' without testing whether return depends on ending short-notice cancellations.
  • Missing unit-cost and shift-frequency data: The survey did not measure shifts per month per volunteer, transport mode share, or average out-of-pocket cost per shift, which are required to model exact annual reimbursement liability before setting a per-shift cap.

Next steps

  • decisionCost and procure an off-the-shelf charity rota/booking tool (with recurring shifts, peer shift-swapping, and automatic SMS/email change alerts) alongside recruiting a ~0.5 FTE volunteer coordinator (~£25,000 combined budget).
  • experimentAudit volunteer shift frequency from the database and run a 3-month pilot of an opt-in, receipted travel and parking reimbursement scheme (e.g., capped at £5/shift) at 3 sites before rolling out the ~£20,000 travel pot network-wide.
  • decisionHave the coordinator negotiate council car-park permits/designated loading bays at affected sites (e.g., Northgate, Eastfield, Westbury, Hollins Park) and set up opt-in site lift-share boards.
  • researchConduct 15-20 brief telephone check-ins with non-responding lapsed under-25 and 25-44 volunteers from the database.
  • monitorRun a targeted reactivation campaign to the 1,000 lapsed volunteers once online booking, SMS alerts, and opt-in travel expenses are live, and track actual returners and quarterly retention by age.

Open-text themes it coded

Nothing needed / happy as it is 38Travel, fuel, and parking cost support (reimbursement, passes, or lift-sharing) 22Self-service online shift booking and swapping 20Timely, direct communication of shift or site changes 13Fewer last-minute shift changes and cancellations / rota stability 10Shorter journey time or a closer distribution site 9On-site parking bays and physical loading access 2Volunteer recognition and feeling valued 2Induction training before first shift 1

The survey it planned

1 screening question and 10 questions, as the model wrote them.

  1. S1

    Have you volunteered for at least one shift at a Riverside Food Network distribution site in the last 12 months?

    One answer
    • Yes, within the last 3 months
    • Yes, between 3 and 12 months ago, and I plan to do more shifts
    • Yes, between 3 and 12 months ago, and I have stopped or paused volunteering
    • No, my last shift was more than 12 months ago
    • No, I have never volunteered at a Riverside Food Network site

    Continues if Yes, within the last 3 months; Yes, between 3 and 12 months ago, and I plan to do more shifts; Yes, between 3 and 12 months ago, and I have stopped or paused volunteering

  2. Q1

    Which Riverside Food Network distribution site did you volunteer at most often in the last 12 months?

    One answer
    • Site 1 [Insert actual site name before launch]
    • Site 2 [Insert actual site name before launch]
    • Site 3 [Insert actual site name before launch]
    • Site 4 [Insert actual site name before launch]
    • Site 5 [Insert actual site name before launch]
    • Site 6 [Insert actual site name before launch]
    • Site 7 [Insert actual site name before launch]
    • Site 8 [Insert actual site name before launch]
    • Site 9 [Insert actual site name before launch]
  3. Q2

    Thinking about your experience volunteering with Riverside Food Network in the last 12 months, which of the following, if any, have been difficult or inconvenient for you? Please select all that apply.

    Any that apply
    • Booking or swapping shifts by phone or message with my site lead
    • Not knowing which shifts need cover ahead of time
    • Paying for bus fares, fuel, or parking to get to the site
    • The time or hassle of travelling to and from the site
    • Shift times or lengths not fitting around my work, study, or caring responsibilities
    • The physical demands of sorting donations and handing out parcels
    • Not enough training, guidance, or support during shifts
    • Shifts feeling disorganised or short-staffed
    • None of these have been difficult for me
  4. Q3

    Roughly how much does a typical return trip to your distribution site cost you in bus/train fares, parking, or fuel? Enter 0 if you walk, cycle, get a free lift, or use a free travel pass.

    One answer
    • £0 (no travel cost)
    • £0.01 to £2.99
    • £3.00 to £4.99
    • £5.00 to £7.99
    • £8.00 or more
    • Not sure
  5. Q4

    What were the main reasons you stopped or paused volunteering with Riverside Food Network? Please select up to 3.

    Any that apply
    • Changes in my work, study, health, or family commitments
    • I only intended to volunteer temporarily (e.g. seasonal, student break, between jobs)
    • I moved away from the area
    • The cost of bus fares, fuel, or parking to get to shifts
    • The time or difficulty of getting to the site
    • It was hard or slow to book, swap, or cancel shifts with my site lead
    • Shift times did not suit my schedule
    • Shifts felt disorganised, stressful, or under-supported
    • The physical effort required on shifts
    • Other reason

    Routing Shown only if S1 is: Yes, between 3 and 12 months ago, and I have stopped or paused volunteering

  6. Q5

    If there is anything else you would like to share about why you stopped or paused volunteering, please write it below. (Optional)

    Open text

    Routing Shown only if S1 is: Yes, between 3 and 12 months ago, and I have stopped or paused volunteering

  7. Q6

    If Riverside Food Network introduced an online system where you could see open shifts and book or swap them yourself (with phone/message still available if preferred), how likely would you be to volunteer again in the next 6 months?

    Scale

    Scale 1-5: Very unlikely to Very likely

    Routing Shown only if S1 is: Yes, between 3 and 12 months ago, and I have stopped or paused volunteering

  8. Q7

    If Riverside Food Network reimbursed your out-of-pocket travel costs (bus fares, standard mileage, or parking) for shifts, how likely would you be to volunteer again in the next 6 months?

    Scale

    Scale 1-5: Very unlikely to Very likely

    Routing Shown only if S1 is: Yes, between 3 and 12 months ago, and I have stopped or paused volunteering

  9. Q8

    Riverside Food Network has a fixed £45,000 budget next year to better support volunteers. Which ONE of these options would make the biggest positive difference to your ability to volunteer with us?

    One answer
    • Full reimbursement of travel and parking costs for all shifts (keeping phone/message shift booking as it is)
    • An online shift-booking system plus a part-time volunteer coordinator to help manage rotas and support volunteers (no travel reimbursement)
    • Capped travel reimbursement (e.g. up to £4 per shift) alongside a simpler online shift-booking calendar
    • None of these would make a difference to me
  10. Q9

    How likely are you to volunteer for at least one shift with Riverside Food Network in the next 3 months, assuming nothing changes?

    Scale

    Scale 1-5: Very unlikely to Very likely

  11. Q10

    Which age group are you in?

    One answer
    • 18 to 29
    • 30 to 44
    • 45 to 59
    • 60 to 69
    • 70 or over
    • Prefer not to say
Sample plan

Invite all ~1,410 people on the volunteer database with at least one shift in the last 12 months (410 active, ~1,000 inactive in the last 3 months) via both email and SMS, with two follow-up reminders spaced 4 days apart to boost response among lapsed volunteers. Target ~330 completes overall (~150 active, ~180 inactive/lapsed; no hard screen-out quotas, as this is a census of a finite database, though post-stratification weights by active/inactive status and site volume will be applied). CRITICAL STATISTICAL NOTE FOR TRUSTEES: Achieving ±7 percentage points at 95% confidence independently for EACH of the 9 sites is mathematically impossible under these constraints. Even ignoring finite-population limits, ±7% per site requires ~196 completes per site (1,764 total—more than the entire 1,410 volunteer population). With ~157 volunteers per site on average in the database, even a finite-population-corrected ±7% margin requires ~87 completes per site (~783 total, a 56% response rate). At 330 total completes (~37 per site), overall network results will achieve ±4.7% precision (with finite-population correction), while individual site estimates will have ±14% precision; grouping sites into 3 geographic/transit clusters (~110 completes each) will achieve ±8.5% precision.