Benchmarks / SurveyBench / Why Riverside Food Network is losing volunteers / Gemini 4 Argon

Measured by Spring Prompt

Gemini 4 Argon: Why Riverside Food Network is losing volunteers

The decision: how to spend next year's £45,000 for volunteer support: reimburse volunteers' travel costs, fund a part-time volunteer coordinator with online shift booking, or do both at a smaller scale. The brief and the data →

Verdict
✗ Not sound
Research score
89 of 100
Analysis rating
985
Head to head, this task
won 12 of 23

Why it is not sound

The analysis

  • Reported a material finding the data does not support
  • Proposed a mistaken next step: Forecast how many lapsed volunteers will return from the likely-to-return answers

Findings the judges found unsupported

  • Answer to the Chair of Trustees: While 40.2% of lapsed respondents claim they would likely return if online booking were introduced (compared to 26.8% for travel reimbursement), you should NOT expect 400 of the 1,000 lost volunteers to come back. Because only 11.2% of lapsed volunteers responded to the survey, and stated return intentions in surveys heavily overstate real-world behaviour, a realistic return projection is roughly 45 to 150 lapsed volunteers. The true primary value of online booking and a coordinator is preventing future churn among the 410 active volunteers and stabilising shift fulfilment.The projected 45–150 returning volunteers, alternative numerical bounds, and assumed conversion discounts are not supported by observed behaviour or evidence about non-respondents. Hypothetical intentions cannot establish a realistic return forecast.
  • Answer to the Volunteer Manager: Do not launch an age-restricted travel scheme for under-25s based on this survey alone. While 72.7% (8 of 11) of lapsed under-25 respondents blamed travel cost for quitting, a sample of 11 is far too small to carry a policy decision, and targeting by age is inequitable and legally risky when active and older volunteers in the open text report severe financial hardship from £4–£7 parking fees, two-bus fares on a pension, and fuel costs. Instead, pilot a targeted, hardship- or receipt-based travel/parking reimbursement scheme open to all ages.The age-weighted scheduling main-reason figure is approximately 41.4%, not 37.8%; including late communication gives approximately 52.1%, not 48.9%. Scheduling nevertheless remains ahead of travel cost.

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What percentage of active volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓37.2%37.2%
What percentage of lapsed volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓40.2%40.2%
Is the difference between active and lapsed respondents in the share choosing 'The cost of travelling to the site' in Q2 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Weighted to the age mix of all 1,000 lapsed volunteers, what percentage of lapsed volunteers would choose 'The cost of travelling to the site' as a difficulty in Q2?✓47.2%47.4%
What percentage of lapsed respondents gave a scheduling reason as their main reason for stopping in Q3 (either of its first two options: shifts changed at short notice, or hard to arrange shifts)?✓42.9%42.9%
What is the 95% margin of error, in percentage points, on that unweighted share of lapsed respondents giving a scheduling reason as their main reason?✓9.2 points9.17 points
On average, how many difficulties did each respondent choose in Q2 (counting every option except 'None of these', which counts as zero), across all respondents?✓1.651.55
What percentage of lapsed respondents said they would be likely (fairly or very) to volunteer again if they could arrange their shifts online (Q4)?✓40.2%40.2%
How many of the open-text answers mention shift scheduling (booking or swapping shifts, or last-minute changes or cancellations)?✓3534
What percentage of the active volunteers invited completed the survey?✓52.4%52.4%
What percentage of the lapsed volunteers invited completed the survey?✓11.2%11.2%
If the lapsed respondents were representative, about how many of the 1,000 lapsed volunteers would give a scheduling reason (either of the first two Q3 options) as their main reason, using the unweighted share?✓429429

Traps in the data 8 of 8 handled

  • ✓Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
  • ✓The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.
  • ✓Q2 is multi-select: respondents chose 1.6 difficulties on average, so columns add up to more than 100%.
  • ✓52% of active volunteers responded but only 11% of lapsed ones, so the lapsed base is small (112) and those who replied may differ from the 89% who did not (for example, still feeling some attachment).
  • ✓Volunteers aged 65 or over are 43% of respondents but about 26% of volunteers in the database; younger volunteers cite travel cost more. Weighted to the lapsed age mix, travel cost among lapsed volunteers rises from 40% to 47%; a scheduling main reason goes from 43% to 41%, still well ahead of travel cost (21%).
  • ✓Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
  • ✓The travel-cost gap between lapsed and active respondents (3.0 points) is not significant at 95%.
  • ✓40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.

The analysis

Recommendation

Choose the hybrid option ('do both at a smaller scale'), heavily weighted toward fixing scheduling and rota governance first: allocate roughly £28,000–£30,000 to hire the part-time volunteer coordinator and deploy a low-cost, off-the-shelf online shift booking/swapping tool with automated SMS/email alerts across all 9 sites, and ring-fence the remaining £15,000–£17,000 for a capped, opt-in travel and parking reimbursement fund (e.g., up to £5 per shift on production of a bus/tram/train/parking ticket, open to any volunteer regardless of age, plus free council parking permits at high-charge sites like Northgate, Eastfield, and Hollins Park).

What would change it

We would shift the majority of the £45,000 into travel reimbursement only if: (1) an operational audit showed that 9 site leads can eliminate last-minute shift changes and adopt a free/low-cost self-serve rota tool without needing a paid part-time coordinator, freeing up salary funds; or (2) a follow-up check with non-responding under-25 and 25–44 lapsed volunteers revealed that scheduling friction was an artefact of older respondents and that capped travel stipends demonstrably prevent drop-out at a lower cost per retained shift than coordinator support.

Findings

  1. Answer to the Operations Lead: Although travel cost is the top general complaint overall (38.2%), it is NOT what drives volunteers to quit. Active volunteers (37.2%) and lapsed volunteers (40.2%) experience travel cost difficulties at almost the exact same rate (a statistically insignificant gap of 3.0 percentage points, p=0.60). In contrast, scheduling and communication failures are massive drivers of attrition: lapsed volunteers are 3.6 times more likely than active ones to cite short-notice shift changes (51.8% vs 14.4%) or difficulty arranging shifts (42.0% vs 11.6%), and 5.6 times more likely to cite not being told about changes in good time (41.1% vs 7.4%). When asked for their single main reason for stopping, 42.9% of lapsed volunteers blame shift scheduling (55.4% if including late notice of changes), compared to just 11.6% blaming travel cost.

    High confidenceBase 327

    Q2 Total base=327 (Active=215, Lapsed=112): Travel cost 38.2% total (37.2% active vs 40.2% lapsed, z=-0.52, not significant). Short-notice changes: 14.4% active vs 51.8% lapsed. Hard to arrange shifts: 11.6% active vs 42.0% lapsed. Not told in good time: 7.4% active vs 41.1% lapsed. Q3 Lapsed main reason for stopping (base=112): 29.5% shifts changed at short notice + 13.4% hard to arrange shifts = 42.9% scheduling (48/112, ±9.2% MoE; or ±8.6% with finite population correction); plus 12.5% not told about changes in good time = 55.4% total rota/communication failures, vs 11.6% travel cost.

  2. Answer to the Chair of Trustees: While 40.2% of lapsed respondents claim they would likely return if online booking were introduced (compared to 26.8% for travel reimbursement), you should NOT expect 400 of the 1,000 lost volunteers to come back. Because only 11.2% of lapsed volunteers responded to the survey, and stated return intentions in surveys heavily overstate real-world behaviour, a realistic return projection is roughly 45 to 150 lapsed volunteers. The true primary value of online booking and a coordinator is preventing future churn among the 410 active volunteers and stabilising shift fulfilment.

    Medium confidenceBase 112

    Q4 & Q5 (base=112 lapsed): 40.2% (45/112) likely to return with online booking (17.0% very likely, 23.2% fairly likely); among the 48 who left due to scheduling, 60.4% say they are likely to return. For travel reimbursement (Q5), 26.8% (30/112) are likely to return (9.8% very likely). However, response rate among the 1,000 lapsed volunteers was 11.2% (112/1,000). If the 888 non-respondents are completely unengaged (0% return) and only 'very likely' respondents actually return, the lower bound is ~19-45 volunteers; if respondents are moderately representative and discounted for standard stated-intent drop-off, ~100-150 may return.

  3. Answer to the Volunteer Manager: Do not launch an age-restricted travel scheme for under-25s based on this survey alone. While 72.7% (8 of 11) of lapsed under-25 respondents blamed travel cost for quitting, a sample of 11 is far too small to carry a policy decision, and targeting by age is inequitable and legally risky when active and older volunteers in the open text report severe financial hardship from £4–£7 parking fees, two-bus fares on a pension, and fuel costs. Instead, pilot a targeted, hardship- or receipt-based travel/parking reimbursement scheme open to all ages.

    Medium confidenceBase 27

    Q3 Lapsed Under 25 base=11: 72.7% (8/11) chose travel cost as main reason for stopping (95% CI roughly 39% to 94%). Q2 Under 25 total base=27 (16 active, 11 lapsed): 74.1% chose travel cost (81.8% lapsed, 68.8% active). Even when weighting Q3 across all 1,000 lapsed volunteers to correct for youth under-representation (Under 25s are 22% of the lapsed population but 9.8% of the sample), scheduling remains the #1 cause of churn (37.8%, or 48.9% with notice of changes) well ahead of travel cost (20.8%). Open text (Q1) shows older and middle-aged volunteers struggling with bus fares on a pension (V234), £6-£7 parking charges (V034, V158), and £9 train fares (V189).

  4. Online booking software alone will fail unless paired with a coordinator who enforces notice periods and communication standards. In both the quantitative tables and open-text responses, volunteers are just as frustrated by last-minute cancellations, changes made without warning, and turning up to locked doors or moved sessions as they are by the mechanics of booking and swapping shifts.

    High confidenceBase 112

    Q2 Lapsed (base=112): 51.8% experienced shifts changed at short notice and 41.1% were not told about changes in good time, alongside 42.0% having no easy way to arrange shifts. Open-text Q1 (109 responses): 35 responses mention scheduling/rota issues, of which 17 specifically highlight last-minute cancellations, uncommunicated rota changes, and hearing about shift changes second-hand or at the door (e.g., V009, V019, V055, V058, V098, V248, V308).

Caveats

  • Severe age skew (non-response bias): Younger volunteers are heavily under-represented. Under-25s and 25–44s make up 44% of the active database and 52% of the lapsed database, but only 20.4% of active respondents and 28.6% of lapsed respondents. Because younger volunteers report higher travel cost issues, unweighted totals underestimate travel cost difficulties (weighting the lapsed sample by age raises Q2 travel cost difficulty from 40.2% to 47.2%, and Q3 travel cost as main reason for leaving from 11.6% to 20.8%—though scheduling still remains the #1 cause of leaving even after weighting).
  • Low response rate among lapsed volunteers (11.2%): 888 of the 1,000 lapsed volunteers did not complete the survey. Those who answered are likely more attached to Riverside Food Network or had stronger grievances than non-respondents, so direct projections to all 1,000 lapsed volunteers (such as Q4/Q5 return rates) must be heavily discounted.
  • Tiny subgroup sample sizes by age: Breakdowns of lapsed volunteers by age rest on very small bases—especially Under-25s (n=11) and 25–44s (n=21)—giving margins of error exceeding ±20–30 percentage points.
  • Stated intention vs. actual behaviour: Questions Q4 and Q5 measure hypothetical likelihood to return on a 1–5 scale, which routinely overstates actual re-enrolment by a factor of 2 to 4.
  • No cost per volunteer shift in the dataset: The survey does not capture how many shifts per month volunteers work or their exact travel spend per shift, which is needed to model how far a travel reimbursement budget would stretch across 410+ volunteers.

Next steps

  • decisionHire the part-time volunteer coordinator, roll out a simple mobile-friendly online shift booking and swap board across all 9 sites, and mandate a minimum 48-hour notice protocol (with automated SMS/email alerts) for any rota changes.
  • experimentNegotiate free volunteer parking permits/validation with the local council or landlords at Northgate, Eastfield, and Hollins Park, and launch a 6-month pilot travel fund (£15,000–£17,000 ring-fenced) reimbursing bus/tram/train fares and parking up to £5 per shift upon receipt claim for any volunteer who needs it.
  • experimentRun a targeted reactivation campaign emailing and texting the 1,000 lapsed volunteers once online booking and the travel stipend pilot are live, with a one-click link to claim a recurring or single shift.
  • researchConduct 10–12 brief phone interviews (5 minutes each) with non-responding lapsed volunteers aged under 45 from the database.
  • monitorAdd a mandatory 1-question exit pulse whenever a volunteer becomes inactive for 60 days, and track quarterly retention by age band and site in the volunteer database.

Open-text themes it coded

Shift scheduling, online booking/swapping, and notice of rota changes 35Rota stability without software (fixed schedule requests) 1Travel and parking costs (bus/train/tram fares, fuel, mileage, and parking charges) 22Journey time, distance to site, and physical parking access 11Training, volunteer recognition, and feeling valued 3Satisfied / nothing needed 38

The survey it planned

2 screening questions and 10 questions, as the model wrote them.

  1. S1

    When did you last do a volunteer shift with Riverside Food Network?

    One answer
    • Within the last 3 months
    • Between 3 and 12 months ago
    • More than 12 months ago
    • I have never volunteered with Riverside Food Network

    Continues if Within the last 3 months; Between 3 and 12 months ago

  2. S2

    Which of the following best describes your current volunteering with Riverside Food Network?

    One answer
    • I still volunteer with Riverside Food Network (or plan to do a shift in the next month)
    • I am taking a temporary break, bust plan to return within the next 3 months
    • I have stopped volunteering with Riverside Food Network (or have no plans to return soon)

    Continues if I still volunteer with Riverside Food Network (or plan to do a shift in the next month); I am taking a temporary break, bust plan to return within the next 3 months; I have stopped volunteering with Riverside Food Network (or have no plans to return soon)

  3. Q1

    Which Riverside Food Network distribution site did you volunteer at most often in the last 12 months?

    One answer
    • Site 1 (Central)
    • Site 2 (North)
    • Site 3 (South)
    • Site 4 (East)
    • Site 5 (West)
    • Site 6 (Riverside)
    • Site 7 (Hilltop)
    • Site 8 (Parkside)
    • Site 9 (Valley)
    • Not sure / Prefer not to say
  4. Q2

    Thinking about your experience with Riverside Food Network over the last 12 months, which of the following difficulties, if any, have you experienced? (Select all that apply. Options are shown in random order.)

    Any that apply
    • The cost of bus fares, fuel, or parking to get to the site
    • Wait times or difficulty contacting my site lead to book or swap shifts
    • Not being able to see available shifts without phoning or messaging
    • Shifts being understaffed or too busy
    • Lack of training, guidance, or support during shifts
    • The physical demands of sorting donations and handing out parcels
    • Shift times not fitting around my work, study, or family commitments
    • Difficulty getting to the site by public transport or finding parking
    • Other difficulties
    • None of these — I have not experienced any difficulties
  5. Q3

    How much of a problem, if at all, was the cost of travelling to and from the site (such as bus fares, fuel, or parking) for you personally?

    Scale

    Scale 1-5: Not a problem at all to A very big problem

  6. Q4

    How much of a problem, if at all, was booking or swapping shifts by phoning or messaging your site lead?

    Scale

    Scale 1-5: Not a problem at all to A very big problem

    Judges: assumptive

    Assumes respondents have booked or swapped shifts by phoning or messaging their site lead. Neither eligibility nor display logic establishes this, and there is no not-applicable response.

    Assumes all respondents have had to book or swap shifts by phoning or messaging a site lead.

  7. Q5

    On a typical shift, about how much did you spend out of your own pocket on travel and parking (round trip)?

    One answer
    • £0 (for example, I walked, cycled, or had a free travel pass)
    • £1 to £2.99
    • £3 to £4.99
    • £5 to £7.99
    • £8 or more
    • Not sure
  8. Q6

    What were the main reasons you stopped volunteering (or are taking a break) with Riverside Food Network? (Select up to 3 reasons. Options are shown in random order.)

    Any that apply
    • Changes in my personal life, work, study, or health
    • I only ever intended to volunteer for a short time
    • The cost of travel or parking to get to the site
    • It was too much hassle to book, change, or swap shifts
    • My site lead was hard to reach or slow to reply
    • Shifts were too busy, stressful, or understaffed
    • The work was too physically demanding
    • I did not feel supported, valued, or well-organized on site
    • The journey to the site took too long or was inconvenient
    • Other reason

    Routing Shown only if S2 is: I am taking a temporary break, bust plan to return within the next 3 months; I have stopped volunteering with Riverside Food Network (or have no plans to return soon)

  9. Q7

    In your own words, what is the single main thing that caused you to stop volunteering, or that makes volunteering with Riverside Food Network difficult?

    Open text
    Judges: assumptive

    Shown to everyone, it assumes respondents either stopped volunteering or experience a difficulty. Active volunteers who experience no difficulties cannot answer its premise.

    Q7 is shown to everyone, including active volunteers. It asks for 'the single main thing' that makes volunteering difficult, which assumes every respondent has a difficulty. There is no option for those who have none, although Q2 allows 'None of these'.

  10. Q8

    Riverside Food Network has a fixed budget to improve volunteer support next year. Which ONE of the following options would make the biggest positive difference to your ability or willingness to volunteer with us?

    One answer
    • Option A: Full reimbursement of bus fares, mileage, and parking costs for every shift (shift booking stays via phone/message with site leads)
    • Option B: A new online system to view and book/swap shifts instantly + a dedicated coordinator to support volunteers (no travel reimbursement)
    • Option C: A combination: a simple online shift-booking calendar + reimbursing travel costs up to £4 per shift
    • None of these would make a difference to me
  11. Q9

    If Riverside Food Network introduced the option you chose in the previous question, how likely would you be to volunteer with us for at least one shift in the next 3 months?

    Scale

    Scale 1-5: Definitely would not volunteer to Definitely would volunteer

    Routing Shown only if Q8 is: Option A: Full reimbursement of bus fares, mileage, and parking costs for every shift (shift booking stays via phone/message with site leads); Option B: A new online system to view and book/swap shifts instantly + a dedicated coordinator to support volunteers (no travel reimbursement); Option C: A combination: a simple online shift-booking calendar + reimbursing travel costs up to £4 per shift

  12. Q10

    Which age group are you in?

    One answer
    • 18 to 29
    • 30 to 44
    • 45 to 59
    • 60 to 69
    • 70 or over
    • Prefer not to say
Sample plan

INVITATIONS & QUOTAS: Invite the entire eligible database of ~1,410 volunteers (410 active in the last 3 months; ~1,000 lapsed with a shift 3–12 months ago) via both email and SMS reminders to prevent non-response bias among lapsed volunteers. Target ~330 total completes (~150 active, ~180 lapsed) using soft monitoring by site and status, then weight the final data to the known database proportions by status (29% active / 71% lapsed) and by site. CRITICAL PRECISION CAVEAT FOR TRUSTEES: With 330 total completes across 9 sites (an average of ~37 respondents per site out of ~156 population per site), site-level results CANNOT achieve ±7% margin of error at 95% confidence. Even with finite population correction (FPC), achieving ±7% at 95% confidence for each of the 9 sites requires ~87 completes per site (~783 total completes, a 55% census response rate). At ~37 completes per site, site-level margin of error is approximately ±14 percentage points (with FPC). However, the overall sample (n=330, ±4.7% with FPC), comparisons between active and lapsed volunteers (n≈150 vs n≈180, ±6.5–7.0%), and grouping the 9 sites into 3 tiers (e.g., high/medium/low turnover or area clusters, n≈110 per tier, ±8%) WILL meet or closely approach the trustees' precision expectations.