Benchmarks / SurveyBench / Why Riverside Food Network is losing volunteers / Gemini 4 Neon
Gemini 4 Neon: Why Riverside Food Network is losing volunteers
The decision: how to spend next year's £45,000 for volunteer support: reimburse volunteers' travel costs, fund a part-time volunteer coordinator with online shift booking, or do both at a smaller scale. The brief and the data →
- Verdict
- ✗ Not sound
- Research score
- 68 of 100
- Analysis rating
- 629
- Head to head, this task
- won 9 of 11
Why it is not sound
The analysis
- Reported a material finding the data does not support
- Made a recommendation that does not follow from the data
Findings the judges found unsupported
Shift scheduling and rota communication are the primary controllable drivers of volunteer churn—and software alone will not fix them without operational discipline led by a coordinator.
Multi-site respondents are more prevalent among lapsed respondents, but the data does not establish that they suffer most from the separate spreadsheets or that this explains their leaving.Chair of trustees' question: We cannot reliably project how many lost volunteers will return with online booking, and applying the 40.2% stated-intent figure to 1,000 lapsed volunteers (~402 returners) would severely overestimate reality. Actual reactivation is likely to be in the tens rather than hundreds, making retention of active and new volunteers the stronger case for investment.
The prediction of tens rather than hundreds of returners has no empirical basis. Neither the stated intentions nor the non-response figures provide a defensible conversion rate. Respondents being the warmest segment is a possibility, not an established fact, and the 28% intention among those with other main exit reasons does not itself demonstrate intention inflation.Severe age skew in survey responses understates the true prevalence of travel costs among lapsed volunteers, strengthening the case for a hybrid approach rather than ignoring travel entirely.
Different response rates within main-reason groups do not establish non-overlapping audiences for booking and reimbursement. The tables do not show which individuals express interest in both interventions.Low-cost operational fixes surfaced in open text—especially site-specific parking validation, lift-sharing, cross-site shift visibility, and basic onboarding—can reduce travel and friction barriers without draining cash reimbursement funds.
The suggestions support investigating these changes, but their costs and feasibility were not measured. Describing parking validation, a shared rota and other changes as having minimal ongoing cost could mislead budget allocation.
Scorecard
Numbers 12 of 12 right
| Question | Its answer | True | |
|---|---|---|---|
| What percentage of active volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2? | ✓ | 37.2% | 37.2% |
| What percentage of lapsed volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2? | ✓ | 40.2% | 40.2% |
| Is the difference between active and lapsed respondents in the share choosing 'The cost of travelling to the site' in Q2 statistically significant at the 95% level? (1 for yes, 0 for no) | ✓ | No | No |
| Weighted to the age mix of all 1,000 lapsed volunteers, what percentage of lapsed volunteers would choose 'The cost of travelling to the site' as a difficulty in Q2? | ✓ | 47.4% | 47.4% |
| What percentage of lapsed respondents gave a scheduling reason as their main reason for stopping in Q3 (either of its first two options: shifts changed at short notice, or hard to arrange shifts)? | ✓ | 42.9% | 42.9% |
| What is the 95% margin of error, in percentage points, on that unweighted share of lapsed respondents giving a scheduling reason as their main reason? | ✓ | 9.2 points | 9.17 points |
| On average, how many difficulties did each respondent choose in Q2 (counting every option except 'None of these', which counts as zero), across all respondents? | ✓ | 1.55 | 1.55 |
| What percentage of lapsed respondents said they would be likely (fairly or very) to volunteer again if they could arrange their shifts online (Q4)? | ✓ | 40.2% | 40.2% |
| How many of the open-text answers mention shift scheduling (booking or swapping shifts, or last-minute changes or cancellations)? | ✓ | 29 | 34 |
| What percentage of the active volunteers invited completed the survey? | ✓ | 52.4% | 52.4% |
| What percentage of the lapsed volunteers invited completed the survey? | ✓ | 11.2% | 11.2% |
| If the lapsed respondents were representative, about how many of the 1,000 lapsed volunteers would give a scheduling reason (either of the first two Q3 options) as their main reason, using the unweighted share? | ✓ | 429 | 429 |
Traps in the data 8 of 8 handled
- ✓Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
- ✓The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.
- ✓Q2 is multi-select: respondents chose 1.6 difficulties on average, so columns add up to more than 100%.
- ✓52% of active volunteers responded but only 11% of lapsed ones, so the lapsed base is small (112) and those who replied may differ from the 89% who did not (for example, still feeling some attachment).
- ✓Volunteers aged 65 or over are 43% of respondents but about 26% of volunteers in the database; younger volunteers cite travel cost more. Weighted to the lapsed age mix, travel cost among lapsed volunteers rises from 40% to 47%; a scheduling main reason goes from 43% to 41%, still well ahead of travel cost (21%).
- ✓Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
- ✓The travel-cost gap between lapsed and active respondents (3.0 points) is not significant at 95%.
- ✓40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.
The analysis
Findings
-
Operations lead's question: Travel cost being the most-cited difficulty overall does NOT justify spending £45,000 on travel reimbursement. Travel cost is a background friction shared equally by active and lapsed volunteers, whereas scheduling and communication breakdowns are what separate leavers from stayers.
In Q2, travel cost is chosen by 37.2% of active volunteers (n=215) and 40.2% of lapsed volunteers (n=112)—a 3.0 percentage-point gap that is not statistically significant (z=0.52, p=0.60). It tops the unweighted Total column (38.2%, n=327) only because active volunteers make up 65.7% of respondents despite being 29.1% of the invited pool (52.4% vs 11.2% completion rate). By contrast, lapsed volunteers are 3.5x to 5.5x more likely than active volunteers to report short-notice shift changes (51.8% vs 14.4%), hard-to-arrange shifts (42.0% vs 11.6%), and not being told about changes in good time (41.1% vs 7.4%).
-
Shift scheduling and rota communication are the primary controllable drivers of volunteer churn—and software alone will not fix them without operational discipline led by a coordinator.
Among lapsed respondents (Q3, n=112), 42.9% (±9.2 pp) cite scheduling as their main reason for leaving: 29.5% short-notice shift changes and 13.4% difficulty arranging/swapping shifts (41.5% when age-weighted to the database). Adding not being told about changes in good time (12.5%) brings rota and communication failures to 55.4% of leavers (52.1% weighted), versus 11.6% unweighted for travel cost and 23.2% for changed personal circumstances. In Q1 open text (n=109), 29 respondents mention scheduling (20 booking/swapping, 9 last-minute changes/cancellations) and 13 mention poor communication/lack of notice (8 exclusively). Multi-site volunteers—who suffer most from 9 separate spreadsheets—are overrepresented among leavers (10.7% of lapsed vs 4.2% of active, Q6, z=2.28, p=0.02).
-
Chair of trustees' question: We cannot reliably project how many lost volunteers will return with online booking, and applying the 40.2% stated-intent figure to 1,000 lapsed volunteers (~402 returners) would severely overestimate reality. Actual reactivation is likely to be in the tens rather than hundreds, making retention of active and new volunteers the stronger case for investment.
In Q4 (n=112), 40.2% of lapsed respondents say they are fairly (23.2%) or very (17.0%) likely to return if they could book shifts online (vs 26.8% for travel reimbursement in Q5). However, four factors heavily discount this figure: (1) stated-intent inflation—28.0% of those who left for unrelated reasons such as personal circumstances (n=50) still say they are likely to return for online booking; (2) severe non-response bias—the 11.2% of lapsed volunteers who answered (112/1,000) are likely the warmest segment; (3) Q4 tests online booking/swapping only, not reducing last-minute cancellations (the larger 29.5% exit driver); and (4) older respondents are overrepresented (39.3% are 65+ vs 25% in the database). Even discounting from the 17.0% 'very likely' share and accounting for 888 non-respondents implies dozens—not hundreds—of returning volunteers.
-
Volunteer manager's question: Under-25 leavers show a striking travel-cost signal, but the base (n=11) is far too small to launch an under-25s-only travel scheme—and travel hardship also affects older volunteers on fixed incomes.
Among lapsed under-25s (n=11, representing just ~5% of the ~220 lapsed under-25s in the database), 72.7% (8/11) cite travel cost as their main exit reason (Q3) and 81.8% (9/11) as a difficulty (Q2), carrying a 95% margin of error of roughly ±26 pp (Wilson CI ~43%–90%). Across all under-25s (n=27), 74.1% cite travel cost (including 68.8% of active under-25s, n=16), suggesting a genuine pattern that merits verification. However, lapsed under-25s also heavily report poor change notice (54.5%) and short-notice changes (45.5%), and open-text comments show older volunteers—including pensioners and drivers facing £4–£7 parking or £5–£9 transit fares (e.g., V034, V158, V189, V234, V237)—also struggle with travel costs. Any travel support should be opt-in and expense-capped for anyone in need rather than age-gated.
-
Severe age skew in survey responses understates the true prevalence of travel costs among lapsed volunteers, strengthening the case for a hybrid approach rather than ignoring travel entirely.
Under-45s represent 44% of active volunteers and 52% of lapsed volunteers in the database, but only 20.4% of active respondents (44/215) and 28.6% of lapsed respondents (32/112). Weighting lapsed Q2 responses to the database age mix raises travel-cost difficulty from 40.2% to 47.4% (and active travel cost from 37.2% to 41.5%). Weighting Q3 raises travel cost as main exit reason from 11.6% to roughly 20.7% (though heavily driven by the noisy n=11 under-25 cell), while scheduling remains the top driver at 41.5% weighted (52.1% with poor communication). Q4 and Q5 also show non-overlapping audiences: scheduling leavers respond to online booking (60.4% vs 27.1%, n=48), whereas travel leavers respond to reimbursement (57.1% vs 14.3%, n=14).
-
Low-cost operational fixes surfaced in open text—especially site-specific parking validation, lift-sharing, cross-site shift visibility, and basic onboarding—can reduce travel and friction barriers without draining cash reimbursement funds.
Among the 71 substantive Q1 suggestions (excluding 38 'nothing needed' replies), volunteers specifically highlight £4–£7 pay-and-display parking fees, expensive public transport links, inability to see open shifts across the 9 sites, and inconsistent induction. Negotiating free or validated parking with site hosts, facilitating lift-shares, replacing the 9 separate spreadsheets with a single cross-site rota, and standardising onboarding directly address recurring complaints at minimal ongoing cost.
The survey it planned
1 screening question and 10 questions, as the model wrote them.
-
S1
When did you last complete a volunteer shift with Riverside Food Network?
One answer- Within the last 3 months
- Between 3 and 12 months ago
- More than 12 months ago
- I have never completed a shift with Riverside Food Network
Continues if Within the last 3 months; Between 3 and 12 months ago
-
Q1
In your own words, what are the main reasons you have not volunteered with Riverside Food Network in the last 3 months?
Open textRouting Shown only if S1 is: Between 3 and 12 months ago
-
Q2
Which of the following reasons played a part in you stopping or pausing your volunteering with Riverside Food Network? Please select all that apply.
Any that apply- Changes in my work, study, health, or family commitments
- Moving away or changes in how easily I can get to the site
- The cost of travelling to the site (bus fares, fuel, or parking)
- Difficulty booking, confirming, or swapping shifts by phone or message
- Shift times not fitting around my availability
- Shifts feeling short-staffed or disorganised
- The physical or emotional demands of the role
- Not feeling well supported or kept up to date by the charity
- Taking up another volunteering role or commitment
- Other
- Prefer not to say
Routing Shown only if S1 is: Between 3 and 12 months ago
-
Q3
Thinking about your time volunteering with Riverside Food Network in the last 12 months, which of the following, if any, have been difficult for you? Please select all that apply.
Any that apply- Paying for travel to and from the site (bus fares, fuel, or parking)
- Getting to the site or finding parking
- Booking, confirming, or swapping shifts by phone or message
- Finding shift times that fit my schedule
- Dealing with last-minute changes or short-staffed shifts
- Getting timely communication or support from a site lead
- The physical or emotional demands of sorting donations or handing out parcels
- None of these have been difficult for me
- Other
-
Q4
How much of a problem, if at all, has the cost of travelling to and from your shifts been for you?
ScaleScale 1-5: Not a problem at all to A major problem
-
Q5
How much of a problem, if at all, has booking, confirming, or swapping shifts by phone or message been for you?
ScaleScale 1-5: Not a problem at all to A major problem
-
Q6
Roughly how much does it cost you in total (in pounds, rounded to the nearest whole pound) to travel to and from a single volunteer shift, including bus fares, fuel, or parking? Please enter 0 if you walk, cycle, use a free bus pass, or have no travel costs.
Number -
Q7
If Riverside Food Network introduced the changes below, which would make you likely to start volunteering again?
One answer- Reimbursing my travel costs alone would make me likely to return
- Online shift booking with a volunteer coordinator alone would make me likely to return
- Either of these changes on its own would make me likely to return
- I would only return if both travel reimbursement and online shift booking with a coordinator were introduced
- Neither of these changes would make me likely to return
- Not sure
Routing Shown only if S1 is: Between 3 and 12 months ago
-
Q8
Riverside Food Network has a fixed budget next year to support volunteers. Which of the following options would be most helpful to you personally?
One answer- Reimbursing volunteers' full travel costs (bus fares, fuel, or parking)
- Introducing online shift booking supported by a part-time volunteer coordinator
- A smaller version of both (for example, capped travel reimbursement alongside simpler online shift booking)
- Something else
- None of these would make a difference to me
-
Q9
Which Riverside Food Network distribution site have you volunteered at most often in the last 12 months?
One answer- Distribution Site 1
- Distribution Site 2
- Distribution Site 3
- Distribution Site 4
- Distribution Site 5
- Distribution Site 6
- Distribution Site 7
- Distribution Site 8
- Distribution Site 9
- I volunteered equally across multiple sites
- Not sure / prefer not to say
-
Q10
Which age group do you belong to?
One answer- Under 25
- 25 to 34
- 35 to 44
- 45 to 54
- 55 to 64
- 65 to 74
- 75 or over
- Prefer not to say
Sample plan
Invite a full census of all ~1,410 volunteers on the database who completed at least one shift in the last 12 months (410 active volunteers with a shift in the last 3 months and ~1,000 lapsed volunteers with no shift in the last 3 months) via both email and SMS, with up to two reminders spaced 4-5 days apart. Do not use hard quotas, because turning away eligible respondents from a finite census reduces precision; instead, monitor completes closely and send targeted reminders to lapsed volunteers (who typically respond at lower rates than active volunteers) and under-responding sites, aiming for roughly 330 total completes (~150-180 active and ~150-180 lapsed, ~23% overall response rate) and post-stratification weighting by active/lapsed status and site for overall estimates. Randomise response option order for Q2, Q3, and Q8 (keeping 'Other', 'None', and 'Prefer not to say' fixed at the end) to prevent order bias and counter stakeholder assumptions about travel costs. CRITICAL PRECISION FEASIBILITY NOTE FOR TRUSTEES: The requested precision of +/-7 percentage points at 95% confidence for each of the 9 sites is mathematically unachievable with ~330 completes. Even applying a finite population correction (average ~157 volunteers per site), +/-7 pp at 95% confidence (worst-case p=0.5) requires ~88 completes per site (~790 completes total, or a ~56% response rate at every site including lapsed volunteers). With ~330 total completes (~37 per site on average), achievable 95% margins of error (with finite population correction) are approximately +/-4.7 pp overall, +/-6.4 to +/-6.6 pp for active versus lapsed comparisons, and roughly +/-14 pp per individual site (wider at smaller sites). Trustees should be briefed before launch that individual site results will be indicative rather than +/-7 pp estimates, and sites can be grouped by characteristics (e.g., transport access or size) where needed for reliable statistical comparisons. Also monitor non-response bias by comparing early vs. reminder responders and checking completed sample composition against database records by site and status.