Benchmarks / SurveyBench / Why Riverside Food Network is losing volunteers / Inkling 2 Mini
Inkling 2 Mini: Why Riverside Food Network is losing volunteers
The decision: how to spend next year's £45,000 for volunteer support: reimburse volunteers' travel costs, fund a part-time volunteer coordinator with online shift booking, or do both at a smaller scale. The brief and the data →
- Verdict
- ✗ Not sound
- Research score
- 75 of 100
- Analysis rating
- 567
- Head to head, this task
- won 8 of 11
Why it is not sound
The survey
- A research objective the survey could not answer
- Answer bands that overlap or leave gaps
The analysis
- Reported a material finding the data does not support
- Missed a trap: Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
- Missed a trap: 40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.
- Proposed a mistaken next step: Forecast how many lapsed volunteers will return from the likely-to-return answers
- Proposed a mistaken next step: Set up a travel scheme for under-25s on the strength of the lapsed under-25 result
Findings the judges found unsupported
The two policy levers work on different groups: scheduling-motivated lapsed would be likely to return with online booking (60.4%, n=48) while travel-motivated lapsed would return with reimbursement (57.1%, n=14) but NOT with booking alone (14.3%); travel reimbursement does little for scheduling-motivated people (27.1%) and vice versa (booking lifts travel-motivated to 14.3%).
The questions show hypothetical stated willingness, not that the policy levers work or that respondents would actually return. Saying reimbursement does little for scheduling-motivated volunteers also treats intention levels as intervention effects without a baseline or behavioural test.Projected lapsed volunteers with a scheduling reason = ~429 (42.9% of 1,000); if 60.4% would return with online booking this is ~259 likely returns, with a ±9.2-pt unweighted margin on the 42.9% share (±92 in absolute terms).
Projecting approximately 259 returns multiplies an unrepresentative sample's main-reason share by hypothetical return intentions. Neither the population count nor actual returns is established. The low-confidence label and sampling margin do not make this a defensible forecast or account for nonresponse bias.The lapsed-volunteer responses are directional, not decision-grade: only 112/1000 lapsed completed (11.2%) vs 215/410 active (52.4%), and the lapsed sample skews older and heavier in 65+ (39.3% vs 25%) and lighter in under-25s (9.8% vs 22%), so unweighted percentages overstate travel/scheduling pressures among senior and understate them among young volunteers.
Age imbalance changes the aggregate mix; it does not establish that pressures within senior volunteers are overstated or those within younger volunteers understated. Younger respondents report higher travel-cost difficulty, but scheduling difficulties do not show a consistent younger-heavy pattern.
Scorecard
Numbers 12 of 12 right
| Question | Its answer | True | |
|---|---|---|---|
| What percentage of active volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2? | ✓ | 37.2% | 37.2% |
| What percentage of lapsed volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2? | ✓ | 40.2% | 40.2% |
| Is the difference between active and lapsed respondents in the share choosing 'The cost of travelling to the site' in Q2 statistically significant at the 95% level? (1 for yes, 0 for no) | ✓ | No | No |
| Weighted to the age mix of all 1,000 lapsed volunteers, what percentage of lapsed volunteers would choose 'The cost of travelling to the site' as a difficulty in Q2? | ✓ | 47.4% | 47.4% |
| What percentage of lapsed respondents gave a scheduling reason as their main reason for stopping in Q3 (either of its first two options: shifts changed at short notice, or hard to arrange shifts)? | ✓ | 42.9% | 42.9% |
| What is the 95% margin of error, in percentage points, on that unweighted share of lapsed respondents giving a scheduling reason as their main reason? | ✓ | 9.2 points | 9.17 points |
| On average, how many difficulties did each respondent choose in Q2 (counting every option except 'None of these', which counts as zero), across all respondents? | ✓ | 1.55 | 1.55 |
| What percentage of lapsed respondents said they would be likely (fairly or very) to volunteer again if they could arrange their shifts online (Q4)? | ✓ | 40.2% | 40.2% |
| How many of the open-text answers mention shift scheduling (booking or swapping shifts, or last-minute changes or cancellations)? | ✓ | 37 | 34 |
| What percentage of the active volunteers invited completed the survey? | ✓ | 52.4% | 52.4% |
| What percentage of the lapsed volunteers invited completed the survey? | ✓ | 11.2% | 11.2% |
| If the lapsed respondents were representative, about how many of the 1,000 lapsed volunteers would give a scheduling reason (either of the first two Q3 options) as their main reason, using the unweighted share? | ✓ | 429 | 429 |
Traps in the data 6 of 8 handled
- ✓Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
- ✓The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.
- ✓Q2 is multi-select: respondents chose 1.6 difficulties on average, so columns add up to more than 100%.
- ✓52% of active volunteers responded but only 11% of lapsed ones, so the lapsed base is small (112) and those who replied may differ from the 89% who did not (for example, still feeling some attachment).
- ✓Volunteers aged 65 or over are 43% of respondents but about 26% of volunteers in the database; younger volunteers cite travel cost more. Weighted to the lapsed age mix, travel cost among lapsed volunteers rises from 40% to 47%; a scheduling main reason goes from 43% to 41%, still well ahead of travel cost (21%).
- ✗Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
- ✓The travel-cost gap between lapsed and active respondents (3.0 points) is not significant at 95%.
- ✗40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.
The analysis
Do both, weighted toward the coordinator and online booking — do NOT spend the £45,000 purely on travel reimbursement. The data says scheduling is the dominant driver for lapsed volunteers (51.8% short-notice changes, 42.0% hard to arrange, 41.1% not informed in good time; 42.9% main reason), and 60.4% of scheduling-motivated lapsed would return with online booking, whereas travel matters most for under-25s (81.8% cite cost, 72.7% give it as main reason; reimbursement returns 57.1%). Fund the coordinator + a self-service booking/swap/cancellation system (with mandatory notice/confirmation emails) as the core; reserve a smaller line for a capped under-25 travel scheme; and tighten rota-change communication regardless of the money split. Stakeholder answers: (Operations lead) No — travel cost is the most-mentioned difficulty overall (38.2%) but that finding applies equally to the 215 active volunteers and does not explain lapsed volunteers; the lapsed/active travel gap is not significant, so £45k of reimbursements alone solves the wrong problem. (Chair) If scheduling is fixed, expect roughly 40% of lapsed volunteers to respond positively (40.2% in Q4, ~402 of 1,000, with scheduling-motivated ~60.4% or ~259 likely returns); do not bank on this without a re-survey. (Volunteer manager) Yes, targeted under-25 scheme — justify it with the age-weighted travel share (47.4% of all lapsed) and the under-25 72.7%/81.8% figures; keep it capped and measured.
The recommendation would flip toward travel-heavy spending if a properly powered lapsed-volunteer survey (n>=200, stratified for under-25s and older volunteers) shows a travel-cost-driven main reason across all ages, or if travel costs per visit plus actual per-visit spending per returned volunteer is measured and found to be more cost-efficient than booking; or if actual booking-pilot results show materially lower return rates than the Q4 hypothetical suggests (e.g., booking raises returns near zero), then the money shifts to travel. Either way, more evidence before the budget is committed.
Findings
-
Scheduling is the lapsed-volunteer problem: 51.8% cite short-notice changes, 42.0% a hard-to-arrange booking/swap process, and 41.1% not being told about changes in time (vs 14.4%/11.6%/7.4% for active volunteers).
Q2: active base 215, lapsed base 112. Q3 main reason for stopping: shifts changed 29.5% + hard to arrange 13.4% = 42.9% scheduling-motivated lapsed; 29.5+13.4=42.9, +23.2% circumstances, 12.5% notice, 11.6% travel cost, 6.2% valuation, 0.9% distance, remainder ~3.6%
-
Travel cost is the most-mentioned difficulty across all respondents (38.2% overall, the same 37.2% among actives), but the active/lapsed difference (37.2% vs 40.2%, ~3 pts, z≈0.53) is not statistically significant, so it is not the differentiator between the two groups.
Q2: active n=215 (37.2% travel), lapsed n=112 (40.2% travel); difference 3.0 points not significant at 95%
-
Under-25 lapsed volunteers are travel-heavy: 81.8% cite travel cost as a difficulty and 72.7% give travel as their main reason for stopping, but the base is tiny (11 lapsed under-25s, n=8 of 11 give travel as reason) and the sample under-represents that group (9.8% of sample vs 22% of the lapsed population).
Q2 lapsed by age (base 11): 81.8% cost; Q3 lapsed under-25 (base 11): 72.7% travel cost as main reason; weighted travel-cost share of all 1,000 lapsed = 47.4% (population mix 22%/30%/23%/25%, shares 81.8%/42.9%/47.2%/22.7%)
-
The two policy levers work on different groups: scheduling-motivated lapsed would be likely to return with online booking (60.4%, n=48) while travel-motivated lapsed would return with reimbursement (57.1%, n=14) but NOT with booking alone (14.3%); travel reimbursement does little for scheduling-motivated people (27.1%) and vice versa (booking lifts travel-motivated to 14.3%).
Q4: scheduling 60.4% (base 48), travel 14.3% (base 14), other 28.0% (base 50). Q5: scheduling 27.1%, travel 57.1%, other 18.0%
-
Projected lapsed volunteers with a scheduling reason = ~429 (42.9% of 1,000); if 60.4% would return with online booking this is ~259 likely returns, with a ±9.2-pt unweighted margin on the 42.9% share (±92 in absolute terms).
Unweighted lapsed share 42.9% of 112; fictional Q4/Q5 hypothetical-likelihoods; sample not population-representative
-
The lapsed-volunteer responses are directional, not decision-grade: only 112/1000 lapsed completed (11.2%) vs 215/410 active (52.4%), and the lapsed sample skews older and heavier in 65+ (39.3% vs 25%) and lighter in under-25s (9.8% vs 22%), so unweighted percentages overstate travel/scheduling pressures among senior and understate them among young volunteers.
Q7 lapsed: under 25 9.8%, 25-44 18.8%, 45-64 32.1%, 65+ 39.3% (population: 22%/30%/23%/25%); lapsed response rate 11.2% vs active 52.4%
Caveats
- Lapsed-volunteer responses rest on a small, off-representative base (112/1,000; 11.2% response rate, skewed 65+ and under-weighted under-25s); all lapsed-based shares, including the 42.9% scheduling share and the 57.1%/60.4% returns, should be treated as directional with the stated margins (±9.2 pts for the 42.9% scheduling share; under-25 cell n=11).
- Active-vs-lapsed travel-cost difference (37.2% vs 40.2%) is not statistically significant at 95%; the operations lead's inference that travel is the differentiator is not supported by the data.
- Q4 and Q5 are hypothetical intention questions ('if you could arrange shifts online', 'if your travel costs were paid back') — they measure stated willingness to return, not actual behaviour, and are subject to social-desirability bias.
- The 37/109 open-text answers (34%) that mention scheduling vs 23 travel-cost (21%) and 8 distance (7%) are impressionistic and unweighted; 38 of 109 (35%) say nothing is needed.
- The scheduling-vs-travel return split is based on Q3's main-reason choices (single-select) and the grouping in the table (scheduling = first two options; travel = cost/distance); the underlying cell proportions (n=48, n=14, n=50) are small.
- No dollar weights are given: £45,000 cannot be allocated by ROI without cost-per-shift, average travel amounts, and per-category budgets from the charity's own books.
Next steps
- researchRun a second, properly powered survey of lapsed volunteers (stratified for under-25s, 45-64 and 65+, re-weighted); target ~200+ lapsed responses and at least 50 under-25s
- decisionApprove the coordinator + online booking/swap/cancellation system as the core investment (majority of £45,000), with published rota windows and mandatory shift-change notice emails
- experimentRun a capped under-25 travel pilot (max £X-capped per booking/shift) for under-25 and under-30 lapsed volunteers only, with explicit per-visit cap and reimbursement record
- monitorTrack, monthly, booking uptake, cancellation/short-notice events, and actual lapsed-volunteer returns (not just stated willingness); report to the operations lead with the under-25 scheme pilot
- decisionDecide the under-25 travel scheme scale at month 6 based on pilot results; reallocate any unused coordinator/booking budget to travel if pilot shows travel drives returns better
Open-text themes it coded
Shift scheduling (booking, swapping, cancellations, notice) 37Travel costs (fares, fuel, mileage, parking fees, reimbursement) 23Travel distance/location (nearest site, closer site, travel time) 8Recognition/appreciation 2Training/induction 1Nothing/fine as it is 38
The survey it planned
1 screening question and 10 questions, as the model wrote them.
-
SCR1
In the last 12 months, did you ever do a shift with Riverside Food Network?
One answer- Yes
- No
Continues if Yes
-
Q1
When did you last do a shift with Riverside Food Network?
One answer- In the last month
- 1–3 months ago
- 3–6 months ago
- Over 6 months ago
-
Q2
Which of the following, if introduced, would make it most likely that you keep volunteering with Riverside Food Network (or return to volunteer if you have stopped)? Choose one.
One answer- Online shift booking (book and swap shifts online)
- Travel cost reimbursement (bus fares, parking and so on)
- Both online shift booking and travel cost reimbursement
- Neither — some other change would help
Judges: assumptiveThe combined option offers both benefits without the smaller-scale trade-off in the decision, favouring that option. 'Neither — some other change would help' assumes another change would help and does not accommodate someone who would not return under any circumstances. Choosing the most promising option does not establish that it would actually bring someone back.
The only alternative to the two interventions is 'Neither — some other change would help'. This assumes some change would bring the respondent back or keep them. There is no option for people who would not return whatever changed, such as those with health, family or time reasons. They are forced into a false answer, which distorts O3.
-
Q3
Which of the following difficulties, if any, have you had when volunteering with Riverside Food Network in the last 12 months? Choose all that apply.
Any that apply- The cost of travel (bus fares, parking)
- The time it takes to travel
- Difficulty parking or getting to the site
- Unavailable or inconvenient transport
- Difficulty booking or swapping shifts
- Not enough free time
- Health or mobility problems
- Other
-
Q4
In the last month you volunteered, about how much did you spend on travel to Riverside Food Network sites (e.g., bus fares, parking, driving)? Enter an amount in £, or leave blank if it was zero.
Number -
Q5
Which of the following were reasons you stopped volunteering? Choose all that apply.
Any that apply- The cost of travel
- The time it took to travel
- Difficulty parking or getting to the site
- Difficulty booking or swapping shifts
- Not enough free time
- Health or family commitments
- Other
Routing Shown only if Q1 is: 3–6 months ago; Over 6 months ago
Judges: assumptiveHaving no shift for three months does not establish that someone has stopped volunteering. The routing assumes these respondents stopped and excludes people who stopped more recently.
It is shown to anyone whose last shift was 3+ months ago and asks why they 'stopped'. Some of these people (seasonal, occasional or on a break) do not see themselves as having stopped, and there is no option to say so.
-
Q6
In your own words, what was the main reason you stopped volunteering?
Open textRouting Shown only if Q1 is: 3–6 months ago; Over 6 months ago
Judges: assumptiveThe routing assumes everyone whose last shift was over three months ago has stopped volunteering, while excluding recent stoppers.
It uses the same display logic as Q5, so it carries the same assumption: people with no shift for 3+ months may not consider that they have stopped volunteering.
-
Q7
For this question only, please select 'Neither agree nor disagree'.
ScaleScale 1-5: Strongly disagree to Strongly agree
-
Q8
Which Riverside Food Network distribution site have you volunteered at most in the last 12 months? (Choose one.)
One answer- Site 1
- Site 2
- Site 3
- Site 4
- Site 5
- Site 6
- Site 7
- Site 8
- Site 9
-
Q9
Which age group do you fall in?
One answer- 18–24
- 25–39
- 40–54
- 55–64
- 65+
-
Q10
How many shifts have you done with Riverside Food Network in the last 12 months?
Number
Sample plan
Invite everyone on the volunteer database who did at least one shift in the last 12 months: approximately 410 active volunteers (at least one shift in the last 3 months) and approximately 1,000 lapsed volunteers (no shift in the last 3 months), ~1,410 contacts, via email and text message, with two reminders. Expect roughly 330 completed responses (~23%); this will NOT be a quota-based sample — completions self-select, so weight at analysis so estimates reflect the database profile (active:lapsed ≈29%:71%) rather than who happens to respond. Monitor completions by site, status and age in real time; if any of the 9 sites is well below ~35 completes, report it descriptively and flag the low precision. Honest precision check for the trustees: ±7 percentage points at 95% confidence requires ~196 valid respondents per site (n = 1.96² × 0.25 / 0.07² ≈ 196), i.e. ~1,760 completes across 9 sites — not feasible with ~330 (~37 per site, giving roughly ±16 percentage points). With 330 completes the overall pooled estimate is ≈ ±5.4 percentage points. Recommend trustees either accept pooled ±6–7 pp + descriptive site-level results, or raise the completes target to ~1,800 (or drop the per-site ±7 pp requirement) before launch. The nine real site names should replace the 'Site 1–9' placeholders in Q8 before launch. Note: the nine site names used in Q8 are placeholders pending the actual site list.