Benchmarks / SurveyBench / Why Riverside Food Network is losing volunteers / Claude Opus 5.5

Measured by Spring Prompt

Claude Opus 5.5: Why Riverside Food Network is losing volunteers

The decision: how to spend next year's £45,000 for volunteer support: reimburse volunteers' travel costs, fund a part-time volunteer coordinator with online shift booking, or do both at a smaller scale. The brief and the data →

Verdict
✓ Sound research
Research score
97 of 100
Analysis rating
1,646
Head to head, this task
won 9 of 11

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What percentage of active volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓37.2%37.2%
What percentage of lapsed volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓40.2%40.2%
Is the difference between active and lapsed respondents in the share choosing 'The cost of travelling to the site' in Q2 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Weighted to the age mix of all 1,000 lapsed volunteers, what percentage of lapsed volunteers would choose 'The cost of travelling to the site' as a difficulty in Q2?✓47.4%47.4%
What percentage of lapsed respondents gave a scheduling reason as their main reason for stopping in Q3 (either of its first two options: shifts changed at short notice, or hard to arrange shifts)?✓42.9%42.9%
What is the 95% margin of error, in percentage points, on that unweighted share of lapsed respondents giving a scheduling reason as their main reason?✓9.2 points9.17 points
On average, how many difficulties did each respondent choose in Q2 (counting every option except 'None of these', which counts as zero), across all respondents?✓1.551.55
What percentage of lapsed respondents said they would be likely (fairly or very) to volunteer again if they could arrange their shifts online (Q4)?✓40.2%40.2%
How many of the open-text answers mention shift scheduling (booking or swapping shifts, or last-minute changes or cancellations)?✓3634
What percentage of the active volunteers invited completed the survey?✓52.4%52.4%
What percentage of the lapsed volunteers invited completed the survey?✓11.2%11.2%
If the lapsed respondents were representative, about how many of the 1,000 lapsed volunteers would give a scheduling reason (either of the first two Q3 options) as their main reason, using the unweighted share?✓429429

Traps in the data 8 of 8 handled

  • ✓Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
  • ✓The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.
  • ✓Q2 is multi-select: respondents chose 1.6 difficulties on average, so columns add up to more than 100%.
  • ✓52% of active volunteers responded but only 11% of lapsed ones, so the lapsed base is small (112) and those who replied may differ from the 89% who did not (for example, still feeling some attachment).
  • ✓Volunteers aged 65 or over are 43% of respondents but about 26% of volunteers in the database; younger volunteers cite travel cost more. Weighted to the lapsed age mix, travel cost among lapsed volunteers rises from 40% to 47%; a scheduling main reason goes from 43% to 41%, still well ahead of travel cost (21%).
  • ✓Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
  • ✓The travel-cost gap between lapsed and active respondents (3.0 points) is not significant at 95%.
  • ✓40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.

The analysis

Recommendation

Spend the majority (roughly £30–35k) on a part-time volunteer coordinator plus a simple online booking and swapping system with automatic text or email alerts when shifts change. Put a rota-change rule in place alongside it, for example a minimum notice period and a single channel for announcing changes. This targets the problems that most distinguish leavers, which account for about 55% of main reasons for stopping. Reserve a smaller, capped travel fund (roughly £10–15k). Make it needs-based (for example, claimable by any volunteer who says cost is a barrier, with a per-trip cap) rather than for under-25s only, while making sure young volunteers know about it. The evidence for an age-specific scheme rests on 11 people, and lapsed 45 to 64s report travel cost at a similar rate. Do not put the whole £45,000 into travel reimbursement. Travel cost is a common complaint, but it does not separate leavers from stayers, and only 12% of leavers gave it as their main reason.

What would change it

Three kinds of evidence would shift the balance. First, a booking pilot that brings back very few lapsed volunteers while a travel-fund pilot does, especially if, once weighted, travel turns out to be the main reason for a much larger share of leavers. Second, a larger young-volunteer sample confirming travel as the dominant barrier, together with high take-up and retention gains from reimbursement. Third, costings showing that needs-based reimbursement would take far more than the reserved amount, or that booking software is available cheaply without a coordinator.

Findings

  1. Travel cost is the most commonly mentioned difficulty, but active and lapsed volunteers mention it about equally. It is a widespread irritation rather than something that sets leavers apart, so on its own it does not show that travel is what drives people away.

    High confidenceBase 327

    Q2 travel cost: 37.2% of active (base 215) vs 40.2% of lapsed (base 112). The 3-point gap is not statistically significant (z≈0.5). Weighted to the lapsed age profile the lapsed figure rises to about 47%, still a modest gap. Only 11.6% of lapsed respondents (13 of 112) gave travel cost as their main reason for stopping.

  2. Scheduling and rota communication problems are what most clearly separate people who left from people who stayed. They are the largest single group of main reasons for stopping.

    High confidenceBase 112

    Q2, lapsed vs active: shifts changed at short notice 51.8% vs 14.4%; no easy way to arrange shifts 42.0% vs 11.6%; not told about changes in good time 41.1% vs 7.4%. Q3: 42.9% of lapsed respondents (48 of 112, ±9.2pp) gave a scheduling reason as their main reason. A further 12.5% said they were not told about changes in good time, a closely related communication problem. That puts about 55% on rota and scheduling issues, against 12.5% for travel (cost or time) and 23.2% for changed circumstances.

  3. Online booking appeals most to people who left over scheduling. The 40% 'likely to return' figure is stated intention from the most engaged tenth of lapsed volunteers. It is not a forecast of how many will actually come back.

    Medium confidenceBase 112

    Q4 net likely: 40.2% overall (45 of 112); 60.4% among those whose main reason was scheduling (base 48); 14.3% among travel leavers (base 14). Q5 (travel repaid) net likely: 26.8% overall; 57.1% among travel leavers (base 14), but only 27.1% among scheduling leavers. Response rate among lapsed volunteers was 11.2%.

  4. Travel cost is a real barrier for the youngest volunteers, and it is the main reason most young leavers gave for stopping. However, the numbers are too small to size a scheme on.

    Low confidenceBase 11

    Q2 travel cost: 74.1% of all under-25 respondents (base 27); 81.8% of lapsed under-25s (9 of 11). Q3: 72.7% of lapsed under-25s (8 of 11) gave travel cost as their main reason, with a margin of roughly ±26pp. Travel cost is also high among lapsed 45 to 64s (47.2%, base 36). Under-25s are under-represented among lapsed respondents (9.8% vs 22% of the lapsed population).

  5. The open-text answers back this up. Booking, swapping and late changes come up more often than travel, and most of the people raising scheduling problems are lapsed volunteers. Travel comments come mostly from active volunteers, describing a cost they currently put up with.

    Medium confidenceBase 109

    Of 109 open-text answers, 36 mention scheduling (booking or swapping, late changes or cancellations) and about 29 mention travel cost or distance. 38 say nothing needs changing, mostly active volunteers. Several comments show changes being passed on by word of mouth only (V009, V055, V134, V308). Some of this could be fixed through process alone, without new technology.

  6. A meaningful minority of leavers stopped for reasons no spending can address.

    Medium confidenceBase 112

    Q3: 23.2% of lapsed respondents gave changed circumstances (work, study, health or moving), rising to 38.9% of lapsed 45 to 64s (base 36). Among those who left for 'something else', only 28% would be likely to return with online booking and 18% with travel repaid (base 50).

Caveats

  • Only 11.2% of lapsed volunteers completed the survey (112 of 1,000), against 52.4% of active volunteers. Lapsed respondents are probably the more engaged and goodwill-rich leavers, so projections to all 1,000 are uncertain and likely optimistic.
  • The sample is skewed older. Under-25s are 9.8% of lapsed respondents vs 22% of the lapsed population, and 65+ are 39.3% vs 25%. Tables are unweighted. Weighting raises lapsed travel-cost mentions from 40.2% to about 47%.
  • Under-25 results rest on 11 lapsed and 27 total respondents. These are directional only (margin around ±26pp on the 72.7% figure).
  • Q4 and Q5 are hypothetical stated intentions, and they were asked one after the other. They will overstate actual return rates and may be affected by question order. The 'travel' column in those tables has a base of only 14.
  • Lapsed volunteers are defined as no shift in the last 3 months but one within 12 months. Those gone for more than 12 months were screened out. Respondents were classed by their own answer to S1, which may not match the database.
  • Q2 asks about difficulties ever experienced. Lapsed respondents are recalling past experience, and may rationalise why they left.
  • The survey did not measure costs: fares per trip, trips per year, or likely take-up of reimbursement. So the cost of a travel scheme cannot be estimated from these data.
  • Some scheduling complaints are about process: rotas changed without anyone being told, and reliance on one person's inbox. Software alone will not fix these without a coordinator owning communication.

Next steps

  • decisionCommit to a part-time coordinator and online booking with change alerts, and introduce a rota-change notice standard at all 9 sites.
  • experimentBefore full rollout, pilot online booking at 2–3 sites and invite lapsed volunteers from those sites back, with a direct link to book.
  • experimentPilot a capped, needs-based travel reimbursement at a few sites, tracking claims per volunteer, cost per shift and take-up by age.
  • researchRun short follow-up interviews or a targeted survey with young volunteers (active and lapsed), boosting the sample to at least 50.
  • monitorTrack monthly lapse rate, reasons given at exit, and late-change incidents per site from the new system.

Open-text themes it coded

Travel cost (fares, fuel, parking charges) 22Travel time or distance to site 9Online booking, swapping or recurring shifts 20Last-minute changes and not being told in good time 19Parking space and physical access 2Recognition and feeling valued 2Training 1Nothing to change 38

The survey it planned

1 screening question and 10 questions, as the model wrote them.

  1. S1

    When did you last do a volunteer shift with Riverside Food Network?

    One answer
    • In the last 3 months
    • Between 3 and 12 months ago
    • More than 12 months ago
    • I have never done a shift with Riverside Food Network

    Continues if In the last 3 months; Between 3 and 12 months ago

  2. Q1

    Which of these best describes your volunteering with Riverside Food Network now?

    One answer
    • I still volunteer about as often as I used to
    • I still volunteer, but less often than I used to
    • I have stopped volunteering, for now or for good
  3. Q2

    In the last 12 months, which of these, if any, made it harder for you to volunteer? Choose all that apply.

    Any that apply
    • The cost of getting to the site (bus fares, parking or fuel)
    • How long it took, or how far it was, to get to the site
    • Shift times did not fit around my work, study or caring
    • It was hard to book or swap a shift by phoning or messaging the site lead
    • I did not hear about available shifts early enough
    • Shifts were cancelled or changed at short notice
    • I did not feel welcomed, supported or valued
    • The work was too physically demanding
    • My health changed
    • My circumstances changed (for example a new job, moving home or caring for someone)
    • Something else
    • None of these
  4. Q3

    What was the main reason you stopped volunteering, or now volunteer less often? Choose one.

    One answer
    • The cost of getting to the site (bus fares, parking or fuel)
    • How long it took, or how far it was, to get to the site
    • Shift times did not fit around my work, study or caring
    • It was hard to book or swap a shift by phoning or messaging the site lead
    • I did not hear about available shifts early enough
    • Shifts were cancelled or changed at short notice
    • I did not feel welcomed, supported or valued
    • The work was too physically demanding
    • My health changed
    • My circumstances changed (for example a new job, moving home or caring for someone)
    • I only planned to volunteer for a short time
    • Something else

    Routing Shown only if Q1 is: I still volunteer, but less often than I used to; I have stopped volunteering, for now or for good

  5. Q4

    On a typical shift, about how much did it cost you to get to the site and back home? Include bus fares, parking and fuel. Enter a whole number of pounds. Enter 0 if it cost you nothing.

    Number
  6. Q5

    Suppose Riverside Food Network paid back your bus fares and parking for every shift. How likely would you be to volunteer again, or to volunteer more often?

    Scale

    Scale 1-5: Not at all likely to Very likely

    Routing Shown only if Q1 is: I still volunteer, but less often than I used to; I have stopped volunteering, for now or for good

  7. Q6

    Suppose Riverside Food Network had a volunteer coordinator you could contact, and a website where you could see, book and swap shifts yourself at any time. How likely would you be to volunteer again, or to volunteer more often?

    Scale

    Scale 1-5: Not at all likely to Very likely

    Routing Shown only if Q1 is: I still volunteer, but less often than I used to; I have stopped volunteering, for now or for good

  8. Q7

    If Riverside Food Network could only make one of these changes, which would make the bigger difference to you?

    One answer
    • Paying back bus fares and parking
    • A volunteer coordinator and a website to book and swap shifts
    • Both would make the same difference
    • Neither would make a difference to me
  9. Q8

    What one change would most help you to volunteer with Riverside Food Network, or to volunteer more often?

    Open text
  10. Q9

    At which distribution site have you volunteered most in the last 12 months?

    One answer
    • [Site 1 name]
    • [Site 2 name]
    • [Site 3 name]
    • [Site 4 name]
    • [Site 5 name]
    • [Site 6 name]
    • [Site 7 name]
    • [Site 8 name]
    • [Site 9 name]
    • I have volunteered at more than one site about equally
    • I'm not sure
  11. Q10

    How old are you?

    One answer
    • Under 18
    • 18 to 24
    • 25 to 34
    • 35 to 49
    • 50 to 64
    • 65 to 74
    • 75 or over
    • Prefer not to say
Sample plan

BEFORE SENDING: Replace the bracketed placeholders in Q9 with the real names of the 9 sites, exactly as volunteers know them. This is the only wording in the survey that must change.

WHO IS INVITED: Everyone on the volunteer database with at least one shift in the last 12 months, about 1,410 people. That is 410 active volunteers (a shift in the last 3 months) and about 1,000 with no shift in the last 3 months. This is a census, so there is no sampling and no quotas.

  • Send the first invitation by both email and text message.
  • Send two reminders, about 4 and 9 days later.
  • Send extra reminders by text to lapsed volunteers, who are less likely to respond.
  • Use a unique link for each person. The link should carry their database status (active or lapsed), so that the S1 answer can be checked against the database and the results weighted.
  • Do not close the survey at 330 completes. Accept every response that comes in.

PRECISION: The trustees' target cannot be met. Results for each of the 9 sites cannot reach plus or minus 7 percentage points at 95% confidence with about 330 completes.

  • Each site has about 157 eligible volunteers on average.
  • Even after correcting for the small population, each site needs about 88 completes. That is about 790 in total, or a 56% response rate from everyone invited.
  • About 330 completes gives roughly 37 per site. That means a margin of about plus or minus 14 points per site.
  • For the whole sample, 330 completes gives about plus or minus 4.7 points. For the main comparison, stopped versus still volunteering (likely about 190 and 140), each group's margin is about plus or minus 6 to 8 points.

The trustees should agree one of these options before fieldwork starts: (a) accept about plus or minus 14 points per site, treating site results as directional; (b) report sites in 3 or 4 groups, such as by area or size; (c) fund extra effort to reach about 790 completes, such as site leads personally asking their volunteers to take part.

WEIGHTING: Monitor responses daily by site and by database status. Weight the final data by site and by active or lapsed status to match the database counts.

DATA QUALITY: Remove responses that:

  • finish in under 90 seconds;
  • fail S1 when checked against the database;
  • give implausible travel costs in Q4 (more than £50 per shift is checked by hand).

NEUTRALITY: The operations lead believes travel cost is the main reason volunteers leave. The survey is built to test this belief, not to confirm it.

  • Travel cost is one item among 11 in Q2 and Q3. The order of these items is randomised for each respondent, apart from 'Something else' and 'None of these', which stay at the end.
  • 'None of these' cannot be combined with other answers.
  • The order of Q5 and Q6 is randomised. The order of the first two options in Q7 is also randomised.
  • Both funding options are described in the same concrete way.
  • The invitation says only that the charity wants to understand volunteers' experiences. It does not mention travel or money.

The survey has 10 questions after the screener. It should take about 3 to 5 minutes.