Benchmarks / SurveyBench / Why Riverside Food Network is losing volunteers / Claude Haiku 5.5

Measured by Spring Prompt

Claude Haiku 5.5: Why Riverside Food Network is losing volunteers

The decision: how to spend next year's £45,000 for volunteer support: reimburse volunteers' travel costs, fund a part-time volunteer coordinator with online shift booking, or do both at a smaller scale. The brief and the data →

Verdict
✓ Sound research
Research score
93 of 100
Analysis rating
1,412
Head to head, this task
won 7 of 9

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What percentage of active volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓37.2%37.2%
What percentage of lapsed volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓40.2%40.2%
Is the difference between active and lapsed respondents in the share choosing 'The cost of travelling to the site' in Q2 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Weighted to the age mix of all 1,000 lapsed volunteers, what percentage of lapsed volunteers would choose 'The cost of travelling to the site' as a difficulty in Q2?✓47.4%47.4%
What percentage of lapsed respondents gave a scheduling reason as their main reason for stopping in Q3 (either of its first two options: shifts changed at short notice, or hard to arrange shifts)?✓42.9%42.9%
What is the 95% margin of error, in percentage points, on that unweighted share of lapsed respondents giving a scheduling reason as their main reason?✓9.2 points9.17 points
On average, how many difficulties did each respondent choose in Q2 (counting every option except 'None of these', which counts as zero), across all respondents?✓1.551.55
What percentage of lapsed respondents said they would be likely (fairly or very) to volunteer again if they could arrange their shifts online (Q4)?✓40.2%40.2%
How many of the open-text answers mention shift scheduling (booking or swapping shifts, or last-minute changes or cancellations)?✓3734
What percentage of the active volunteers invited completed the survey?✓52.4%52.4%
What percentage of the lapsed volunteers invited completed the survey?✓11.2%11.2%
If the lapsed respondents were representative, about how many of the 1,000 lapsed volunteers would give a scheduling reason (either of the first two Q3 options) as their main reason, using the unweighted share?✓429429

Traps in the data 8 of 8 handled

  • ✓Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
  • ✓The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.
  • ✓Q2 is multi-select: respondents chose 1.6 difficulties on average, so columns add up to more than 100%.
  • ✓52% of active volunteers responded but only 11% of lapsed ones, so the lapsed base is small (112) and those who replied may differ from the 89% who did not (for example, still feeling some attachment).
  • ✓Volunteers aged 65 or over are 43% of respondents but about 26% of volunteers in the database; younger volunteers cite travel cost more. Weighted to the lapsed age mix, travel cost among lapsed volunteers rises from 40% to 47%; a scheduling main reason goes from 43% to 41%, still well ahead of travel cost (21%).
  • ✓Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
  • ✓The travel-cost gap between lapsed and active respondents (3.0 points) is not significant at 95%.
  • ✓40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.

The analysis

Recommendation

Do not put the whole £45,000 into travel reimbursement. The evidence points first to scheduling. Lapsed volunteers mainly left over changed or hard-to-manage shifts, and they say online booking and reliable notice would bring them back more often than travel repayment. Recommended split: about £30,000–£35,000 on a part-time volunteer coordinator who sets up online booking, swapping and a fixed notice rule for rota changes (for example, changes communicated at least 48 hours ahead, by text or email to everyone on the shift). Hold about £10,000–£15,000 for a targeted, time-limited travel pilot for under-25s and long-distance volunteers, with a comparison group, and decide whether to extend it once there is evidence. This is an indicative split; the coordinator and booking costs must be confirmed before the figures are final. Answers to the stakeholders: (1) Operations lead: travel is the most-cited difficulty overall, but for lapsed volunteers it is not the main reason, so it should not take the whole budget. (2) Chair: the stated intent suggests up to about 400 of 1,000 lapsed volunteers could return with online booking, most likely far fewer in practice; no cost-per-return can be stated yet. (3) Volunteer manager: the under-25 travel signal is real but rests on 11 people, so run a targeted pilot rather than a permanent scheme.

What would change it

(a) A larger follow-up of lapsed volunteers (including non-respondents) showing travel as the main reason for leaving for a large share of the 1,000, especially under-25s and long-distance volunteers. (b) Evidence from the notice pilot that fixing communication does not raise return rates, while a travel pilot does. (c) Costing showing the coordinator role cannot be delivered at this budget with online booking, which would favour a simpler booking tool plus a smaller travel scheme. (d) Expense data showing the real per-trip cost is small, which would reduce the case for reimbursement. (e) Confirmation that the online intent answers do not hold once tested with real volunteers.

Findings

  1. Scheduling, not travel, is the main reason lapsed volunteers stopped. Only 11.6% gave travel as their main reason, against 42.9% who gave a scheduling reason (shifts changed at short notice 29.5%, hard to arrange shifts 13.4%). Operations lead: travel is the most-cited difficulty overall, but it is not the main reason people left.

    Medium confidenceBase 112

    Q3 lapsed (base 112): scheduling 29.5% + 13.4% = 42.9%; travel 11.6%; circumstances changed 23.2%. Q2 lapsed: shifts changed at short notice 51.8%, travel 40.2%, no easy way to arrange shifts 42.0%, not told about changes 41.1%. The 95% margin on the 42.9% is about ±9 points (about ±8.6 with a finite-population correction). Response from lapsed volunteers was 11.2%, so treat the exact share with caution.

  2. Travel cost is the most-cited difficulty across all respondents (38.2%) and is concentrated among under-25s (74.1%), but it is not a stronger driver of lapsing than scheduling for the lapsed group as a whole. Reweighted to the lapsed population's age mix, travel rises to 47.4%, because under-25s are under-represented in the sample. That raises the case for targeted travel help but does not make it the main lever overall.

    Medium confidenceBase 327

    Q2 total (base 327): travel 38.2%. Q2 by age: under 25 74.1% (base 27), 25 to 44 40.8%, 45 to 64 35.5%, 65 or over 32.6%. Lapsed by age (base 11/21/36/44): travel 81.8%, 42.9%, 47.2%, 22.7%. Weighted to population age mix (22/30/23/25): 47.4%. Under-25 lapsed base is 11.

  3. There is no evidence that active and lapsed volunteers differ on travel cost (37.2% vs 40.2%). The gap is not statistically significant (two-proportion z of about 0.5, p of about 0.6). Travel cost therefore does not explain who left; scheduling does.

    High confidenceBase 327

    Q2 total: active (base 215) 37.2%, lapsed (base 112) 40.2%. Difference 3.0 points; pooled SE about 5.7 points; z about 0.53. By contrast, shifts changed at short notice is 14.4% active vs 51.8% lapsed, and not being told about changes is 7.4% vs 41.1%, both large and significant.

  4. Online booking and swapping is the lever most likely to bring lapsed volunteers back. Lapsed volunteers say they would be likely to return if they could arrange shifts online (40.2%) more often than if travel costs were repaid (26.8%). The split is sharp by reason for leaving: scheduling-reason leavers are 60.4% likely with online booking vs 27.1% with travel repayment; travel-reason leavers reverse this (14.3% vs 57.1%), but that group is only 14.

    Low confidenceBase 112

    Q4 lapsed (base 112): net likely 40.2%. Q5 lapsed (base 112): net likely 26.8%. By main reason, Q4 scheduling (base 48) 60.4%, travel (base 14) 14.3%, other (base 50) 28.0%. Q5 scheduling 27.1%, travel 57.1%, other 18.0%. These are stated intentions to a hypothetical, not behaviour.

  5. Chair of trustees: online booking could bring some lost volunteers back, but the figure is an upper bound. Applied to the 1,000 lapsed volunteers, 40.2% stated intent would be about 400 people. Stated intent to a survey, from the 11.2% who responded, typically overstates behaviour. The scheduling-reason group (about 429 people in the population) is the most relevant segment, at about 60% stated intent, or roughly 260 people on the same basis. No cost-per-return figure can be given until the coordinator and booking costs are known.

    Low confidenceBase 112

    Q4 lapsed net likely 40.2% (base 112); scheduling-reason subgroup 60.4% (base 48); scheduling share 42.9% of 1,000 = about 429. Lapsed response rate 11.2% (112 of 1,000), with likely over-representation of the more engaged.

  6. Volunteer manager: a travel scheme restricted to under-25s is not yet justified. The under-25 signal is strong but rests on 11 lapsed respondents. Eight of the 11 gave travel as their main reason for leaving. Under-25s make up only 8.3% of the survey and 18 to 22% of the database. Before committing money, test a travel offer for under-25s and for long-distance volunteers, and check the travel pattern with a larger sample.

    Low confidenceBase 11

    Q3 lapsed under 25 (base 11): travel 72.7%, shifts changed at short notice 18.2%, not told about changes 0.0%. Q2 under 25 (base 27): travel 74.1%. Open text under 25 (base 8 answers): 4 mention fares, fuel or distance (V008, V047, V081, V182), one parking (V060), one distance (V228).

  7. The most common complaints in open text are about notice of changes and the lack of online booking, not travel. Of 109 open-text answers, 37 mention scheduling (changes, cancellations, booking or swapping). Travel cost is raised in 15, and parking in 6. Around 40 answers say nothing needs changing.

    Medium confidenceBase 109

    Q1 open text (109 answers, mostly from active volunteers). Theme counts: scheduling 37 (last-minute changes/notice 19; online booking/swapping 20; 2 in both). Travel cost 15 (V047, V053, V070, V075, V081, V138, V182, V186, V189, V222, V234, V236, V237, V299, V316, V323, plus V062 partly). Distance/journey 9. Parking 6. Recognition 2. Training 1.

  8. Lapsed volunteers are much more likely than active ones to report poor notice of changes and no easy way to manage shifts. This is a communication and process gap that can be fixed without spending on travel. The gap is large and consistent across age groups.

    High confidenceBase 112

    Q2 lapsed vs active: shifts changed at short notice 51.8% vs 14.4%; not told about changes in good time 41.1% vs 7.4%; no easy way to arrange shifts 42.0% vs 11.6%. Lapsed by age, 'not told in good time' is 54.5% (under 25), 52.4% (25 to 44), 30.6% (45 to 64), 40.9% (65 or over).

  9. Sample bias limits what can be generalised. Active volunteers responded at 52.4%, lapsed at 11.2%. Respondents skew older (65 or over 43.1% of the sample vs 25 to 30% of the database) and under-25s are under-represented. Non-respondent lapsed volunteers (888) may differ, for example in how many left because of travel or scheduling.

    High confidenceBase 327

    Response rates: active 215/410 = 52.4%; lapsed 112/1,000 = 11.2%. Q7 age: under 25 8.3%, 25 to 44 15.0%, 45 to 64 33.6%, 65 or over 43.1% (total sample). Database: active under 25 18%, 65 or over 30%; lapsed under 25 22%, 65 or over 25%.

  10. Parking is a cost and access issue at specific sites, mainly Northgate, Eastfield and Hollins Park. It is raised in open text but is a low-volume issue overall.

    Low confidenceBase 109

    Q2 physical access (parking, steps, doors) total 7.0% (base 327). Open text: 6 mentions (V034, V060, V095, V118, V158, V289), concentrated on named sites (Northgate, Eastfield, Hollins Park, Westbury).

Caveats

  • Stated intentions (Q4, Q5) are hypothetical. They overstate what volunteers will do, and the travel-repayment and online-booking options were not costed or tested.
  • Lapsed volunteers are only 11.2% of the population, so Q3–Q5 rest on 112 people. Subgroups are very small: 48 scheduling-reason, 14 travel-reason, 11 under-25s.
  • The Q4 and Q5 'travel' subgroup (n=14) is too small to support a firm conclusion about which fix suits travel-motivated leavers.
  • Q2 is multi-select, so the percentages do not sum to 100 and do not show which difficulty was most important. Q3 is the better guide to main reasons, but it is asked of lapsed volunteers only.
  • The population breakdown in the age-weighted estimate (47.4%) is for lapsed volunteers only and rests on an 11-person under-25 cell, so it is unstable.
  • The survey measured difficulties and reasons, not the cost of each fix or how many volunteers each would retain. The decision needs costing before a return can be estimated.
  • The open-text answers are self-selected and about 40 of 109 say nothing needs changing. They illustrate themes; they do not measure prevalence. Several respondents appear in more than one theme.
  • The margin of error (±9.2 points) assumes a simple random sample. Lapsed non-response means it may understate uncertainty.

Next steps

  • decisionApprove a split of the budget: about £30–35k for a part-time coordinator and online shift booking with swapping, and about £10–15k for a targeted travel pilot. Confirm costs before committing.
  • experimentIntroduce a rota-change rule now (no-cost): every change or cancellation goes to all shift volunteers by text or email, at least 48 hours ahead, and the site lead logs it. Start at two or three sites and compare with the rest.
  • experimentRun a 6-month travel pilot for under-25s and volunteers travelling more than a set distance (for example, a fixed monthly fare or fuel allowance), with a matched comparison group and clear eligibility.
  • researchRe-contact the 1,000 lapsed volunteers (including non-responders) with a short follow-up by phone or text, offering a shorter survey and a prize or thank-you to raise response, and ask about the main reason for leaving and their travel distance and cost.
  • researchInterview around 10 lapsed under-25s and long-distance volunteers, plus several lapsed 45–64s, on the cost, time and reliability of getting to sites.
  • monitorTrack monthly: active shift numbers, cancellations and changes with short notice, sign-ups per site, return of lapsed volunteers, and pulse-check items on notice and booking, starting now as a baseline.

Open-text themes it coded

Travel cost (fares, fuel, parking-free travel, car sharing) 17Distance and journey time to the site 9Parking cost and access 6Last-minute changes and poor notice 19Online booking, swapping and self-service 20Recognition and feeling valued 2Training before first shift 1Nothing needed or already fine 38

The survey it planned

2 screening questions and 10 questions, as the model wrote them.

  1. S1

    In the last 12 months, have you done at least one shift with Riverside Food Network?

    One answer
    • Yes
    • No

    Continues if Yes

  2. S2

    In the last 3 months, have you done at least one shift with Riverside Food Network?

    One answer
    • Yes
    • No

    Continues if Yes; No

  3. Q1

    In the last 12 months, which of these, if any, have made it harder for you to volunteer with Riverside Food Network? Select all that apply.

    Any that apply
    • The cost of bus fares or other travel to the site
    • The cost of parking
    • The time it takes to travel to the site
    • Shift times clashing with work, study or caring responsibilities
    • Difficulty finding out which shifts are available
    • Difficulty booking, swapping or cancelling a shift by phone or message
    • Site opening hours or changes to shifts
    • Not enough other volunteers to work with
    • Not being clear what I was expected to do
    • The physical demands of the work
    • Moving home or moving away from the site
    • Something else
    • None of these
  4. Q2

    Roughly how much do you spend getting to and from your site for one shift, including the return journey? If you walk, cycle or get a lift, choose the first option.

    One answer
    • £0 (I walk, cycle or get a lift)
    • Under £2
    • £2 to under £4
    • £4 to under £6
    • £6 or more
    • I'm not sure
  5. Q3

    What was the main reason you stopped volunteering with Riverside Food Network?

    One answer
    • The cost of getting to the site (fares or parking)
    • The time it takes to get to the site
    • Shift times did not suit me
    • I could not book, swap or cancel shifts easily
    • Site opening hours were cut or shifts changed
    • Not enough other volunteers to work with
    • Health, caring or other personal commitments
    • I moved away
    • I did not enjoy the work or the team
    • Something else

    Routing Shown only if S2 is: No

  6. Q4

    If Riverside Food Network refunded your travel costs (bus fares and parking) for each shift, how likely would you be to volunteer with them again?

    Scale

    Scale 1-5: Very unlikely to Very likely

    Routing Shown only if S2 is: No

  7. Q5

    If you could see available shifts and book, swap or cancel them online, how likely would you be to volunteer with Riverside Food Network again?

    Scale

    Scale 1-5: Very unlikely to Very likely

    Routing Shown only if S2 is: No

  8. Q6

    How do you usually book, swap or cancel a shift?

    One answer
    • By phoning my site lead
    • By text or messaging my site lead
    • In person at the site
    • Online
    • Other
    • I have not needed to book, swap or cancel a shift
  9. Q7

    What, if anything, most helps you keep volunteering with Riverside Food Network?

    One answer
    • Travel to my site is easy or affordable
    • Shift times suit me
    • Booking, swapping or cancelling shifts is easy
    • The people I volunteer with
    • The work itself
    • My site is open for enough hours
    • Something else

    Routing Shown only if S2 is: Yes

  10. Q8

    What is your age?

    One answer
    • Under 25
    • 25 to 34
    • 35 to 49
    • 50 to 64
    • 65 or over
    • Prefer not to say
  11. Q9

    Which distribution site did you most often volunteer at in the last 12 months?

    One answer
    • Site 1: [insert site name]
    • Site 2: [insert site name]
    • Site 3: [insert site name]
    • Site 4: [insert site name]
    • Site 5: [insert site name]
    • Site 6: [insert site name]
    • Site 7: [insert site name]
    • Site 8: [insert site name]
    • Site 9: [insert site name]
    • I'm not sure
  12. Q10

    Is there anything else you would like us to know about your experience of volunteering with Riverside Food Network? This is optional.

    Open text
Sample plan

Invite by email and text every person on the volunteer database with at least one shift in the last 12 months: about 410 currently active volunteers (shift in the last 3 months) and about 1,000 who have not had a shift in the last 3 months, about 1,410 in total. Screen with S1 (must be Yes) and classify with S2 (Yes = current, No = lapsed). Target 330 completes. Expected response is roughly 35-40% for current volunteers and 15-20% for lapsed, so set quotas of about 150 current and 180 lapsed, and send one reminder after 7 days with the survey open for 2 weeks. Quota the 9 sites in proportion to each site's share of the frame, with a minimum of about 30 completes per site. Insert the 9 real site names into Q9 before launch. Randomise the order of the options in Q1, Q3 and Q6-Q7 (keeping 'Something else', 'Other' and 'None of these' last) and do not reveal the travel or booking hypotheses in the invitation. PRECISION WARNING: ±7 percentage points at 95% confidence per site needs about 196 completes per site (worst case, p = 0.5), roughly 1,760 completes in total. That is more than the entire invited pool (1,410) even at 100% response, so the trustees' requirement cannot be met for all 9 sites. With 330 completes spread across 9 sites (about 37 per site), each site estimate will be within about ±16 percentage points, and the overall estimate within about ±5.4 points. Trustees should either accept ±16 points at site level and rely on the overall and status-level findings for the decision, or extend fieldwork or budget for a larger sample if site-level precision is essential. Weight the final data to the known frame by status and site to correct for differential response.