Sign inTest your prompt

Benchmarks / SurveyBench / Why Riverside Food Network is losing volunteers / Claude Fable 5.2

Measured by Spring Prompt

Claude Fable 5.2: Why Riverside Food Network is losing volunteers

The decision: how to spend next year's £45,000 for volunteer support: reimburse volunteers' travel costs, fund a part-time volunteer coordinator with online shift booking, or do both at a smaller scale. The brief and the data →

Verdict
✗ Not sound
Research score
95 of 100
Analysis rating
1,683
Head to head, this task
won 9 of 10

Why it is not sound

The survey

  • A double-barrelled question: Q6

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What percentage of active volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓37.2%37.2%
What percentage of lapsed volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓40.2%40.2%
Is the difference between active and lapsed respondents in the share choosing 'The cost of travelling to the site' in Q2 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Weighted to the age mix of all 1,000 lapsed volunteers, what percentage of lapsed volunteers would choose 'The cost of travelling to the site' as a difficulty in Q2?✓47.4%47.4%
What percentage of lapsed respondents gave a scheduling reason as their main reason for stopping in Q3 (either of its first two options: shifts changed at short notice, or hard to arrange shifts)?✓42.9%42.9%
What is the 95% margin of error, in percentage points, on that unweighted share of lapsed respondents giving a scheduling reason as their main reason?✓9.2 points9.17 points
On average, how many difficulties did each respondent choose in Q2 (counting every option except 'None of these', which counts as zero), across all respondents?✓1.551.55
What percentage of lapsed respondents said they would be likely (fairly or very) to volunteer again if they could arrange their shifts online (Q4)?✓40.2%40.2%
How many of the open-text answers mention shift scheduling (booking or swapping shifts, or last-minute changes or cancellations)?✓2934
What percentage of the active volunteers invited completed the survey?✓52.4%52.4%
What percentage of the lapsed volunteers invited completed the survey?✓11.2%11.2%
If the lapsed respondents were representative, about how many of the 1,000 lapsed volunteers would give a scheduling reason (either of the first two Q3 options) as their main reason, using the unweighted share?✓429429

Traps in the data 8 of 8 handled

  • ✓Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
  • ✓The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.
  • ✓Q2 is multi-select: respondents chose 1.6 difficulties on average, so columns add up to more than 100%.
  • ✓52% of active volunteers responded but only 11% of lapsed ones, so the lapsed base is small (112) and those who replied may differ from the 89% who did not (for example, still feeling some attachment).
  • ✓Volunteers aged 65 or over are 43% of respondents but about 26% of volunteers in the database; younger volunteers cite travel cost more. Weighted to the lapsed age mix, travel cost among lapsed volunteers rises from 40% to 47%; a scheduling main reason goes from 43% to 41%, still well ahead of travel cost (21%).
  • ✓Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
  • ✓The travel-cost gap between lapsed and active respondents (3.0 points) is not significant at 95%.
  • ✓40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.

The analysis

Recommendation

Choose 'both, at a smaller scale', weighted heavily towards the rota. Put the large majority of the £45,000 (roughly four-fifths, subject to costing) into the part-time coordinator and online booking. Write the coordinator's remit around the bigger problem: a minimum notice period for shift changes, one reliable notification to every affected volunteer, and self-service booking and swapping. Ring-fence the remainder as a capped, needs-based travel fund open to any volunteer for whom cost is a barrier. Promote it actively to under-25s rather than restricting it to them, and run it as a pilot with uptake and retention tracked. Do not spend the whole budget on travel reimbursement: travel cost is no more common among leavers than stayers, and only about one in nine leavers give it as their main reason. Do not promise trustees a number of returners; test it with a real re-invitation. Start the no-cost fixes now: a notice rule for changes, and offering volunteers placement at their nearest site.

What would change it

I would shift money towards travel if a follow-up with lapsed non-responders, especially under-45s, showed travel cost rivalling scheduling as the main reason for leaving, or if the travel pilot retained or returned more volunteers per pound than the rota work. I would rethink the coordinator hire if (a) site-level data showed late changes concentrated in two or three sites, pointing to fixing those rotas rather than a network-wide post; (b) late changes proved to be driven by unpredictable donations or demand that a coordinator could not reduce; (c) the re-invitation test brought back very few lapsed volunteers after booking and notice improved; or (d) costing showed a coordinator plus tool cannot be delivered within the budget.

Findings

  1. Operations lead: no, the fact that travel cost is the most-ticked difficulty does not show the £45,000 should go on reimbursement. Travel cost is a widespread burden, but it does not separate leavers from stayers. Rota problems do: leavers report them three to five times as often as current volunteers.

    High confidenceBase 327

    Q2 travel cost: active 37.2% (80 of 215) vs lapsed 40.2% (45 of 112), a 3-point gap that is not statistically significant. Shifts changed at short notice: 14.4% active vs 51.8% lapsed. No easy way to arrange shifts: 11.6% vs 42.0%. Not told about changes in good time: 7.4% vs 41.1%. Among lapsed, all three rota items are ticked at least as often as travel cost. Travel cost tops the overall list (38.2%, 125 of 327) partly because the rota problem is split across three options.

  2. The rota, not travel, is the main reason people say they stopped. About four in ten leavers give a scheduling reason, more than half if 'not told about changes' is included, and about one in nine give travel cost.

    Medium confidenceBase 112

    Q3, lapsed n=112: shifts changed at short notice 29.5% (33) plus hard to arrange 13.4% (15) = 42.9% (48), margin ±9.2 points (roughly 34% to 52%). Adding 'not told in good time' (12.5%, 14) gives 55.4% (62). Travel cost 11.6% (13), travel time 0.9% (1). Changed circumstances 23.2% (26), who neither option would win back. Weighted to the true age mix of lapsed volunteers, scheduling stays at about 41% and travel cost rises to about 21%, though that rise rests on 8 under-25s. Scheduling leads either way.

  3. Online booking on its own would miss most of the rota problem. Three-quarters of rota-related leavers left because shifts were changed late or they were not told, not because booking was awkward. The coordinator's value lies in notice standards and reliable notification as much as in the booking tool.

    Medium confidenceBase 112

    Of the 62 leavers giving a rota-related main reason, 47 cite short-notice changes (33) or not being told (14); 15 cite difficulty arranging shifts. Open text: 29 of 109 answers are about scheduling (20 booking or swapping, 9 last-minute changes) and 8 more are only about not being told. Among the 40 lapsed open-text answers, 24 are about the rota or communication and 2 about travel cost.

  4. Chair of trustees: the survey cannot give a reliable number who would come back. 40% of lapsed respondents say they would be likely to return with online booking, but that is a ceiling on a small, self-selected group, not a forecast of 400 returners. Plan on tens, not hundreds, until a real re-invitation is tested.

    Low confidenceBase 112

    Q4: 40.2% likely (45 of 112, margin about ±9 points); only 17.0% (19) 'very likely'. Among those who left for scheduling reasons, 60.4% (29 of 48); among other reasons, 28.0% (14 of 50). Q5 travel reimbursement: 26.8% likely (30 of 112). Only 11.2% of lapsed volunteers responded, and those who reply are probably the most attached. Stated intentions about a hypothetical usually overstate behaviour. Q4 tested online booking only, not a coordinator or better notice.

  5. Volunteer manager: the under-25 pattern is striking but rests on 11 people, too few to justify a standing age-restricted scheme. It is enough to justify a small, capped, needs-based travel pilot that under-25s are actively told about.

    Low confidenceBase 11

    Lapsed under-25s: 8 of 11 give travel cost as their main reason (72.7%), against 5 of 101 older leavers. That is too large a gap to be chance among respondents, but the 11 are about 5% of roughly 220 lapsed under-25s. All under-25 respondents: 20 of 27 tick travel cost (74.1%). In absolute numbers most people reporting travel cost are older: 39 aged 45 to 64 and 46 aged 65 or over. Two of the three lapsed under-25 comments (V008, V228) are about distance and journey time, which reimbursement does not fix.

  6. Travel cost is a real fairness issue for current volunteers, and distance may be partly fixable at no cost. It is the top difficulty among active volunteers, and several ask for a nearer site.

    Medium confidenceBase 215

    Active Q2: travel cost 37.2% (80 of 215) is their most-ticked item; 28.4% tick none. Open text: 22 answers mention travel cost, 20 of them from active volunteers, with costs quoted from £2 each way to £12 a trip. 9 mention distance or journey time (e.g. V304 asks to be placed at a closer site).

  7. The sample under-represents exactly the groups whose answers differ most. Lapsed and younger volunteers responded far less, so totals lean towards older, current volunteers.

    High confidenceBase 327

    Response: active 52.4% (215 of 410), lapsed 11.2% (112 of 1,000). Under-25s are 8.3% of respondents against 18% of active and 22% of lapsed volunteers; 65 or over are 43.1% against 30% and 25%. Weighting lapsed answers to the true age mix moves travel cost as a difficulty from 40.2% to 47.4%.

Caveats

  • Only 11.2% of lapsed volunteers responded (112 of 1,000). The projection that about 429 of the 1,000 left for scheduling reasons holds only if respondents are typical; the sampling range alone is roughly 340 to 520, and non-response bias is on top of that.
  • Under-25s are badly under-represented (27 respondents, 11 of them lapsed). Every under-25 figure is indicative only, and age-weighting multiplies the influence of those 11 people.
  • Q4 and Q5 are stated intentions about hypotheticals, asked one after the other. They show which offer appeals more to whom, not how many will return. Neither question tested a coordinator, better notice or a nearer site.
  • Q2 splits the rota problem across three options and travel across two, so 'most-ticked single option' is a misleading way to rank problems. Respondents ticked 1.55 difficulties on average.
  • Reasons for leaving are self-reported after the fact, so the active/lapsed differences show association, not proof of cause. Active volunteers are by definition those who have coped with the travel cost so far.
  • Open-text counts come from 109 answers and show what people raise unprompted, not prevalence. The scheduling count of 29 excludes 8 answers that are only about not being told of changes; counted together they make 37.
  • There is no breakdown of difficulties by site, although each site lead runs their own rota. We cannot tell whether the problem is network-wide or concentrated in a few sites.
  • Neither option has been costed here. The survey says nothing about what full travel reimbursement or a coordinator plus booking tool would actually cost.

Next steps

  • decisionApprove the split: most of the budget to a part-time coordinator and online booking, with a remit covering notice standards and change notifications, and a capped, needs-based travel fund from the remainder. Cost both parts before fixing the amounts.
  • decisionIntroduce now, at no cost, a minimum notice rule for shift changes and a single direct message to every affected volunteer, across all 9 sites. Offer volunteers a move to their nearest site.
  • experimentRoll out online booking and notifications to 2 or 3 sites first, and compare them with the remaining sites before full roll-out.
  • experimentOnce booking and notice improvements are live, re-invite lapsed volunteers with a message describing the change, and count how many book a shift within 8 weeks.
  • experimentRun the travel fund as a 6-month capped pilot, needs-based and open to all ages. Record claims by age band, mode of travel and cost per shift.
  • researchPhone 10 to 15 lapsed under-25s and a sample of lapsed non-responders with three short questions on why they stopped. Re-cut the existing data by site.
  • monitorHave site leads log every shift change with the notice given, track the lapse rate by site and age band quarterly, and add a one-question exit reason when a volunteer stops.

Open-text themes it coded

Shift scheduling: online booking or swapping, and last-minute changes or cancellations 29Not being told about changes directly and in good time 12Travel cost (fares, fuel, parking charges) 22Distance and journey time; wanting a nearer site 9Parking space and physical access at the site 2Recognition and training 3Nothing needed; fine as it is 38

The survey it planned

1 screening question and 10 questions, as the model wrote them.

  1. S1

    When did you last do a volunteer shift with Riverside Food Network?

    One answer
    • In the last 3 months
    • Between 3 and 12 months ago
    • More than 12 months ago
    • I have never volunteered with Riverside Food Network

    Continues if In the last 3 months; Between 3 and 12 months ago

  2. Q1

    Which of these best describes you at the moment?

    One answer
    • I still volunteer with Riverside Food Network
    • I am not volunteering at the moment, but I expect to come back
    • I have stopped volunteering with Riverside Food Network
  3. Q2

    In your own words, what was the main reason you stopped or paused volunteering with Riverside Food Network?

    Open text

    Routing Shown only if Q1 is: I am not volunteering at the moment, but I expect to come back; I have stopped volunteering with Riverside Food Network

  4. Q3

    Thinking about your volunteering with Riverside Food Network in the last 12 months, which of these, if any, were a problem for you? Please select all that apply.

    Any that apply
    • The cost of getting to and from the site (fares, fuel or parking)
    • How long the journey took, or how awkward it was
    • Booking, swapping or cancelling shifts
    • Shift days or times that did not fit around my other commitments
    • Shifts being changed or cancelled at short notice
    • The physical demands of the work (such as lifting or standing)
    • The emotional demands of the work
    • Not knowing what I was expected to do, or not enough training
    • Not feeling valued or thanked
    • Relationships with other volunteers or the site lead
    • Shifts that were disorganised, too busy or too quiet
    • Something else
    • None of these was a problem for me
  5. Q4

    Which of these, if any, were reasons you stopped or paused volunteering? Please select all that apply.

    Any that apply
    • My work or study hours changed
    • My own health
    • Caring for children or relatives
    • I moved home
    • I only ever planned to volunteer for a short time
    • I started volunteering or helping somewhere else instead
    • I could not afford the cost of getting to the site
    • The journey took too long or was too awkward
    • It was too hard to book or swap shifts
    • The shift days or times on offer stopped suiting me
    • I stopped hearing about shifts or lost touch with the site
    • I did not feel valued or supported
    • The work was too physically or emotionally demanding
    • It was not what I expected, or I did not enjoy it
    • Another reason
    • Prefer not to say

    Routing Shown only if Q1 is: I am not volunteering at the moment, but I expect to come back; I have stopped volunteering with Riverside Food Network

  6. Q5

    On a typical day when you volunteered, about how much did it cost you in total to get to the site and home again? Please include any bus or train fares, fuel and parking.

    One answer
    • Nothing (for example, I walked, cycled, got a lift or used a free travel pass)
    • Less than £3
    • £3 to £5.99
    • £6 to £9.99
    • £10 or more
    • Not sure
  7. Q6

    In the last 12 months, how easy or difficult was it to book, swap or cancel shifts at your site?

    One answer
    • Very easy
    • Fairly easy
    • Neither easy nor difficult
    • Fairly difficult
    • Very difficult
    • Don't know or can't remember
    Judges: double-barrelled

    Requires one ease rating for booking, swapping and cancelling shifts, although these processes could differ substantially in difficulty.

    Booking, swapping, and cancelling are distinct processes; one might be easy while another is difficult.

  8. Q7

    Suppose Riverside Food Network paid back what you spent on getting to and from each shift (bus fares, fuel or parking). How likely or unlikely would you be to start volunteering again in the next 6 months?

    Scale

    Scale 1-5: Very unlikely to Very likely

    Routing Shown only if Q1 is: I am not volunteering at the moment, but I expect to come back; I have stopped volunteering with Riverside Food Network

  9. Q8

    Suppose you could see available shifts and book, swap or cancel them online at any time, instead of phoning or messaging your site lead. How likely or unlikely would you be to start volunteering again in the next 6 months?

    Scale

    Scale 1-5: Very unlikely to Very likely

    Routing Shown only if Q1 is: I am not volunteering at the moment, but I expect to come back; I have stopped volunteering with Riverside Food Network

  10. Q9

    Which Riverside Food Network distribution site did you volunteer at most in the last 12 months? Please type the name of the site or the area it is in.

    Open text
  11. Q10

    Which age group are you in?

    One answer
    • Under 25
    • 25 to 34
    • 35 to 44
    • 45 to 54
    • 55 to 64
    • 65 to 74
    • 75 or over
    • Prefer not to say
Sample plan

WHO IS INVITED: Everyone on the volunteer database with at least one shift in the last 12 months, about 1,410 people (410 active, about 1,000 with no shift in the last 3 months). This is a census, not a sample, because the population is too small to sample from. Invitations go by email and text from the charity centrally, not from site leads, so people can be candid about their site. The survey is anonymous. The invitation describes it as 'your experience of volunteering with us' and mentions neither travel costs nor online booking, so it does not attract or prime people on either option.

TARGET: About 330 completes, a response rate of roughly 23%. People who have stopped will respond less readily than current volunteers, and they are the group O1 and O3 depend on. There are no hard quotas, since nobody should be turned away in a census. Soft minimums, monitored daily: at least 165 completes from people who are paused or stopped (Q1), at least 120 from current volunteers, and at least 25 per site. Fieldwork runs about 3 weeks with two reminders (email, then text). Reminders are targeted at lapsed volunteers and at sites that are behind.

PRECISION, AND WHAT CANNOT BE DELIVERED: The trustees' requirement of plus or minus 7 points for each of the 9 sites cannot be met with 330 completes.

  • Required: plus or minus 7 points per site needs about 196 completes per site in a large population. Allowing for each site having only about 157 people on the list, it still needs about 87 per site, roughly 785 in total. That is a 56% response rate from everyone invited, which is unrealistic for a group that is 70% lapsed.
  • Achievable: 330 completes gives about 37 per site on average, which is roughly plus or minus 14 to 16 points per site, and worse at smaller sites.
  • What 330 does support: plus or minus 5 points for all respondents; about plus or minus 7 for lapsed volunteers as a group; about plus or minus 8 for current volunteers.
  • Recommendation to trustees: treat single-site figures as indicative, and act only on site differences of 20 points or more. For firmer geographic results, report sites in 3 groups of about 110 completes each (about plus or minus 9 points). Site-by-status cells (about 18 lapsed per site) will not be reported as percentages.

WEIGHTING: Results are weighted back to the database split of 410 active to 1,000 lapsed, using S1 (last shift within 3 months versus 3 to 12 months ago), and to site totals. Unweighted bases are shown throughout.

KNOWN BIAS: Lapsed volunteers who answer are likely to be warmer towards the charity than those who do not, so return-intention figures are upper bounds.

QUESTIONNAIRE SET-UP:

  • Q3 and Q4 options appear in random order, with 'Something else', 'Another reason', 'None of these' and 'Prefer not to say' fixed at the bottom.
  • The order of Q7 and Q8 is rotated at random between respondents.
  • Q9 is open text because the site names were not in the brief. Answers are coded to the 9 sites. It can be swapped for a pick-list of the nine names before launch.
  • No attention check is included: all 10 permitted questions are needed for the objectives, and the survey takes about 4 to 5 minutes.