Benchmarks / SurveyBench / Why Riverside Food Network is losing volunteers / GPT-6.1 Terra

Measured by Spring Prompt

GPT-6.1 Terra: Why Riverside Food Network is losing volunteers

The decision: how to spend next year's £45,000 for volunteer support: reimburse volunteers' travel costs, fund a part-time volunteer coordinator with online shift booking, or do both at a smaller scale. The brief and the data →

Verdict
✗ Not sound
Research score
86 of 100
Analysis rating
1,273
Head to head, this task
won 3 of 10

Why it is not sound

The survey

  • Answer bands that overlap or leave gaps

The analysis

  • Missed a trap: Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
  • Missed a trap: The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What percentage of active volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓37.2%37.2%
What percentage of lapsed volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓40.2%40.2%
Is the difference between active and lapsed respondents in the share choosing 'The cost of travelling to the site' in Q2 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Weighted to the age mix of all 1,000 lapsed volunteers, what percentage of lapsed volunteers would choose 'The cost of travelling to the site' as a difficulty in Q2?✓47.4%47.4%
What percentage of lapsed respondents gave a scheduling reason as their main reason for stopping in Q3 (either of its first two options: shifts changed at short notice, or hard to arrange shifts)?✓42.9%42.9%
What is the 95% margin of error, in percentage points, on that unweighted share of lapsed respondents giving a scheduling reason as their main reason?✓9.2 points9.17 points
On average, how many difficulties did each respondent choose in Q2 (counting every option except 'None of these', which counts as zero), across all respondents?✓1.551.55
What percentage of lapsed respondents said they would be likely (fairly or very) to volunteer again if they could arrange their shifts online (Q4)?✓40.2%40.2%
How many of the open-text answers mention shift scheduling (booking or swapping shifts, or last-minute changes or cancellations)?✓3634
What percentage of the active volunteers invited completed the survey?✓52.4%52.4%
What percentage of the lapsed volunteers invited completed the survey?✓11.2%11.2%
If the lapsed respondents were representative, about how many of the 1,000 lapsed volunteers would give a scheduling reason (either of the first two Q3 options) as their main reason, using the unweighted share?✓429429

Traps in the data 6 of 8 handled

  • ✗Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
  • ✗The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.
  • ✓Q2 is multi-select: respondents chose 1.6 difficulties on average, so columns add up to more than 100%.
  • ✓52% of active volunteers responded but only 11% of lapsed ones, so the lapsed base is small (112) and those who replied may differ from the 89% who did not (for example, still feeling some attachment).
  • ✓Volunteers aged 65 or over are 43% of respondents but about 26% of volunteers in the database; younger volunteers cite travel cost more. Weighted to the lapsed age mix, travel cost among lapsed volunteers rises from 40% to 47%; a scheduling main reason goes from 43% to 41%, still well ahead of travel cost (21%).
  • ✓Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
  • ✓The travel-cost gap between lapsed and active respondents (3.0 points) is not significant at 95%.
  • ✓40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.

The analysis

Recommendation

Provisionally pursue both at a smaller scale, prioritising coordinator capacity to stabilise rotas and communicate changes, alongside a capped travel-support pilot open on a needs basis rather than age alone. Test a simple online booking and swap option, while retaining phone or message access. Do not commit a final budget split until staffing, technology and likely reimbursement costs are known; reserve funds for adjustments after the pilot.

What would change it

A credible pilot showing that travel reimbursement produces substantially more additional retained or returning volunteer shifts per pound would support shifting more funds to travel. Conversely, a coordinator-led rota and booking service producing greater incremental benefit would support expanding that option. Evidence of low uptake, unaffordable costs, exclusion of volunteers, or demand concentrated in a clearly identifiable group would also change the allocation or eligibility design.

Findings

  1. Travel costs warrant support, but their frequency as a difficulty does not justify spending the entire budget on reimbursement.

    Medium confidenceBase 327

    Travel cost was selected by 125 of 327 respondents: 80 of 215 active respondents (37.2%) and 45 of 112 lapsed respondents (40.2%). The approximately 3-percentage-point difference is not statistically significant at the 95% level. Q2 allowed multiple difficulties.

  2. Prioritise improving rota reliability and communication as well as making booking easier.

    Medium confidenceBase 112

    Of 112 lapsed respondents, 33 cited shifts changing at short notice and 15 difficulty arranging shifts as their main reason for stopping: 48 of 112, or 42.9%. A further 14 cited not being told about changes in good time. The approximate 95% margin of error on the first two reasons combined is 9.2 percentage points, before non-response bias.

  3. The survey cannot tell the chair how many lost volunteers online booking will bring back.

    Medium confidenceBase 112

    45 of 112 lapsed respondents (40.2%) said they were fairly or very likely to return if they could arrange shifts online. This is stated intention, not observed return or an estimate of the effect of introducing booking. Among the 48 citing the first two scheduling reasons, 29 (60.4%) said they were likely to return.

  4. Investigate travel support for younger volunteers, but do not restrict a scheme to under-25s on this evidence alone.

    Low confidenceBase 112

    Nine of 11 lapsed under-25 respondents selected travel cost as a difficulty, and eight of 11 gave it as their main reason for stopping. The base is very small. Travel cost was also selected by 36 of the other 101 lapsed respondents.

  5. Age weighting strengthens the case for investigating travel costs, but does not establish the true population rate.

    Low confidenceBase 112

    Weighting the four lapsed age groups to the database mix gives an estimated 47.4% selecting travel cost, compared with 40.2% unweighted. The estimate relies on only 11 lapsed under-25 respondents and cannot correct other differences between respondents and non-respondents.

  6. Test different forms of support rather than assuming either intervention addresses every barrier.

    Medium confidenceBase 112

    Of 112 lapsed respondents, 45 (40.2%) said they were likely to return with online booking and 30 (26.8%) with travel reimbursement. Among 14 citing travel cost or journey time as their main reason, eight said they were likely to return with reimbursement; this subgroup is small. The tables do not show overlap between the two answers.

Caveats

  • The tables are unweighted and based on completes. Completion rates were 52.4% of 410 invited active volunteers and 11.2% of 1,000 invited lapsed volunteers. Non-response, especially among lapsed volunteers, could materially bias results.
  • Respondent status is self-reported, whereas invitation groups use recent-shift dates. They need not describe identical groups. The screener excludes people who say they stopped 12 months ago or longer or are unsure.
  • Younger volunteers are under-represented: under-25s are 9.8% of lapsed respondents versus 22% in the lapsed database, and 7.4% of active respondents versus 18% in the active database.
  • The margin of error and significance test use conventional sampling approximations; the survey was not a random sample of respondents. They do not account for non-response or other selection bias.
  • The projection of 429 is a mechanical application of 48/112 to 1,000, not a reliable population count. Age weighting alone does not correct all potential bias.
  • Q2 is multiple-choice: mentioning a difficulty does not establish its importance, whether it caused someone to leave, or whether paying costs would change behaviour. Q3 asks for one closest main reason, so can conceal multiple causes.
  • Q4 and Q5 measure hypothetical likelihood, not actual return, incremental impact or cost-effectiveness. The survey does not establish the cost of either option, demand for reimbursement, or whether booking alone would prevent late changes.
  • Open-text themes are interpretive and overlap. The scheduling count includes 36 answers about booking, swaps or shift changes; V308 discusses a site-location change and is included in the broader communication theme but not that count. Q1 has 109 answers, not 327.
  • Site proportions alone do not establish where the problems are concentrated; no site-by-difficulty results or reliable site-level comparisons are supplied.

Next steps

  • decisionCost a part-time coordinator, a simple booking system with non-digital access, and a capped travel scheme before approving the budget split.
  • experimentPilot coordinator-led rota standards, including earlier publication, fewer avoidable changes, direct notifications and booking or swapping online or by phone.
  • experimentPilot capped reimbursement across ages, with simple claims and attention to volunteers who face the greatest barriers.
  • experimentUse random assignment or a phased rollout with a comparison where feasible; track actual behaviour rather than only stated likelihood.
  • researchFollow up with a deliberately diverse sample of lapsed volunteers, particularly under-25s and survey non-respondents, and check travel costs and digital access.
  • monitorMonitor outcomes and spending by site, age and support type, then review the allocation.

Open-text themes it coded

Booking, swapping and viewing shifts 20Unreliable changes or inadequate communication about changes 19Travel expense or ways to reduce it 22Journey length, distance or a closer site 9Parking availability or access 2Training 1Recognition and feeling valued 2No change suggested or satisfied as things are 38

The survey it planned

1 screening question and 10 questions, as the model wrote them.

  1. S1

    When did you last do a volunteer shift for Riverside Food Network?

    One answer
    • Within the last 3 months
    • More than 3 months ago, but within the last 12 months
    • More than 12 months ago
    • I have never done a shift for Riverside Food Network
    • I'm not sure

    Continues if Within the last 3 months; More than 3 months ago, but within the last 12 months

  2. Q1

    Which best describes your volunteering with Riverside Food Network now?

    One answer
    • I still volunteer
    • I have stopped volunteering
    • I am taking a break
    • I'm not sure
    • Prefer not to say
  3. Q2

    In the last 12 months, which of the following, if any, have made it difficult for you to volunteer or arrange a shift with Riverside? Select all that apply. Select 'None' or 'Prefer not to say' on its own.

    Any that apply
    • Cost of travel or parking
    • Getting to or from a site, apart from the cost
    • Finding shifts that fit around work, study or other commitments
    • Finding out which shifts are available
    • Booking, changing or swapping a shift
    • Contacting the site lead
    • Health, disability or access needs
    • Caring or family responsibilities
    • The tasks or physical demands of volunteering
    • Feeling unsupported or not welcomed
    • Another difficulty
    • None
    • Prefer not to say
  4. Q3

    What contributed to your decision to stop volunteering or take a break from Riverside? Select all that apply. Select 'No single reason' or 'Prefer not to say' on its own.

    Any that apply
    • Cost of travel or parking
    • Difficulty getting to or from a site, apart from the cost
    • Shifts did not fit around work, study or other commitments
    • Difficulty finding out which shifts were available
    • Difficulty booking, changing or swapping a shift
    • Difficulty contacting the site lead
    • Health, disability or access needs
    • Caring or family responsibilities
    • The tasks or physical demands of volunteering
    • Feeling unsupported or not welcomed
    • Moved away
    • No longer wanted to volunteer
    • Another reason
    • No single reason
    • Prefer not to say

    Routing Shown only if Q1 is: I have stopped volunteering; I am taking a break

  5. Q4

    With the current arrangements—contacting your site lead to book or swap shifts and paying your own travel costs—how likely are you to do a Riverside shift in the next 3 months? Use 1 for very unlikely, 2 for unlikely, 3 for neither likely nor unlikely, 4 for likely or 5 for very likely.

    Scale

    Scale 1-5: Very unlikely to Very likely

  6. Q5

    Suppose Riverside reimbursed your usual return travel and parking costs for each shift, while booking and swapping shifts stayed as they are now. How likely would you be to do a Riverside shift in the next 3 months? Use 1 for very unlikely, 2 for unlikely, 3 for neither likely nor unlikely, 4 for likely or 5 for very likely. This is a possible option, not a promise.

    Scale

    Scale 1-5: Very unlikely to Very likely

  7. Q6

    Suppose Riverside had a part-time volunteer coordinator to help volunteers, and you could book and swap shifts online, but you still paid your own travel costs. How likely would you be to do a Riverside shift in the next 3 months? Use 1 for very unlikely, 2 for unlikely, 3 for neither likely nor unlikely, 4 for likely or 5 for very likely. This is a possible option, not a promise.

    Scale

    Scale 1-5: Very unlikely to Very likely

  8. Q7

    Suppose Riverside reimbursed half of your usual return travel and parking costs for each shift, and offered online shift booking and swaps plus a volunteer coordinator available for fewer hours. How likely would you be to do a Riverside shift in the next 3 months? Use 1 for very unlikely, 2 for unlikely, 3 for neither likely nor unlikely, 4 for likely or 5 for very likely. This is a possible option, not a promise.

    Scale

    Scale 1-5: Very unlikely to Very likely

  9. Q8

    Thinking about a typical Riverside shift in the last 12 months, roughly how much did you personally pay in total for the return journey and any parking?

    One answer
    • Nothing
    • More than £0 but less than £5
    • £5 to less than £10
    • £10 to less than £20
    • £20 or more
    • Don't know
    • Prefer not to say
  10. Q9

    What is the name or location of the Riverside distribution site where you volunteered most often in the last 12 months? If you volunteered equally often at more than one site, please give either one.

    Open text
  11. Q10

    Which age group are you in?

    One answer
    • Under 18
    • 18–24
    • 25–34
    • 35–44
    • 45–54
    • 55–64
    • 65–74
    • 75 or over
    • Prefer not to say
Sample plan

Invite everyone on the volunteer database recorded as having done at least one shift in the last 12 months, by email and text: approximately 410 with a shift in the last 3 months and 1,000 without one. Target about 330 completes; keep the survey open to all eligible invitees rather than excluding people when a quota fills. Monitor responses by database-recorded site and recent-shift status, and target reminders to underrepresented groups. Test the questionnaire on a phone and check that it takes no more than 6 minutes. The requested site precision is not achievable with 330 completes: an even split gives about 37 per site and an approximate 95% margin of error of ±16 percentage points without a finite-population correction. At an illustrative eligible population of 157 per site, the correction reduces this to about ±14 points; approximately 88 completes per site, or 792 in total, would be needed for ±7 points under random sampling. Actual requirements depend on each site's eligible population. A census invitation with nonresponse is not a random sample, so these margins do not account for nonresponse bias. Report site achieved counts and limitations to trustees before fielding.