Benchmarks / SurveyBench / Why Riverside Food Network is losing volunteers / GPT-6.2 Sol

Measured by Spring Prompt

GPT-6.2 Sol: Why Riverside Food Network is losing volunteers

The decision: how to spend next year's £45,000 for volunteer support: reimburse volunteers' travel costs, fund a part-time volunteer coordinator with online shift booking, or do both at a smaller scale. The brief and the data →

Verdict
✗ Not sound
Research score
88 of 100
Analysis rating
1,354
Head to head, this task
won 3 of 12

Why it is not sound

The survey

  • Answer bands that overlap or leave gaps

The analysis

  • Missed a trap: Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What percentage of active volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓37.2%37.2%
What percentage of lapsed volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓40.2%40.2%
Is the difference between active and lapsed respondents in the share choosing 'The cost of travelling to the site' in Q2 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Weighted to the age mix of all 1,000 lapsed volunteers, what percentage of lapsed volunteers would choose 'The cost of travelling to the site' as a difficulty in Q2?✓47.4%47.4%
What percentage of lapsed respondents gave a scheduling reason as their main reason for stopping in Q3 (either of its first two options: shifts changed at short notice, or hard to arrange shifts)?✓42.9%42.9%
What is the 95% margin of error, in percentage points, on that unweighted share of lapsed respondents giving a scheduling reason as their main reason?✓9.2 points9.17 points
On average, how many difficulties did each respondent choose in Q2 (counting every option except 'None of these', which counts as zero), across all respondents?✓1.551.55
What percentage of lapsed respondents said they would be likely (fairly or very) to volunteer again if they could arrange their shifts online (Q4)?✓40.2%40.2%
How many of the open-text answers mention shift scheduling (booking or swapping shifts, or last-minute changes or cancellations)?✓3634
What percentage of the active volunteers invited completed the survey?✓52.4%52.4%
What percentage of the lapsed volunteers invited completed the survey?✓11.2%11.2%
If the lapsed respondents were representative, about how many of the 1,000 lapsed volunteers would give a scheduling reason (either of the first two Q3 options) as their main reason, using the unweighted share?✓429429

Traps in the data 7 of 8 handled

  • ✗Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
  • ✓The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.
  • ✓Q2 is multi-select: respondents chose 1.6 difficulties on average, so columns add up to more than 100%.
  • ✓52% of active volunteers responded but only 11% of lapsed ones, so the lapsed base is small (112) and those who replied may differ from the 89% who did not (for example, still feeling some attachment).
  • ✓Volunteers aged 65 or over are 43% of respondents but about 26% of volunteers in the database; younger volunteers cite travel cost more. Weighted to the lapsed age mix, travel cost among lapsed volunteers rises from 40% to 47%; a scheduling main reason goes from 43% to 41%, still well ahead of travel cost (21%).
  • ✓Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
  • ✓The travel-cost gap between lapsed and active respondents (3.0 points) is not significant at 95%.
  • ✓40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.

The analysis

Recommendation

Do both at a smaller scale, with the coordinator and rota improvements as the first priority. Fund enough coordinator time to improve rota reliability, introduce simple online booking and swapping, and send prompt confirmations and change alerts. Keep phone or text booking available. Use a capped travel pilot for reasonable out-of-pocket costs, open to volunteers of all ages at pilot sites, with an option that does not require people who cannot afford the fare to pay upfront. Cost both components within £45,000 before committing, and hold back funds for expansion until there is evidence from actual shifts and claims.

What would change it

Shift more funding to travel if a comparison shows reimbursement improves attendance, retention, or access more per pound than the rota package, including among volunteers who otherwise cannot afford to attend. Shift more to coordination if the rota package materially improves reliability and filled shifts while reimbursement adds little even after claim barriers are removed. Consider age-specific eligibility only if better evidence shows that age predicts unmet need or benefit better than a travel-cost criterion.

Findings

  1. Travel costs are a real barrier, especially for supporting current volunteers, but being the most selected difficulty overall does not justify spending the full £45,000 on reimbursement.

    Medium confidenceBase 327

    Travel cost was selected by 125 of 327 respondents (38.2%): 80 of 215 active respondents (37.2%) and 45 of 112 lapsed respondents (40.2%). The 3.0 percentage-point gap was not statistically significant in a conventional two-sided test (p ≈ 0.60). Among lapsed respondents, short-notice changes were more commonly selected (58 of 112, 51.8%), and only 13 of 112 (11.6%) named travel cost as their main reason for stopping.

  2. Make rota reliability the coordinator's first job. Online booking can make shifts easier to arrange, but it will not by itself prevent late changes or ensure volunteers hear about them.

    Medium confidenceBase 112

    48 of 112 lapsed respondents (42.9%) named either short-notice changes (33) or difficulty arranging shifts (15) as their main reason for stopping. Another 14 (12.5%) separately named poor notice of changes and are not included in the 48. Open comments distinguish self-service booking and swapping from earlier, direct alerts and fewer cancellations.

  3. There is no defensible headcount for how many lost volunteers would come back with online booking. Use the results to design a pilot, not to promise returns.

    Medium confidenceBase 112

    45 of 112 lapsed respondents (40.2%) said they were fairly or very likely to return with online booking, including 19 who said very likely. The figure was 29 of 48 (60.4%) among those who stopped mainly for scheduling reasons. For reimbursement, 30 of 112 (26.8%) said they were likely to return, rising to 8 of 14 (57.1%) among those who stopped mainly because of travel cost or time. These are stated intentions, not observed or incremental returns.

  4. Do not restrict a travel scheme to under-25s on this evidence. Make sure younger volunteers are included in the pilot, while letting volunteers of any age access support when travel cost is a barrier.

    Low confidenceBase 11

    Nine of 11 lapsed under-25 respondents selected travel cost as a difficulty and eight named it as their main reason for stopping. That is a strong signal within a very small group: each answer moves the subgroup result by 9.1 percentage points, and under-25s were 9.8% of lapsed respondents versus 22% of the lapsed database. Older volunteers also described affordability problems, including an active volunteer living on a pension.

  5. The unweighted survey may understate travel-related difficulty among lapsed volunteers, but age weighting does not establish the true population rate.

    Low confidenceBase 112

    Applying the database age mix to the four lapsed respondent age-group rates raises the travel-cost estimate from 40.2% to 47.4%. The calculation gives more weight to younger age groups, including the under-25 group of just 11 respondents. It cannot correct for differences between respondents and nonrespondents of the same age.

Caveats

  • Completion was much lower for lapsed than active volunteers (112/1,000 versus 215/410). These are the reported completion-to-invitation ratios; respondent status is self-reported, while invitation groups were defined by shift records. An invitation-source response rate would require linking the two. The 65+ group was overrepresented among both active respondents (45.1% versus 30% in the database) and lapsed respondents (39.3% versus 25%).
  • The 9.2 percentage-point margin is the conventional unweighted 95% binomial margin under a simple random sampling assumption. Neither it nor the significance test accounts for nonresponse or differences between sites. The nonsignificant travel gap does not establish that active and lapsed volunteers face the same barrier.
  • The projection of 429 is 48/112 multiplied by 1,000, conditional on respondents being representative. It is not a count of people who would return, and the response pattern is not strong enough to use it as a planning forecast.
  • Q2 measures any difficulty and allows multiple selections; Q3 asks for one main reason. Neither shows what spending would change. Q4 and Q5 asked about separate hypothetical offers, in a fixed order, with no status-quo or combined-offer comparison. The percentages cannot be added or used to predict the effect of the coordinator role.
  • The 36 scheduling mentions count unique open-text respondents in the first two themes, with two respondents in both. A venue-only change is coded separately. The comments explain how problems arise but do not estimate their prevalence by site or age.
  • The survey does not provide annual eligible journey volumes, actual claim rates, full staffing and software costs, or comparative site-level outcomes. It cannot establish a precise budget split or which sites would benefit most.

Next steps

  • decisionAgree the outcomes the £45,000 should buy and cost both options before setting the split.
  • researchCheck invitation status against shift records and follow up with a sample of nonrespondents, especially younger lapsed volunteers, across sites.
  • experimentStagger rollout across comparable sites, randomising timing where feasible. Introduce rota support and travel reimbursement separately where practical so their effects can be distinguished.
  • monitorMonitor take-up and access during the pilot, then make a scale-up decision.

Open-text themes it coded

Easier shift booking, confirmation, recurring shifts, or swaps 20More predictable shifts, fewer late cancellations, or earlier notice of schedule changes 18Direct notification when a session changes venue 1Travel affordability: fares, fuel, paid parking, or sharing travel costs 22Shorter or easier journeys, including a nearer site or better transport connections 9More available or convenient parking or drop-off space 2Training before the first shift 1More thanks or recognition 2No change needed or no suggestion 38

The survey it planned

1 screening question and 10 questions, as the model wrote them.

  1. S1

    In the past 12 months, have you done at least one volunteer shift with Riverside Food Network?

    One answer
    • Yes
    • No
    • I'm not sure

    Continues if Yes

  2. Q1

    Which best describes your volunteering with Riverside Food Network at the moment?

    One answer
    • I still volunteer with Riverside
    • I'm taking a break from volunteering with Riverside
    • I've stopped volunteering with Riverside
    • I'm not sure which best describes me
  3. Q2

    What were the main reasons you stopped volunteering or took a break from volunteering with Riverside? A short answer is fine. Please do not include anyone's name.

    Open text

    Routing Shown only if Q1 is: I'm taking a break from volunteering with Riverside; I've stopped volunteering with Riverside

  4. Q3

    At which Riverside distribution site did you do your most recent shift? If you don't know the site name, a street or neighbourhood is fine.

    Open text
  5. Q4

    Think about the three months up to and including your most recent Riverside shift. Which of the following, if any, made it harder for you to volunteer during that time? Select all that apply.

    Any that apply
    • Work or study commitments
    • Caring or family commitments
    • Health or accessibility needs
    • The cost of travelling to and from the site, including parking
    • Getting to and from the site for reasons other than cost
    • Shift times that didn't suit me
    • Finding out which shifts were available
    • Booking, changing or swapping a shift by phone or message
    • Not getting the information, training or support I needed during shifts
    • The volunteer tasks weren't a good fit for me
    • Something else
    • Nothing made it harder for me
    • I don't remember
  6. Q5

    How much did you pay out of your own pocket for travel to and from your most recent Riverside shift, including any parking? Your best estimate is fine.

    One answer
    • £0
    • More than £0 but less than £3
    • £3 to less than £6
    • £6 to less than £10
    • £10 or more
    • I don't know or don't remember
  7. Q6

    If Riverside kept its current arrangements for booking shifts and did not pay back travel costs, how likely would you be to do at least one Riverside shift in the next three months?

    One answer
    • Very unlikely
    • Fairly unlikely
    • Neither likely nor unlikely
    • Fairly likely
    • Very likely
    • Not sure
  8. Q7

    Imagine you could book or swap shifts online and a part-time volunteer coordinator could help with shift questions. Riverside would not pay back travel costs. If this were available now, how likely would you be to do at least one Riverside shift in the next three months?

    One answer
    • Very unlikely
    • Fairly unlikely
    • Neither likely nor unlikely
    • Fairly likely
    • Very likely
    • Not sure
  9. Q8

    Imagine Riverside paid back the full cost of your bus fares or parking for volunteer shifts, and you still booked or swapped shifts by phoning or messaging your site lead. If this were available now, how likely would you be to do at least one Riverside shift in the next three months?

    One answer
    • Very unlikely
    • Fairly unlikely
    • Neither likely nor unlikely
    • Fairly likely
    • Very likely
    • Not sure
  10. Q9

    Imagine Riverside paid back part, but not all, of your bus fares or parking costs for volunteer shifts. You could also book or swap shifts online, but a volunteer coordinator would be available for fewer hours than in the option without travel payments. If this were available now, how likely would you be to do at least one Riverside shift in the next three months?

    One answer
    • Very unlikely
    • Fairly unlikely
    • Neither likely nor unlikely
    • Fairly likely
    • Very likely
    • Not sure
    Judges: ambiguous

    Neither the proportion of travel costs repaid nor the coordinator's reduced availability is specified. Respondents could envisage materially different packages, making likelihood answers an unclear basis for evaluating the combined option.

    As with Q8, specifies 'bus fares or parking', ignoring other modes of travel like driving (fuel) or train travel.

  11. Q10

    What is your age group?

    One answer
    • Under 25
    • 25 to 34
    • 35 to 44
    • 45 to 54
    • 55 to 64
    • 65 or over
    • Prefer not to say
Sample plan

Invite everyone on the deduplicated volunteer database with a recorded shift in the past 12 months: about 410 with a shift in the past three months and 1,000 without one. Send both an email and a text message with the same unique link per volunteer; accept one completion per person. Aim for about 330 eligible completes, with soft targets of about 100 recent-shift and 230 no-recent-shift respondents. Monitor response by roster site and shift recency, and prioritise reminders to underrepresented groups rather than closing quotas. Record recency and most recent site from the roster; use Q3 to check site assignment. Keep invitations neutral about why people stop and which support should be funded. Test the longest route on mobile before launch to verify it takes no more than six minutes. In Q4, make 'Nothing made it harder for me' and 'I don't remember' exclusive choices. Flag the precision shortfall before launch: if completes and eligible volunteers were evenly distributed, about 37 completes per site would give a worst-case margin of about ±14 percentage points at 95% confidence under simple random sampling with a finite-population correction. With about 157 eligible volunteers per site, ±7 points would require about 88 completes per site, or roughly 790 in total under that illustrative split. Calculate targets using the actual eligible count at each site. If ±7 points is mandatory, revise the completion target and fieldwork plan; if 330 is fixed, agree to wider site intervals. These calculations do not account for nonresponse bias.