Sign inTest your prompt

Benchmarks / SurveyBench / Why Riverside Food Network is losing volunteers / GPT-6 Luna

Measured by Spring Prompt

GPT-6 Luna: Why Riverside Food Network is losing volunteers

The decision: how to spend next year's £45,000 for volunteer support: reimburse volunteers' travel costs, fund a part-time volunteer coordinator with online shift booking, or do both at a smaller scale. The brief and the data →

Verdict
✗ Not sound
Research score
94 of 100
Analysis rating
1,102
Head to head, this task
won 4 of 10

Why it is not sound

The analysis

  • Missed a trap: Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What percentage of active volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓37.2%37.2%
What percentage of lapsed volunteers who responded chose 'The cost of travelling to the site' as a difficulty in Q2?✓40.2%40.2%
Is the difference between active and lapsed respondents in the share choosing 'The cost of travelling to the site' in Q2 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Weighted to the age mix of all 1,000 lapsed volunteers, what percentage of lapsed volunteers would choose 'The cost of travelling to the site' as a difficulty in Q2?✓47.4%47.4%
What percentage of lapsed respondents gave a scheduling reason as their main reason for stopping in Q3 (either of its first two options: shifts changed at short notice, or hard to arrange shifts)?✓42.9%42.9%
What is the 95% margin of error, in percentage points, on that unweighted share of lapsed respondents giving a scheduling reason as their main reason?✓9.2 points9.17 points
On average, how many difficulties did each respondent choose in Q2 (counting every option except 'None of these', which counts as zero), across all respondents?✓1.551.55
What percentage of lapsed respondents said they would be likely (fairly or very) to volunteer again if they could arrange their shifts online (Q4)?✓40.2%40.2%
How many of the open-text answers mention shift scheduling (booking or swapping shifts, or last-minute changes or cancellations)?✓3734
What percentage of the active volunteers invited completed the survey?✓52.4%52.4%
What percentage of the lapsed volunteers invited completed the survey?✓11.2%11.2%
If the lapsed respondents were representative, about how many of the 1,000 lapsed volunteers would give a scheduling reason (either of the first two Q3 options) as their main reason, using the unweighted share?✓429429

Traps in the data 7 of 8 handled

  • ✗Travel cost is the most-cited difficulty overall, but it is cited about as often by active volunteers (37%) as by lapsed ones (40%), so it does not explain who left. Short-notice shift changes (14% active against 52% lapsed), no way to book or swap shifts and not being told about changes differ sharply.
  • ✓The operations lead believes volunteers leave because of travel costs. Among lapsed respondents, 43% give a scheduling reason as their main reason against 12% travel cost.
  • ✓Q2 is multi-select: respondents chose 1.6 difficulties on average, so columns add up to more than 100%.
  • ✓52% of active volunteers responded but only 11% of lapsed ones, so the lapsed base is small (112) and those who replied may differ from the 89% who did not (for example, still feeling some attachment).
  • ✓Volunteers aged 65 or over are 43% of respondents but about 26% of volunteers in the database; younger volunteers cite travel cost more. Weighted to the lapsed age mix, travel cost among lapsed volunteers rises from 40% to 47%; a scheduling main reason goes from 43% to 41%, still well ahead of travel cost (21%).
  • ✓Lapsed under-25s (base 11) mostly give travel cost as their main reason (8 of 11).
  • ✓The travel-cost gap between lapsed and active respondents (3.0 points) is not significant at 95%.
  • ✓40% of lapsed respondents say they would be likely to volunteer again with online booking: a stated intention about a hypothetical, from a small and self-selected base.

The analysis

Recommendation

Do not spend the full £45,000 on blanket travel reimbursement. Make rota reliability and communication the core investment: fund sufficient coordination to publish shifts consistently, notify volunteers promptly about changes, and make booking and swaps straightforward, using an online tool only where it supports those processes. In parallel, reserve a smaller, capped budget for a travel-support pilot that is available according to need rather than age alone. The survey indicates a particular potential travel barrier for younger lapsed respondents, but the base of 11 is too small to justify an under-25-only policy. Do not forecast that 40.2% of lapsed volunteers will return because they rated online booking positively; measure actual reactivation and ongoing participation.

What would change it

A well-run, adequately powered pilot showing that travel reimbursement produces substantially more additional volunteer shifts per pound than coordinator-led rota improvements would shift more of the budget toward travel support. Conversely, evidence that reliable scheduling and self-service booking materially increase actual return and retention at a sustainable cost would support making that the larger investment. A larger, representative study of younger lapsed volunteers could justify targeted under-25 support if it confirms both unmet need and effective take-up.

Findings

  1. Travel cost is a common difficulty, but the respondent data do not show it is more common among lapsed than active volunteers. It should not, by itself, justify putting the whole budget into travel reimbursement.

    Medium confidenceBase 327

    In Q2, 40.2% of lapsed respondents (45/112) and 37.2% of active respondents (80/215) selected travel cost, a 3.0 percentage-point difference that is not statistically significant at the 95% level. Weighting lapsed responses to the database age mix raises the estimate to 47.4%.

  2. Scheduling reliability and access are the stronger overall retention signal in this survey. Prioritise fixing rota changes, notice and shift booking, rather than treating an online tool alone as the solution.

    Medium confidenceBase 112

    Among lapsed respondents, 42.9% (48/112) named a scheduling issue as their main reason for stopping in Q3: 29.5% cited short-notice changes and 13.4% difficulty arranging shifts. Travel cost or distance together was the main reason for 12.5% (14/112). In Q2, 51.8% reported short-notice changes, 42.0% difficulty arranging shifts and 41.1% poor notice of changes. Open-text answers also repeatedly describe cancellations and changes that were not communicated in time.

  3. An online booking option is promising for some former volunteers, particularly those whose main reason for stopping was scheduling, but the survey cannot tell trustees how many will actually return.

    Medium confidenceBase 112

    40.2% of lapsed respondents (45/112) said they would be fairly or very likely to volunteer again if they could arrange shifts online. This was 60.4% of those whose main reason was scheduling (29/48), versus 14.3% of those whose main reason was travel cost or distance (2/14). These are hypothetical intentions, not observed returns.

  4. The under-25 results support testing travel help for younger volunteers, but are too small and potentially unrepresentative to justify an under-25-only scheme yet.

    Low confidenceBase 11

    Among lapsed respondents under 25, 81.8% (9/11) selected travel cost as a Q2 difficulty and 72.7% (8/11) named it as their main Q3 reason for stopping. The lapsed under-25 base is only 11, while under-25s are 22% of the lapsed database. For lapsed respondents of all ages, 57.1% (8/14) whose main reason was travel cost or distance said they would be likely to return if travel costs were reimbursed.

  5. Use the £45,000 for a measured combination: address rota management and communication as the core intervention, while piloting accessible travel support rather than committing to blanket reimbursement.

    Medium confidenceBase 112

    Scheduling was the main reason for stopping for 48/112 lapsed respondents; travel cost or distance was the main reason for 14/112. In the travel-reason subgroup, 57.1% (8/14) said reimbursement would make them likely to return, compared with 14.3% (2/14) for online booking. Neither hypothetical response establishes actual take-up or return.

  6. The Q2 burden is not just travel: respondents selected an average of about 1.55 difficulties each. This supports addressing more than one barrier, while recognising that the lapsed sample has substantial nonresponse risk.

    Medium confidenceBase 327

    Across 327 respondents, the nine difficulty options other than 'None of these' sum to approximately 508 selections, or 1.55 per respondent. Completion was 215/410 active invitees (52.4%) and 112/1,000 lapsed invitees (11.2%).

Caveats

  • The lapsed response rate is only 11.2%, so people who replied may differ from the 1,000 lapsed volunteers who did not. Age weighting adjusts only for age mix; it cannot correct other nonresponse differences.
  • The under-25 lapsed findings rest on 11 respondents. Under-25s are underrepresented in the lapsed responses relative to the database (9.8% versus 22%), making age-specific conclusions especially uncertain.
  • The 47.4% age-weighted travel estimate applies the database age distribution to rounded Q2 subgroup percentages. It is an adjusted estimate, not a directly observed population figure.
  • The 95% margin of error of 9.2 percentage points for the scheduling share is a simple sampling-error estimate for the unweighted lapsed respondent share. It does not account for nonresponse bias or other survey errors.
  • Q4 and Q5 ask separate hypothetical questions. Likelihood ratings are not evidence of actual return, reimbursement take-up, or the effect of combining interventions; they also do not establish that an app would fix poor rota decisions or communication.
  • The survey does not provide intervention costs, expected reimbursement claims, volunteer hours gained, or a comparison of cost-effectiveness. These are essential to allocating the £45,000.
  • The reported completion rates use completed surveys divided by invited volunteers. The lapsed screener and the database definition of lapsed should be checked against the final routing and analysis data before using these estimates as a population forecast.

Next steps

  • decisionCost the coordinator, booking/notification system and travel-support options, including administration, expected eligible claims and the volunteer hours each could enable within £45,000.
  • experimentPilot a coordinator-led rota process at a small number of sites: publish shifts earlier, reduce avoidable last-minute changes, give direct change alerts, and offer simple online booking and swaps.
  • experimentRun a capped travel-support pilot, with transparent eligibility and options suitable for bus, rail, mileage and parking costs; invite volunteers of all ages to apply based on need.
  • researchInvite a sample of nonresponding lapsed volunteers, including under-25s, to a short follow-up by phone or text asking about main barriers, travel cost and what support would enable a return.
  • researchCheck the survey routing and reconcile S1/S2 classifications with the volunteer database before using the results in formal forecasts.
  • monitorMonitor actual reactivation and retention after each intervention, rather than using stated likelihood as a return forecast.

Open-text themes it coded

Travel cost, journey distance and practical transport or parking barriers 29Self-service shift booking, swapping or viewing available shifts 20Rota stability, short-notice changes or cancellations, and timely communication 20No change suggested; volunteering is already working well for them 38More training or clearer induction 1Feeling thanked and recognised 2

The survey it planned

2 screening questions and 10 questions, as the model wrote them.

  1. S1

    Have you completed at least one volunteer shift with Riverside Food Network in the past 12 months?

    One answer
    • Yes
    • No
    • Not sure

    Continues if Yes

  2. S2

    Are you happy to take part in this survey? Your answers will be analysed with other volunteers’ answers and used to inform Riverside Food Network’s volunteer support plans.

    One answer
    • Yes, I agree
    • No, I do not agree

    Continues if Yes, I agree

  3. Q1

    Which best describes your volunteering with Riverside Food Network at the moment?

    One answer
    • I have volunteered at least once in the past 3 months
    • I have not volunteered in the past 3 months, but consider myself on a temporary pause
    • I have not volunteered in the past 3 months and consider myself to have stopped
    • I am not sure
  4. Q2

    At which Riverside distribution site did you do most of your shifts in the past 12 months? If you did shifts equally at more than one site, please enter “equally at multiple sites”.

    Open text
  5. Q3

    What is your age group?

    One answer
    • Under 18
    • 18–24
    • 25–34
    • 35–44
    • 45–54
    • 55–64
    • 65 or over
    • Prefer not to say
  6. Q4

    Thinking about when you stopped or paused volunteering, what were the main reasons? Select all that apply.

    Any that apply
    • Work or study commitments
    • Caring or family commitments
    • My health, disability or energy levels
    • The cost of getting to shifts
    • Transport to the site was difficult or unreliable
    • Shift times or opening hours did not suit me
    • It was difficult to book or swap shifts
    • I had difficulty getting information or a response about shifts
    • I did not feel sufficiently supported or welcomed
    • The tasks or role were not right for me
    • A negative experience while volunteering
    • I moved or the site was no longer convenient
    • My circumstances or priorities changed for another reason
    • I had only intended to volunteer for a limited period
    • Other reason
    • Prefer not to say

    Routing Shown only if Q1 is: I have not volunteered in the past 3 months, but consider myself on a temporary pause; I have not volunteered in the past 3 months and consider myself to have stopped

  7. Q5

    In the past 12 months, which of these, if any, have made it difficult for you to volunteer with Riverside Food Network? Select all that apply.

    Any that apply
    • The cost of getting to shifts
    • Transport to the site was difficult or unreliable
    • Shift times or opening hours did not suit me
    • Booking or swapping shifts was difficult
    • Getting information or a response about shifts was difficult
    • Work or study commitments
    • Caring or family commitments
    • My health, disability or energy levels
    • The tasks or role
    • Not feeling sufficiently supported or welcomed
    • Other difficulty
    • None of these
  8. Q6

    About how much did you usually pay yourself for travel to and from one shift, including bus fares and parking? Please enter an amount in pounds. Enter 0 if you had no out-of-pocket travel costs.

    Number
  9. Q7

    In the past 12 months, how often did the cost of getting to a shift make you decide not to volunteer, or make it difficult for you to volunteer?

    Scale

    Scale 1-5: Never to Very often

  10. Q8

    If you were able to volunteer, how likely would you be to return to a shift within the next 6 months if Riverside reimbursed your eligible travel costs for each shift?

    Scale

    Scale 1-5: Very unlikely to Very likely

    Routing Shown only if Q1 is: I have not volunteered in the past 3 months, but consider myself on a temporary pause; I have not volunteered in the past 3 months and consider myself to have stopped

  11. Q9

    If you were able to volunteer, how likely would you be to return to a shift within the next 6 months if Riverside offered online shift booking and swapping, supported by a part-time volunteer coordinator?

    Scale

    Scale 1-5: Very unlikely to Very likely

    Routing Shown only if Q1 is: I have not volunteered in the past 3 months, but consider myself on a temporary pause; I have not volunteered in the past 3 months and consider myself to have stopped

  12. Q10

    If you were able to volunteer, how likely would you be to return to a shift within the next 6 months if Riverside offered both some travel-cost reimbursement and online shift booking and swapping supported by a part-time volunteer coordinator, with each offered at a smaller scale than if it focused on just one of these?

    Scale

    Scale 1-5: Very unlikely to Very likely

    Routing Shown only if Q1 is: I have not volunteered in the past 3 months, but consider myself on a temporary pause; I have not volunteered in the past 3 months and consider myself to have stopped

    Judges: ambiguous

    Some reimbursement and smaller-scale provision do not specify what respondents would receive. Different interpretations could materially change return intentions and make comparisons with Q8 and Q9 misleading.

    'Some travel-cost reimbursement' and 'at a smaller scale' are undefined. Respondents cannot know how much less reimbursement or coordinator support they would get, so they will imagine very different offers and the answers cannot be compared with Q8 and Q9.

Sample plan

Invite everyone on the volunteer database recorded as having completed at least one shift in the past 12 months (approximately 1,410 people: 410 active and 1,000 with no shift in the past 3 months) by email and text. Send reminders and monitor completions by site, using targeted reminders to improve coverage at sites with fewer responses. Target about 330 completes; where site counts allow, aim for at least 20 completes per site and use weights to align results with the eligible volunteer roster. The trustees’ request for results at each of the 9 sites to within ±7 percentage points at 95% confidence cannot be promised with 330 completes. Without finite-population corrections, that precision requires about 196 completes per site (about 1,764 total); actual requirements depend on the number of eligible volunteers at each site. Report site-level estimates with confidence intervals and clearly flag small or imprecise samples rather than presenting them as meeting the requested precision.