Sign inTest your prompt

Benchmarks / SurveyBench / What Hollinbrook residents think of three-weekly bin collections / GPT-6 Luna

Measured by Spring Prompt

GPT-6 Luna: What Hollinbrook residents think of three-weekly bin collections

The decision: whether to move general waste collection to every three weeks across the whole district, to do so with extra support for households that need it (a larger bin, or extra collections of nappies and hygiene waste), to pilot it in some areas first, or to drop the proposal. The brief and the data →

Verdict
✗ Not sound
Research score
93 of 100
Analysis rating
1,102
Head to head, this task
won 3 of 9

Why it is not sound

The survey

  • Presumed the stakeholder's view instead of testing it
  • A leading question: Q3

Scorecard

Numbers 11 of 11 right

QuestionIts answerTrue
What percentage of open-consultation respondents tend to oppose or strongly oppose collecting general waste every three weeks (Q2)?✓77.5%77.5%
What percentage of residents' panel respondents tend to oppose or strongly oppose collecting general waste every three weeks (Q2)?✓45.3%45.3%
What percentage of residents' panel respondents strongly support or tend to support collecting general waste every three weeks (Q2)?✓40.8%40.8%
What is the 95% margin of error, in percentage points, on the share of residents' panel respondents who support the change (strongly or tend to)?✓4.7 points4.69 points
On the residents' panel, is the difference in the share supporting the change (strongly or tend to) between owner-occupiers and renters statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Among residents' panel respondents whose household includes a child under 3 or someone who uses continence or other hygiene products, what percentage chose 'every three weeks, with extra support for households that need it' as their preferred option (Q4)?✓55.6%55.6%
How many of the open-text answers are copies or near-copies of the template comment circulated by a local group?✓208209
How many of the open-text answers from residents' panel respondents mention hygiene: smell, flies, maggots or vermin, or nappies or hygiene waste sitting in the bin?✓99
As a percentage of all residents' panel respondents, how many oppose the change at Q2 and said they would support it if their household could have a larger bin at no extra charge (Q3)?✓10.2%10.2%
What percentage of residents' panel respondents said they would be very or fairly likely to recycle more if general waste were collected every three weeks (Q5)?✓40.5%40.5%
What percentage of the addresses invited by letter to the residents' panel produced a completed questionnaire?✓21.1%21.1%

Traps in the data 8 of 8 handled

  • ✓The open consultation is self-selected: anyone with the link could respond, Brookfield is 38% of it against 12% of households, and households with a young child or hygiene needs are over-represented. It shows 78% opposed, against 45% on the panel, which matches the district's age and tenure profile.
  • ✓209 open-consultation comments (16% of the consultation's responses) are copies or near-copies of a template circulated by Keep Brookfield Tidy.
  • ✓The council leader wants to show that residents support the change. Support is 41% on the panel (with 45% opposed) and 15% in the open consultation.
  • ✓Q3 was asked only of those who oppose: 22% of panel opponents would support the change with a larger bin, which is only 10% of all panel respondents; most opponents would still oppose.
  • ✓Wendle Vale (panel base 16) shows 62% support, far above any other ward.
  • ✓On the panel, owner-occupiers support the change more than renters (43% against 37%, bases 282 and 140), but the gap is not statistically significant at 95%.
  • ✓40% of the panel say they would be likely to recycle more: a stated intention about a hypothetical change.
  • ✓Only 422 of 2,000 addresses invited to the panel completed (about 21%); the panel is matched on age and tenure only, and those who replied may care more about bins than those who did not.

The analysis

Recommendation

Do not claim district-wide backing and do not implement the change across the whole district on this evidence. Do not drop it solely because of the open consultation either. Run a time-limited, independently evaluated pilot in varied town and rural areas, with needs-based larger bins and a practical extra-collection option for nappies and hygiene waste available from the outset. Preserve a comparison group on fortnightly collections and make the final decision only after assessing service outcomes, resident impacts, costs and equity.

What would change it

Move toward wider implementation only if a controlled pilot shows a material, sustained reduction in residual waste or a worthwhile net operational benefit, without unacceptable increases in overflow, hygiene/pest complaints, fly-tipping or unequal impacts, and with workable support for households that need it. Retain fortnightly collections or redesign the proposal if those harms occur, support cannot be delivered reliably, or savings are not sufficient to justify the service and equity impacts.

Findings

  1. The residents' panel does not show majority support: opposition is slightly higher than support. The council leader should not claim residents have backed the change.

    Medium confidenceBase 422

    Among 422 panel respondents, 40.8% support and 45.3% oppose; 10.4% are neutral and 3.6% do not know. The support estimate has an approximate 95% margin of error of 4.7 percentage points under a simple-random-sample assumption, but that does not account for non-response or other biases.

  2. The open consultation's near four-in-five opposition is not a reliable estimate of district-wide opinion and is not, by itself, a reason to drop the proposal.

    High confidenceBase 1318

    Opposition was 77.5% among 1,318 open-consultation respondents, versus 45.3% in the separately reported residents' panel. The open consultation over-represents Brookfield (38.0% versus 12% of households) and households with a child under 3 or hygiene-product use (29% versus the council's estimate of about 15%). There are also 208 copies or near-copies of the circulated comment wording among the 417 open-text answers.

  3. A larger bin alone is unlikely to bring most opponents round. Support should be designed around a package of needs-based measures, not assumed from offering a larger bin.

    Medium confidenceBase 422

    Of 191 panel respondents who opposed at Q2 and were asked Q3, 22.5% said they would support the change with a larger bin, while 61.3% said they would still oppose. The 22.5% represents about 10.2% of the full panel (422 respondents).

  4. Households with a child under 3 or someone using continence or hygiene products need specific consideration before any change. Their panel responses favour extra support, but the subgroup is small.

    Medium confidenceBase 54

    In this panel subgroup (n=54), 74.1% opposed the change at Q2 and 14.8% supported it. At Q4, 55.6% preferred three-weekly collections with extra support. Treat this as a clear design signal, not a precise district-wide estimate.

  5. A staged pilot is more defensible than immediate district-wide implementation or dropping the proposal now.

    Medium confidenceBase 422

    In the panel, the largest single Q4 preference was a trial in some areas first (25.4%); 23.7% preferred three-weekly collections with extra support, 24.6% preferred keeping fortnightly collections, and 22.0% preferred a district-wide change without the specified extra support. These are preferences, not proof that a pilot will work.

  6. There is a plausible recycling benefit, but stated intentions do not establish that the change will increase recycling in practice.

    Medium confidenceBase 422

    In the panel, 40.5% said they would be very or fairly likely to recycle more; 29.1% said they would be fairly or very unlikely, and 11.8% said they already recycle everything they can.

  7. The observed owner–renter support difference is not statistically significant at the 95% level and should not drive separate policy decisions on its own.

    Medium confidenceBase 422

    Panel support was 42.6% among owner-occupiers (n=282) and 37.1% among renters (n=140), a 5.5 percentage-point difference; a two-proportion comparison does not reach the 95% significance threshold.

  8. Open-text concerns identify risks that a pilot should test directly, especially bin capacity, hygiene and pests, fly-tipping, and perceived value for money. Repeated template comments should not be counted as independent evidence of how common those concerns are.

    Medium confidenceBase 417

    There were 417 open-text answers, including 208 copies or near-copies of circulated wording. Nine residents' panel comments explicitly mention hygiene-related issues such as smell, flies, maggots or vermin, or nappies/hygiene waste in the bin.

Caveats

  • The open consultation was voluntary and self-selected, promoted through council channels and then actively shared by a local campaign group. Its percentages describe respondents, not a representative estimate of all district residents.
  • The residents' panel is the stronger source for district-wide views, but only 422 of 2,000 invited addresses completed the questionnaire (21.1%). Matching panel places to age and tenure does not eliminate non-response bias; the reported tables are unweighted.
  • The 4.7-point margin of error is a sampling-only approximation. It does not capture non-response, panel recruitment, measurement or other sources of error.
  • Subgroup results, especially for the 54 panel respondents in households with a child under 3 or hygiene-product use, are imprecise. Ward results also have small bases in some wards.
  • Q3 is a hypothetical follow-up asked only of Q2 opponents. It measures stated response to a larger bin, not actual acceptance after a service change or the effect of other support such as extra hygiene-waste collections.
  • Q5 records stated likelihood of recycling more, not observed recycling behaviour or waste tonnage.
  • The 417 open-text answers are qualitative evidence, not a representative frequency measure. Repeated or campaign-circulated wording reduces the value of counting comments as independent signals.
  • The survey did not measure household size, actual bin fullness, current overflow incidents, waste composition, fly-tipping rates, operational costs or the potential cost and uptake of support options.

Next steps

  • decisionApprove a time-limited pilot rather than district-wide implementation, with a comparison group continuing fortnightly collections.
  • experimentSelect pilot and comparison areas to cover different town/rural settings and waste-service conditions; collect baseline data before changing collection frequency.
  • researchCo-design and offer needs-based support, including larger bins and extra collections for nappies and hygiene waste, with clear eligibility and an accessible request process.
  • monitorMonitor actual outcomes throughout the pilot, using consistent definitions and area-level comparison with the fortnightly service.
  • researchConduct follow-up research with a representative sample and targeted interviews with households using nappies or hygiene products, larger households and residents with limited storage.

Open-text themes it coded

Copies or near-copies of the circulated objection template 208Explicit hygiene concerns in residents' panel open-text answers 9

The survey it planned

2 screening questions and 10 questions, as the model wrote them.

  1. S1

    How old are you?

    One answer
    • Under 18
    • 18–24
    • 25–34
    • 35–44
    • 45–54
    • 55–64
    • 65–74
    • 75 or over
    • Prefer not to say

    Continues if 18–24; 25–34; 35–44; 45–54; 55–64; 65–74; 75 or over

  2. S2

    Do you currently live in the Hollinbrook District Council area?

    One answer
    • Yes
    • No
    • Not sure

    Continues if Yes

  3. Q1

    Does anyone in your household have any of the following needs? Select all that apply.

    Any that apply
    • A child under 5
    • Disposable nappies are used in the household
    • Someone uses continence or other hygiene products that need to be put in the general waste bin
    • None of these
    • Prefer not to say
  4. Q2

    How often is your household’s black bin full or nearly full when it is due to be collected?

    One answer
    • Never
    • Rarely
    • Sometimes
    • Most of the time
    • Every time
    • Not sure
    Judges: assumptive

    Assumes the household has its own black bin. Eligible residents using communal bins or other collection arrangements cannot necessarily answer about their household's bin; there is no applicable response for that situation.

    Assumes the household has its own individual black bin, which may not be true for residents who use communal bins or bags (e.g., in flats).

  5. Q3

    Hollinbrook District Council is considering changing general waste (black bin) collections from every two weeks to every three weeks across the district. Food waste would still be collected weekly and recycling every two weeks. The change is estimated to save about £600,000 a year in collection and disposal costs. The council needs to find £3.1 million in savings next year. Some councils that have moved to three-weekly general waste collections have reported higher recycling, but this is not guaranteed. How much do you support or oppose changing to three-weekly black bin collections across the district, with no extra support specified?

    One answer
    • Strongly support
    • Somewhat support
    • Neither support nor oppose
    • Somewhat oppose
    • Strongly oppose
    • Not sure
    Judges: leading

    The introduction presents savings, the council's financial pressures and potential recycling benefits, without corresponding disadvantages. This one-sided framing steers the support measure towards the proposal.

    The preamble gives only arguments for the change: the £600,000 saving, the £3.1m savings need and the possibility of higher recycling. It mentions no drawbacks, such as overflowing bins, smells or harder management for families. This one-sided framing pushes respondents towards support.

  6. Q4

    If black bin collections changed to every three weeks, how do you think the amount your household recycles would change?

    One answer
    • It would increase a lot
    • It would increase a little
    • It would stay about the same
    • It would decrease
    • Not sure
  7. Q5

    If your black bin were collected every three weeks, how difficult or easy do you think it would be for your household to manage?

    One answer
    • Very difficult
    • Fairly difficult
    • Neither difficult nor easy
    • Fairly easy
    • Very easy
    • Not sure
  8. Q6

    Which option would you prefer the council to take?

    One answer
    • Change black bin collections to every three weeks across the district, without extra support
    • Change black bin collections to every three weeks across the district, with larger bins for households that need more capacity and extra collections for nappies and hygiene waste
    • Pilot three-weekly black bin collections in some areas before deciding whether to introduce them across the district
    • Keep black bin collections every two weeks
    • No preference
    • Not sure
  9. Q7

    If the council offered larger black bins to households that need more capacity, and extra collections for nappies and hygiene waste to households that need them, how would that affect your view of changing to three-weekly black bin collections across the district?

    One answer
    • I would be much more supportive
    • I would be a little more supportive
    • It would make no difference to my view
    • I would be a little less supportive
    • I would be much less supportive
    • Not sure
  10. Q8

    Which, if any, of the following would make three-weekly black bin collections more manageable for your household? Select all that apply.

    Any that apply
    • A larger black bin
    • Extra collections for nappies and hygiene waste
    • Both of these
    • None of these
    • Not sure
  11. Q9

    Which best describes the tenure of the home you live in?

    One answer
    • Owned outright
    • Owned with a mortgage or loan
    • Rented from a private landlord or letting agency
    • Rented from the council or a housing association
    • Other
    • Prefer not to say
  12. Q10

    What is your full postcode? It will be used to identify your ward.

    Open text
Sample plan

Run the same questionnaire in two separate modes. For the residents’ panel, invite randomly selected addresses by letter and aim for about 420 completed surveys, with a target of 70 completes in each of the six wards. Monitor age and housing tenure against district profiles and apply appropriate weighting for district-level estimates. About 70 respondents per ward gives a worst-case margin of error of approximately ±11.7 percentage points at 95% confidence, before any design effect; it cannot provide the requested ±6-point precision. Approximately 267 completes per ward (about 1,602 in total) would be needed for ±6 points at 95% confidence under simple random sampling, so the panel must be increased substantially or the ward-level precision requirement relaxed. Ward results from 70 completes should be treated as indicative. Run the open consultation separately on the council website and promote it through social media and the residents’ magazine; it is self-selecting and should not be combined with the panel or presented as representative of residents. Screen both modes for adult residents of the district.