Benchmarks / SurveyBench / What Hollinbrook residents think of three-weekly bin collections / GPT-6.1 Terra

Measured by Spring Prompt

GPT-6.1 Terra: What Hollinbrook residents think of three-weekly bin collections

The decision: whether to move general waste collection to every three weeks across the whole district, to do so with extra support for households that need it (a larger bin, or extra collections of nappies and hygiene waste), to pilot it in some areas first, or to drop the proposal. The brief and the data →

Verdict
✗ Not sound
Research score
90 of 100
Analysis rating
1,273
Head to head, this task
won 9 of 11

Why it is not sound

The survey

  • Presumed the stakeholder's view instead of testing it
  • A leading question: Q4

Scorecard

Numbers 11 of 11 right

QuestionIts answerTrue
What percentage of open-consultation respondents tend to oppose or strongly oppose collecting general waste every three weeks (Q2)?✓77.5%77.5%
What percentage of residents' panel respondents tend to oppose or strongly oppose collecting general waste every three weeks (Q2)?✓45.3%45.3%
What percentage of residents' panel respondents strongly support or tend to support collecting general waste every three weeks (Q2)?✓40.8%40.8%
What is the 95% margin of error, in percentage points, on the share of residents' panel respondents who support the change (strongly or tend to)?✓4.7 points4.69 points
On the residents' panel, is the difference in the share supporting the change (strongly or tend to) between owner-occupiers and renters statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Among residents' panel respondents whose household includes a child under 3 or someone who uses continence or other hygiene products, what percentage chose 'every three weeks, with extra support for households that need it' as their preferred option (Q4)?✓55.6%55.6%
How many of the open-text answers are copies or near-copies of the template comment circulated by a local group?✓209209
How many of the open-text answers from residents' panel respondents mention hygiene: smell, flies, maggots or vermin, or nappies or hygiene waste sitting in the bin?✓99
As a percentage of all residents' panel respondents, how many oppose the change at Q2 and said they would support it if their household could have a larger bin at no extra charge (Q3)?✓10.2%10.2%
What percentage of residents' panel respondents said they would be very or fairly likely to recycle more if general waste were collected every three weeks (Q5)?✓40.5%40.5%
What percentage of the addresses invited by letter to the residents' panel produced a completed questionnaire?✓21.1%21.1%

Traps in the data 8 of 8 handled

  • ✓The open consultation is self-selected: anyone with the link could respond, Brookfield is 38% of it against 12% of households, and households with a young child or hygiene needs are over-represented. It shows 78% opposed, against 45% on the panel, which matches the district's age and tenure profile.
  • ✓209 open-consultation comments (16% of the consultation's responses) are copies or near-copies of a template circulated by Keep Brookfield Tidy.
  • ✓The council leader wants to show that residents support the change. Support is 41% on the panel (with 45% opposed) and 15% in the open consultation.
  • ✓Q3 was asked only of those who oppose: 22% of panel opponents would support the change with a larger bin, which is only 10% of all panel respondents; most opponents would still oppose.
  • ✓Wendle Vale (panel base 16) shows 62% support, far above any other ward.
  • ✓On the panel, owner-occupiers support the change more than renters (43% against 37%, bases 282 and 140), but the gap is not statistically significant at 95%.
  • ✓40% of the panel say they would be likely to recycle more: a stated intention about a hypothetical change.
  • ✓Only 422 of 2,000 addresses invited to the panel completed (about 21%); the panel is matched on age and tenure only, and those who replied may care more about bins than those who did not.

The analysis

Recommendation

Do not claim district-wide backing or proceed directly to a district-wide change. Subject to operational and financial feasibility, run a time-limited pilot in varied town and rural areas, with accessible larger-bin and nappy or hygiene-waste support available from the start. Retain fortnightly collections elsewhere and reserve a genuine option to stop or revise the proposal after evaluation. If a safe, adequately supported and measurable pilot cannot be delivered, retain the current service while the gaps are resolved.

What would change it

Evidence of unacceptable hygiene, overflow, access or dumping impacts despite support, poor support take-up, or no credible net financial or environmental benefit would favour stopping or redesigning the proposal. Measured benefits without unacceptable harms, viable support and stronger evidence of acceptability across affected households would justify considering a supported rollout. A better-recruited district survey showing decisive opposition to even the supported option would weigh against proceeding.

Findings

  1. The leader should not tell full council that residents back the proposal. Panel opinion is divided, with opposition numerically ahead of support.

    Medium confidenceBase 422

    In the panel, 40.8% supported and 45.3% opposed the basic three-weekly proposal, n=422. A conventional 95% margin for the support estimate is approximately 4.7 percentage points, but it does not account for selection and nonresponse.

  2. The cabinet member is right that opposition was high among consultation respondents, but that alone is not enough to conclude that the district opposes every version of the proposal or that it must be dropped.

    High confidenceBase 1318

    Opposition was 77.5% in the open consultation, n=1,318, compared with 45.3% in the panel, n=422. Brookfield supplied 38.0% of consultation responses but accounts for 12% of district households. The consultation was open and self-selected.

  3. A larger bin alone is unlikely to bring most opponents round.

    Medium confidenceBase 191

    Of the 191 panel respondents who opposed at Q2, 22.5% said they would support the change with a free larger bin and 61.3% would still oppose it. Approximately 43 people, or 10.2% of all 422 panel respondents, were opponents who said they would support under that condition.

  4. Test targeted support, particularly for households with nappy or hygiene waste, rather than assuming a larger bin addresses every concern.

    Medium confidenceBase 54

    Among 54 panel respondents with a child under 3 or someone using hygiene products, 74.1% opposed the basic proposal and 55.6% preferred three-weekly collections with extra support. The latter option included examples of either larger bins or extra hygiene-waste collections; it did not identify which support they need.

  5. A trial is a defensible next step, not evidence that residents have endorsed rollout.

    Medium confidenceBase 422

    The panel's first preferences were a trial 25.4%, keeping fortnightly collections 24.6%, three-weekly with support 23.7%, and district-wide three-weekly collections 22.0%, n=422. These close figures do not establish a clear winning option.

  6. Do not assume the change will deliver the recycling improvement residents anticipate.

    Medium confidenceBase 422

    In the panel, 40.5% said they were very or fairly likely to recycle more, while 11.8% said they already recycled everything they could, n=422. These are stated intentions, not measured behaviour.

  7. The observed tenure difference warrants monitoring but is not a reliable basis for claiming different levels of support.

    Medium confidenceBase 422

    Panel support was 42.6% among owner-occupiers, n=282, and 37.1% among renters, n=140. A conventional two-proportion test gives approximately p=0.28; the panel's selection limits also apply.

  8. Treat capacity, hygiene, dumping and value-for-money concerns as operational questions to test, not as established outcomes of a three-weekly service.

    Medium confidenceBase 422

    Both sources contain accounts of bins already close to full, concerns about nappy or hygiene waste and anticipated dumping. Nine panel open-text answers mention smell, flies, maggots, vermin or nappy or hygiene waste sitting in the bin. Other panel respondents anticipated recycling more or little household impact.

Caveats

  • The open consultation is a record of participating residents' views, not a representative district vote. It and the panel should remain separate.
  • Keep Brookfield Tidy promoted the link and suggested wording. The count of 209 covers copies and close variants of the recurring template in the supplied comments; classifying near-copies involves judgement. Similar wording does not prove duplicate submissions or invalidate respondents' views.
  • The panel was invited from randomly selected addresses and matched on age and tenure, but only 422 of 2,000 invited addresses produced completes. Online participation, nonresponse and quota selection can introduce bias. The 21.1% is a completion rate against addresses invited, not an eligibility-adjusted survey response rate.
  • The 4.7-point margin and tenure significance test are conventional simple-random-sample approximations, not assurances of panel accuracy. The panel's age and tenure match does not establish representativeness on other characteristics.
  • The panel's 54-person child-or-hygiene subgroup and especially small ward subgroups have substantial uncertainty. Wendle Vale has only 16 panel respondents; do not choose pilot areas solely from ward percentages.
  • Q3 was asked only of Q2 opponents and concerns a larger bin for the respondent's own household. It cannot establish support for a district-wide package, for extra hygiene collections, or for the proposal among those not asked.
  • Q4 asks for a single preferred option; a trial and a supported rollout are distinct choices. Q2 support for the basic proposal and Q4 preference are not interchangeable.
  • The listed themes are explicitly limited to panel open-text answers. Comments in both sources illustrate concerns and possible benefits, but their frequency cannot be generalised to households. Anticipated impacts are not evidence that those impacts would occur.

Next steps

  • decisionApprove a pilot design rather than immediate district-wide rollout; publish its duration, support offer, costs, comparison arrangements and stop or rollout criteria before launch.
  • experimentPilot across suitably selected town and rural locations, including varied housing and household needs; include a summer period and comparable fortnightly-collection areas where feasible.
  • researchCo-design and test access to larger bins and separate nappy or hygiene-waste collections with affected households, carers, renters and larger households; assess equality and accessibility impacts.
  • monitorMeasure general waste and recycling tonnages and contamination, overflowing bins, side waste, missed collections, complaints, hygiene incidents, fly-tipping and full service costs in pilot and comparison areas.
  • researchCommission a further address-based survey with accessible offline response routes and follow-up of nonrespondents, alongside interviews with underrepresented and affected households.

Open-text themes it coded

Residents' panel: household bin already full or nearly full 24Residents' panel: hygiene, smells, flies, maggots, vermin or stored nappy and hygiene waste 9Residents' panel: value for money or council tax concern 15Residents' panel: anticipated fly-tipping or dumping 6Residents' panel: positive view of recycling more or reducing waste 42

The survey it planned

2 screening questions and 10 questions, as the model wrote them.

  1. S1

    Are you aged 18 or over?

    One answer
    • Yes
    • No

    Continues if Yes

  2. S2

    Do you live in the Hollinbrook District Council area?

    One answer
    • Yes
    • No
    • Not sure

    Continues if Yes

  3. Q1

    What is your full home postcode? We will use it to identify your ward for analysis. If you do not know it or do not want to provide it, type 'not provided'.

    Open text
  4. Q2

    Which age group are you in?

    One answer
    • 18–24
    • 25–34
    • 35–44
    • 45–54
    • 55–64
    • 65–74
    • 75 or over
    • Prefer not to say
  5. Q3

    Does your household own or rent your home?

    One answer
    • Owns outright
    • Owns with a mortgage or loan
    • Shared ownership
    • Rents from a council or housing association
    • Rents privately
    • Lives rent-free or another arrangement
    • Prefer not to say
  6. Q4

    The council is considering collecting general waste in the black bin every three weeks instead of every two weeks. Food waste would still be collected weekly and recycling every two weeks. The council estimates that the change would save about £600,000 a year towards the £3.1 million of savings it needs to make next year. How much do you support or oppose changing to three-weekly general waste collections?

    One answer
    • Strongly support
    • Tend to support
    • Neither support nor oppose
    • Tend to oppose
    • Strongly oppose
    • Don't know
    Judges: leading

    Immediately before asking for support, the question highlights financial savings and the council's savings requirement without comparable information about potential disadvantages. This frames the proposal in favour of support.

    The preamble gives only the case for the change: £600,000 saved towards £3.1 million of savings the council 'needs to make', which presents the need as fact. It gives no balancing information on drawbacks for households. This one-sided framing pushes respondents towards support on the key support measure.

  7. Q5

    Which of the following apply to your household? Select all that apply.

    Any that apply
    • A child under 5 lives here
    • Someone here uses continence or other hygiene products that are put in the general waste bin
    • Five or more people live here
    • None of these apply
    • Prefer not to say
  8. Q6

    If general waste were collected every three weeks rather than every two, what effects, if any, do you think this would have on your household? Select all that apply. If you select 'No effect', 'Don't know' or 'Prefer not to say', please select it on its own.

    Any that apply
    • Our general waste bin would be too full
    • Storing waste would cause smells or hygiene concerns
    • Managing nappies, continence or other hygiene waste would be more difficult
    • We would have difficulty storing waste because of limited space
    • We would need to take waste elsewhere more often
    • We would find it easier to manage our waste
    • Another effect
    • No effect
    • Don't know
    • Prefer not to say
  9. Q7

    If general waste were collected every three weeks, how do you think the amount your household puts out for recycling would change?

    One answer
    • We would put out more for recycling
    • We would put out about the same amount
    • We would put out less for recycling
    • Don't know
  10. Q8

    Suppose general waste collections changed to every three weeks, with extra support for households that need it, such as a larger bin or extra collections of nappies and hygiene waste. How much would you support or oppose the change with this support available?

    One answer
    • Strongly support
    • Tend to support
    • Neither support nor oppose
    • Tend to oppose
    • Strongly oppose
    • Don't know
  11. Q9

    Which one of these options would you prefer the council to choose?

    One answer
    • Change to three-weekly general waste collections across the whole district
    • Change to three-weekly general waste collections across the whole district, with extra support for households that need it
    • Pilot three-weekly general waste collections in some areas first
    • Drop the proposal and keep general waste collections every two weeks
    • Don't know
  12. Q10

    Is there anything else you would like the council to know about the proposed change or support your household might need? Please do not include names or other information that could identify you or someone else. You may leave this blank.

    Open text
Sample plan

Use the same questionnaire and wording for both routes. Invite adults living in the district to the open consultation through the council website, social media and residents' magazine. Separately, invite the existing residents' panel recruited by letter from randomly selected district addresses, targeting about 420 completes. Monitor age, tenure and ward during fieldwork and use proportionate reminders or recruitment to address gaps; document any quotas and selection procedures. Record the survey route separately from answers, discourage repeat submissions and report removals. Use postcodes to assign wards where possible and report missing or unassignable postcodes.

The requested ward precision is not achievable with 420 completes: even 70 completes in each ward gives an approximate 95% margin of error of ±11.7 percentage points at 50%, under simple random sampling. Approximately 267 completes per ward, or 1,602 across six wards, are needed for ±6 points under those assumptions; weighting, clustering and nonresponse may require more. If ±6 points is essential, expand recruitment and set a minimum of about 267 usable completes per ward, allowing for design effects and missing ward data. Otherwise agree to report the wider ward uncertainty and avoid claiming the requested precision. Margins of error do not account for nonresponse bias.