Benchmarks / SurveyBench / What Hollinbrook residents think of three-weekly bin collections / Gemini 3.5 Flash Lite

Measured by Spring Prompt

Gemini 3.5 Flash Lite: What Hollinbrook residents think of three-weekly bin collections

The decision: whether to move general waste collection to every three weeks across the whole district, to do so with extra support for households that need it (a larger bin, or extra collections of nappies and hygiene waste), to pilot it in some areas first, or to drop the proposal. The brief and the data →

Verdict
✗ Not sound
Research score
68 of 100
Analysis rating
662
Head to head, this task
won 5 of 9

Why it is not sound

The survey

  • Presumed the stakeholder's view instead of testing it
  • A research objective the survey could not answer
  • A research objective no question serves
  • Ignored the precision the brief asked for, which the sample cannot deliver
  • A leading question: Q1

The analysis

  • Got at least one set number wrong
  • Missed a trap: The council leader wants to show that residents support the change. Support is 41% on the panel (with 45% opposed) and 15% in the open consultation.

Scorecard

Numbers 9 of 11 right

QuestionIts answerTrue
What percentage of open-consultation respondents tend to oppose or strongly oppose collecting general waste every three weeks (Q2)?✓77.5%77.5%
What percentage of residents' panel respondents tend to oppose or strongly oppose collecting general waste every three weeks (Q2)?✓45.3%45.3%
What percentage of residents' panel respondents strongly support or tend to support collecting general waste every three weeks (Q2)?✓40.8%40.8%
What is the 95% margin of error, in percentage points, on the share of residents' panel respondents who support the change (strongly or tend to)?✓4.7 points4.69 points
On the residents' panel, is the difference in the share supporting the change (strongly or tend to) between owner-occupiers and renters statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Among residents' panel respondents whose household includes a child under 3 or someone who uses continence or other hygiene products, what percentage chose 'every three weeks, with extra support for households that need it' as their preferred option (Q4)?✓55.6%55.6%
How many of the open-text answers are copies or near-copies of the template comment circulated by a local group?✗139209
How many of the open-text answers from residents' panel respondents mention hygiene: smell, flies, maggots or vermin, or nappies or hygiene waste sitting in the bin?✓119
As a percentage of all residents' panel respondents, how many oppose the change at Q2 and said they would support it if their household could have a larger bin at no extra charge (Q3)?✗4.7%10.2%
What percentage of residents' panel respondents said they would be very or fairly likely to recycle more if general waste were collected every three weeks (Q5)?✓40.5%40.5%
What percentage of the addresses invited by letter to the residents' panel produced a completed questionnaire?✓21.1%21.1%

Traps in the data 7 of 8 handled

  • ✓The open consultation is self-selected: anyone with the link could respond, Brookfield is 38% of it against 12% of households, and households with a young child or hygiene needs are over-represented. It shows 78% opposed, against 45% on the panel, which matches the district's age and tenure profile.
  • ✓209 open-consultation comments (16% of the consultation's responses) are copies or near-copies of a template circulated by Keep Brookfield Tidy.
  • ✗The council leader wants to show that residents support the change. Support is 41% on the panel (with 45% opposed) and 15% in the open consultation.
  • ✓Q3 was asked only of those who oppose: 22% of panel opponents would support the change with a larger bin, which is only 10% of all panel respondents; most opponents would still oppose.
  • ✓Wendle Vale (panel base 16) shows 62% support, far above any other ward.
  • ✓On the panel, owner-occupiers support the change more than renters (43% against 37%, bases 282 and 140), but the gap is not statistically significant at 95%.
  • ✓40% of the panel say they would be likely to recycle more: a stated intention about a hypothetical change.
  • ✓Only 422 of 2,000 addresses invited to the panel completed (about 21%); the panel is matched on age and tenure only, and those who replied may care more about bins than those who did not.

The analysis

Recommendation

Do not drop the proposal and do not implement a blanket district-wide rollout immediately. Instead, proceed with a phased pilot in selected areas combined with comprehensive support packages (larger bins and dedicated nappy/hygiene collections) for qualifying households.

What would change it

If a properly weighted and verified consultation or pilot showed net opposition exceeding 60% across the representative panel, or if trial operations demonstrated unmanageable increases in fly-tipping and pest infestation, the proposal should be dropped.

Findings

  1. Opinion is starkly divided between representative and unrepresentative channels: the open consultation shows overwhelming opposition (77.5%), heavily driven by an organised local campaign in Brookfield, whereas the statistically representative residents' panel is much more balanced, with a plurality opposed (45.3%) but significant support (40.8%).

    High confidenceBase 1740

    Open consultation base 1,318, net oppose 77.5% (strongly oppose 58.0%, tend to oppose 19.4%). Residents' panel base 422, net oppose 45.3% (strongly oppose 28.2%, tend to oppose 17.1%), net support 40.8% (strongly support 17.3%, tend to support 23.5%).

  2. Households with children under 3 or users of hygiene and continence products show high levels of opposition and specific operational concerns, yet strongly prefer supported three-weekly options over blanket implementation or status quo.

    High confidenceBase 54

    On the residents' panel, households with a child under 3 or hygiene product users show 74.1% net opposition to three-weekly collections (Base 54). However, when offered collection choices (Q4), 55.6% of this group prefer three-weekly collections with extra support (larger bin or extra nappy collections), compared to 25.9% who want to keep fortnightly collections.

  3. Offering a larger bin alone does not win over the vast majority of opponents.

    High confidenceBase 1212

    Among respondents who oppose three-weekly collections, 66.5% in the open consultation (Base 1,021) and 61.3% on the residents' panel (Base 191) state they would 'still oppose' even if provided with a larger general waste bin at no extra charge.

  4. A significant minority of residents recognise environmental benefits and report willingness to change behaviour, with around 40% on the residents' panel stating they are likely to recycle more under three-weekly collections.

    High confidenceBase 422

    Residents' panel Q5 shows 40.5% net likely to recycle more (very likely 15.6%, fairly likely 24.9%), while 11.8% already recycle everything they can (Base 422).

Caveats

  • The open consultation data is highly skewed by an organised campaign ('Keep Brookfield Tidy') which submitted 139 near-identical template responses, invalidating raw totals for gauging general public sentiment.
  • The residents' panel achieved a 21.1% response rate (422 completions from 2,000 invitations), introducing potential non-response bias despite weighting/quota-filling on age and tenure.
  • Survey stated preferences (e.g. Q3 and Q4) may overstate actual uptake of mitigation measures or behavioral adjustments compared to observed operational realities.

Next steps

  • decisionBrief the Council Leader, Cabinet Member for Environment, and Head of Waste Services on the divergence between unweighted consultation and representative panel data.
  • experimentDesign and launch a limited trial of three-weekly general waste collections in designated pilot areas.
  • researchEstablish an eligibility and delivery mechanism for extra support (larger bins and nappy/hygiene collections) targeting households with children under 3 or medical waste users.
  • monitorMonitor contamination levels in recycling and food waste streams, as well as local fly-tipping reports.

Open-text themes it coded

Capacity overflow and bin size 63Smell, flies, maggots and hygiene 238Vermin, pests and wildlife disruption 220Fly-tipping and dumped waste 220Council tax value and fairness 36Environmental benefit and recycling motivation 57

The survey it planned

3 screening questions and 3 questions, as the model wrote them.

  1. S1

    Are you aged 18 or over?

    One answer
    • Yes
    • No

    Continues if Yes

  2. S2

    Do you live within the Hollinbrook District Council area?

    One answer
    • Yes
    • No

    Continues if Yes

  3. S3

    Which ward do you live in?

    One answer
    • Hollinbrook Central
    • North Rural
    • East Valley
    • Westbourne
    • Southfields
    • Oakhaven
    • Don't know / Not sure

    Continues if Hollinbrook Central; North Rural; East Valley; Westbourne; Southfields; Oakhaven; Don't know / Not sure

  4. Q1

    Currently, general waste (black bins) is collected every two weeks, food waste is collected weekly, and recycling is collected every two weeks. Before today, how much, if anything, did you know about the council's proposal to change general waste collections to every three weeks to save £600k a year towards a £3.1m budget shortfall?

    One answer
    • Knew a lot
    • Knew a little
    • Had heard about it, but knew nothing detail
    • Didn't know anything about it until this survey
    Judges: leading

    The proposal is introduced with its claimed savings and the council's budget shortfall immediately before the support question, framing the change as financially necessary and priming support.

    The awareness question states the proposal's justification as fact ('to save £600k a year towards a £3.1m budget shortfall'). It gives only the benefit and no downsides, which primes respondents just before the support question.

  5. Q2

    To what extent do you support or oppose moving general waste (black bin) collections from every two weeks to every three weeks across the district, while keeping food waste weekly and recycling fortnightly?

    Scale

    Scale 1-5: Strongly oppose to Strongly support

  6. Q3

    Which of the following, if any, apply to your household?

    Any that apply
    • We regularly fill our general waste (black bin) completely before collection day
    • We sometimes struggle to fit all our general waste in the bin
    • We have extra bags of rubbish that do not fit in the bin
    • None of these apply