Benchmarks / SurveyBench / What Hollinbrook residents think of three-weekly bin collections / Gemini 3.8 Flash

Measured by Spring Prompt

Gemini 3.8 Flash: What Hollinbrook residents think of three-weekly bin collections

The decision: whether to move general waste collection to every three weeks across the whole district, to do so with extra support for households that need it (a larger bin, or extra collections of nappies and hygiene waste), to pilot it in some areas first, or to drop the proposal. The brief and the data →

Verdict
✗ Not sound
Research score
74 of 100
Analysis rating
898
Head to head, this task
won 7 of 10

Why it is not sound

The survey

  • Presumed the stakeholder's view instead of testing it
  • A leading question: Q3
  • A leading question: Q7
  • A double-barrelled question: Q5
  • A double-barrelled question: Q9

The analysis

  • Reported a material finding the data does not support
  • Missed a trap: 40% of the panel say they would be likely to recycle more: a stated intention about a hypothetical change.
  • Missed a trap: Only 422 of 2,000 addresses invited to the panel completed (about 21%); the panel is matched on age and tenure only, and those who replied may care more about bins than those who did not.

Findings the judges found unsupported

  • The Cabinet Member's instinct to drop the proposal based on 77.5% consultation opposition is flawed: the open consultation is heavily skewed by geographic self-selection and an organised local campaign.The 71.1% calculation combines distinct preferences, including a trial followed by a decision. It does not establish that 71.1% favour implementing three-weekly collections, and a trial is not necessarily a phased rollout.
  • The Head of Waste Services' hypothesis that larger bins would win over opponents is incorrect: offering larger bins would fail to flip most opponents and risks defeating the waste-reduction objective.The survey does not establish that larger bins would disincentivise recycling or defeat waste reduction. Adding converted opponents to existing supporters also assumes existing supporters would retain their support; Q3 did not test that.
  • Opposition is heavily concentrated among households with infants or medical hygiene needs, but targeted service support neutralises their resistance.The 55.6% preference for the supported option is supported, but it does not demonstrate that support neutralises resistance. Q4 measures a preferred option rather than a change in opposition, and the subgroup has only 54 respondents. It also does not isolate the effect of tailored collections from the other support examples.
  • The policy has strong potential to drive recycling behaviour across the district.Hypothetical intentions to recycle more do not establish strong district-wide behavioural effects. The finding is marked high confidence without acknowledging that actual recycling changes must be measured.

Scorecard

Numbers 11 of 11 right

QuestionIts answerTrue
What percentage of open-consultation respondents tend to oppose or strongly oppose collecting general waste every three weeks (Q2)?✓77.5%77.5%
What percentage of residents' panel respondents tend to oppose or strongly oppose collecting general waste every three weeks (Q2)?✓45.3%45.3%
What percentage of residents' panel respondents strongly support or tend to support collecting general waste every three weeks (Q2)?✓40.8%40.8%
What is the 95% margin of error, in percentage points, on the share of residents' panel respondents who support the change (strongly or tend to)?✓4.7 points4.69 points
On the residents' panel, is the difference in the share supporting the change (strongly or tend to) between owner-occupiers and renters statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Among residents' panel respondents whose household includes a child under 3 or someone who uses continence or other hygiene products, what percentage chose 'every three weeks, with extra support for households that need it' as their preferred option (Q4)?✓55.6%55.6%
How many of the open-text answers are copies or near-copies of the template comment circulated by a local group?✓209209
How many of the open-text answers from residents' panel respondents mention hygiene: smell, flies, maggots or vermin, or nappies or hygiene waste sitting in the bin?✓99
As a percentage of all residents' panel respondents, how many oppose the change at Q2 and said they would support it if their household could have a larger bin at no extra charge (Q3)?✓10.2%10.2%
What percentage of residents' panel respondents said they would be very or fairly likely to recycle more if general waste were collected every three weeks (Q5)?✓40.5%40.5%
What percentage of the addresses invited by letter to the residents' panel produced a completed questionnaire?✓21.1%21.1%

Traps in the data 6 of 8 handled

  • ✓The open consultation is self-selected: anyone with the link could respond, Brookfield is 38% of it against 12% of households, and households with a young child or hygiene needs are over-represented. It shows 78% opposed, against 45% on the panel, which matches the district's age and tenure profile.
  • ✓209 open-consultation comments (16% of the consultation's responses) are copies or near-copies of a template circulated by Keep Brookfield Tidy.
  • ✓The council leader wants to show that residents support the change. Support is 41% on the panel (with 45% opposed) and 15% in the open consultation.
  • ✓Q3 was asked only of those who oppose: 22% of panel opponents would support the change with a larger bin, which is only 10% of all panel respondents; most opponents would still oppose.
  • ✓Wendle Vale (panel base 16) shows 62% support, far above any other ward.
  • ✓On the panel, owner-occupiers support the change more than renters (43% against 37%, bases 282 and 140), but the gap is not statistically significant at 95%.
  • ✗40% of the panel say they would be likely to recycle more: a stated intention about a hypothetical change.
  • ✗Only 422 of 2,000 addresses invited to the panel completed (about 21%); the panel is matched on age and tenure only, and those who replied may care more about bins than those who did not.

The analysis

Recommendation

Do not roll out district-wide three-weekly collections immediately, and do not drop the proposal. Instead, implement a 6- to 9-month phased pilot of three-weekly collections in 1-2 representative wards, incorporating dedicated weekly or fortnightly nappy and hygiene waste collections and targeted bin upsizing for large households (5+ occupants). A phased trial is the single most preferred option among the representative panel (25.4%), commands broad consent when paired with hygiene support, and provides the operational evidence needed to disprove fly-tipping and vermin fears before full rollout.

What would change it

The recommendation would shift to dropping the proposal if a pilot reveals substantial uncontained side-waste, persistent pest contamination, or sustained increases in local fly-tipping that mitigation measures cannot resolve. Conversely, the recommendation would shift to immediate district-wide rollout (with hygiene support) if waste compositional audits prove that >75% of residual black bin volume in typical households consists of recyclable materials or unsegregated food waste that weekly caddies easily absorb.

Findings

  1. The Council Leader cannot claim majority backing: district residents are divided in a statistical dead-heat between opposition and support, and an outright district-wide rollout without support has very low appeal.

    High confidenceBase 422

    On the representative residents' panel, 40.8% support vs 45.3% oppose the change at Q2 (margin of error +/-4.7 percentage points, making them statistically indistinguishable). When asked for their most preferred option at Q4, only 22.0% of the panel chose moving to three-weekly across the whole district without support.

  2. The Cabinet Member's instinct to drop the proposal based on 77.5% consultation opposition is flawed: the open consultation is heavily skewed by geographic self-selection and an organised local campaign.

    High confidenceBase 1740

    Brookfield represents 38.0% of consultation respondents despite being only 12% of the district's households, and 209 open-text comments (15.9% of all consultation responses and 50.1% of all open-text comments) were exact or near-exact copies of a template circulated by Keep Brookfield Tidy. On the demographically matched panel, only 24.6% want to keep fortnightly collections, meaning 71.1% favour moving to three-weekly collections in some form (22.0% district-wide, 23.7% with extra support, 25.4% via a phased trial).

  3. The Head of Waste Services' hypothesis that larger bins would win over opponents is incorrect: offering larger bins would fail to flip most opponents and risks defeating the waste-reduction objective.

    High confidenceBase 422

    At Q3, among panel opponents, 61.3% would still oppose three-weekly collections even with a free larger bin; only 22.5% would switch to support. Across the total panel, offering larger bins brings over only 10.2% of residents, leaving net support under 51% while providing increased landfill capacity that disincentivises recycling.

  4. Opposition is heavily concentrated among households with infants or medical hygiene needs, but targeted service support neutralises their resistance.

    High confidenceBase 54

    Panel households with under-3s or hygiene products oppose three-weekly collections overwhelmingly at baseline (74.1% oppose, 44.4% strongly oppose). However, when offered tailored support at Q4, 55.6% choose 'three-weekly with extra support' as their preferred policy, while only 25.9% prefer retaining fortnightly collections.

  5. The policy has strong potential to drive recycling behaviour across the district.

    High confidenceBase 422

    40.5% of the panel report they are very or fairly likely to recycle more under three-weekly collections, and another 11.8% state they already recycle everything possible. Only 29.1% say they are unlikely to recycle more.

Caveats

  • Open consultation figures cannot be used to infer district-wide public opinion: respondents self-selected, Brookfield ward was overrepresented by more than three-fold (38.0% vs 12.0% pop share), and an active campaign generated over 200 coordinated template responses.
  • The residents' panel sample size (N=422) carries an overall margin of error of +/-4.7 percentage points at the 95% confidence level, meaning the difference between net support (40.8%) and net opposition (45.3%) is not statistically significant.
  • Subgroup analysis for households with children under 3 or hygiene waste users within the panel relies on a small base (n=54), giving wide confidence intervals (+/-13.3 percentage points), though the directional preference for extra support over fortnightly retention is decisive.
  • Open-text answers may reflect heightened summer anxiety regarding odour and pests, as fielding occurred in May and June.

Next steps

  • experimentDesign and launch a pilot in two contrasting wards (one suburban/town, one rural) testing three-weekly black bin collections paired with an opt-in hygiene collection service.
  • decisionDevelop clear qualification and delivery criteria for opt-in hygiene collections (nappies, incontinence pads) and larger bins for households with 5 or more residents.
  • monitorConduct bi-weekly street-level monitoring of fly-tipping hotspots, alleyway dumping, and side-waste presentation across pilot and control zones.
  • researchExecute mid-pilot and post-pilot survey and focus groups among participating pilot residents, specifically tracking hygiene and capacity satisfaction.

Open-text themes it coded

Coordinated campaign template text (Keep Brookfield Tidy) 209Bin capacity and overflowing bins in larger households 62Hygiene, smell, flies, maggots, and vermin risks 34Council tax, value for money, and perceived cuts to service 39Environmental support and incentive to recycle more 57Fear of increased fly-tipping and roadside dumping 15Minimal impact on small or low-waste households 16

The survey it planned

3 screening questions and 9 questions, as the model wrote them.

  1. S1

    Are you aged 18 or over?

    One answer
    • Yes
    • No

    Continues if Yes

  2. S2

    Do you live within the Hollinbrook District Council area?

    One answer
    • Yes
    • No
    • Don't know

    Continues if Yes

  3. S3

    Which ward in Hollinbrook do you live in? (If unsure, please enter the first part of your postcode below)

    One answer
    • Hollinbrook Central
    • Hollinbrook North & Rural
    • Highgate & Parkside
    • Meadow Vale
    • Stourford & Riverside
    • Westwood & Abbey
    • Not sure / Prefer to enter postcode

    Continues if Hollinbrook Central; Hollinbrook North & Rural; Highgate & Parkside; Meadow Vale; Stourford & Riverside; Westwood & Abbey; Not sure / Prefer to enter postcode

  4. Q1

    Which of the following apply to your household? (Select all that apply)

    Any that apply
    • Children in nappies (aged under 3)
    • Someone who uses medical, continence, or sanitary hygiene products producing regular waste
    • Five or more people living in the home
    • Limited outdoor storage space for bins (e.g. flat, terraced house with no front garden/rear access)
    • None of these apply to my household
  5. Q2

    On average, how full is your black general waste bin when it is collected every two weeks?

    One answer
    • Overflowing (the lid does not close fully / extra bags placed alongside)
    • Completely full (lid closes)
    • About three-quarters full
    • About half full
    • Less than half full
    Judges: assumptive

    Assumes every eligible household uses a black bin collected fortnightly and can assess its fullness. Residents using communal bins or other arrangements have no applicable response.

    Assumes every respondent has their own black bin collected fortnightly. Residents of flats who use communal bins, a group Q1 explicitly anticipates, cannot answer meaningfully.

  6. Q3

    The council is proposing to collect black general waste bins every three weeks instead of every two weeks, to help save £600k per year and encourage more recycling. Food waste would continue to be collected every week, and dry recycling every two weeks. To what extent do you support or oppose moving general waste collections to every three weeks?

    Scale

    Scale 1-5: Strongly oppose to Strongly support

    Judges: leading

    Immediately frames the proposal with financial and recycling benefits, without corresponding drawbacks, before measuring support.

    The preamble presents only benefits: saving £600k and, stated as fact, encouraging more recycling. It gives no drawbacks just before asking for support, which steers respondents towards supporting the change.

  7. Q4

    If black general waste bins were collected every three weeks, what impact, if any, would this have on your household? (Select up to three main concerns)

    Any that apply
    • No noticeable impact / manageable for our household
    • Not enough space in the black bin for our general waste
    • Smells or hygiene concerns (e.g. from nappies or incontinence products)
    • Smells or pest issues from general rubbish
    • Difficulty storing additional waste outside
    • Confusion about collection dates / calendar changes
    • Other
  8. Q5

    If general waste collections moved to every three weeks, how likely would your household be to increase the amount of food waste and dry recycling you separate?

    Scale

    Scale 1-5: Much less likely to Much more likely

    Judges: double-barrelled

    Requires one likelihood rating for increasing food-waste separation and dry recycling, although a household's response could differ between the two.

    Combines food waste and dry recycling in one answer. The scale anchors ('Much less likely' to 'Much more likely') do not fit the question 'how likely would your household be to increase', so a respondent could read the scale as likelihood or as direction of change.

  9. Q6

    Would any of the following support measures make a three-weekly general waste collection more acceptable to your household? (Select all that apply)

    Any that apply
    • A free separate, frequent collection for disposable nappies and hygiene/continence waste
    • An option to request a larger general waste bin for households of 5+ or with special needs
    • An extra or larger recycling bin/box at no extra cost
    • Clearer guides on what can and cannot be recycled
    • None of these would make it acceptable to me
    • I already support the change without needing extra support
  10. Q7

    Council members must decide between four possible approaches in the autumn. Which of the following options would you most prefer Hollinbrook District Council to take?

    One answer
    • Option A: Move to three-weekly collections across the whole district as proposed
    • Option B: Move to three-weekly collections, but with extra support (e.g. larger bins or separate hygiene collections) for qualifying households
    • Option C: Pilot three-weekly collections in selected areas first to test how it works before rolling out district-wide
    • Option D: Keep collections every two weeks, and find the £600,000 savings from other council services
    Judges: leading

    Keeping fortnightly collections is explicitly tied to finding savings from other council services, making that option less attractive. The pilot option also presupposes eventual district-wide rollout rather than leaving its outcome open.

    Only Option D (dropping the proposal) carries a stated cost, 'find the £600,000 savings from other council services', which implies cuts elsewhere. Options A–C have no trade-offs attached, which tilts respondents away from D.

  11. Q8

    Please explain the reason for your choice in the previous question, or share any specific suggestions for managing household waste in Hollinbrook.

    Open text
  12. Q9

    What type of home do you live in, and what is your housing tenure?

    One answer
    • House / bungalow - Owned (outright or with mortgage)
    • House / bungalow - Rented (council, housing association, or private)
    • Flat / apartment / maisonette - Owned or rented
    • Other
    Judges: double-barrelled

    Asks for both dwelling type and tenure, but the flat/apartment/maisonette response combines owned and rented homes, so those respondents cannot report their tenure.

    Asks about dwelling type and tenure in one item. Owned and rented flats are merged, and council, housing association and private renting are merged, so tenure cannot be recovered for weighting against the panel's tenure profile.

Sample plan

The research employs two distinct data streams: 1) Probability-based Representative Panel: Direct mail recruitment letters with unique survey access codes sent to randomly sampled addresses from the Royal Mail Postcode Address File across the 6 wards, matched to census benchmarks for age, tenure, and ward. Note on Ward Precision: To achieve ±6.0% margin of error at a 95% confidence level for each of the 6 wards individually, an achieved sample of ~265 completes per ward is mathematically required (total N ≈ 1,590). An overall sample of N = 420 yields ~70 completes per ward, providing ±11.7% precision per ward (though ±4.8% at the district level). We recommend boosting the panel to N=1,600 (approx. 267/ward) if ward-level ±6% precision is a hard legal/political requirement, or reporting ward-level results with clearly stated ±12% confidence intervals if capped at N=420. 2) Open Public Consultation: Unweighted, self-selected web link hosted on the council website and promoted via resident magazines and social media. Data from the two streams will be analyzed and reported separately to avoid self-selection bias skewing representative findings, counteracting political pressure while maintaining consultation transparency.