Benchmarks / SurveyBench / What Hollinbrook residents think of three-weekly bin collections / Mistral Medium 3.5

Measured by Spring Prompt

Mistral Medium 3.5: What Hollinbrook residents think of three-weekly bin collections

The decision: whether to move general waste collection to every three weeks across the whole district, to do so with extra support for households that need it (a larger bin, or extra collections of nappies and hygiene waste), to pilot it in some areas first, or to drop the proposal. The brief and the data →

Verdict
✗ Not sound
Research score
74 of 100
Analysis rating
540
Head to head, this task
won 6 of 11

Why it is not sound

The survey

  • Presumed the stakeholder's view instead of testing it
  • Ignored the precision the brief asked for, which the sample cannot deliver
  • A leading question: Q1

The analysis

  • Got at least one set number wrong
  • Reported a material finding the data does not support
  • Missed a trap: Wendle Vale (panel base 16) shows 62% support, far above any other ward.
  • Proposed a mistaken next step: Start the change in Wendle Vale, or treat Wendle Vale as proof of support, on the strength of its survey result

Findings the judges found unsupported

  • Residents do not support three-weekly general waste collections. In the open consultation, 77.5% oppose the change, and even in the more representative residents' panel, 45.3% oppose while only 40.8% support.The panel is split, not evidence that residents generally reject the change. The stated base of 1,740 also adds two sources that must remain separate; there is no pooled district-opinion estimate.
  • The open consultation was heavily influenced by a campaign. 42 of the 417 open-text answers are copies or near-copies of a template circulated by 'Keep Brookfield Tidy', and 38% of open-consultation respondents are from Brookfield ward (vs. 12% of the district).There are 209 campaign-template comments, not 42. The asserted base of 417 open-text answers is not established by the supplied data. This substantially understates the campaign's contribution.
  • Hygiene concerns are a major driver of opposition. In the residents' panel, 10 open-text answers mention hygiene issues (smell, flies, maggots, vermin, nappies), and 74.1% of households with hygiene needs oppose the change.The supplied panel comments do not support the stated count of 10 hygiene-related answers. The association between household needs and opposition, alongside illustrative comments, also does not establish hygiene as a causal driver.

Scorecard

Numbers 10 of 11 right

QuestionIts answerTrue
What percentage of open-consultation respondents tend to oppose or strongly oppose collecting general waste every three weeks (Q2)?✓77.5%77.5%
What percentage of residents' panel respondents tend to oppose or strongly oppose collecting general waste every three weeks (Q2)?✓45.3%45.3%
What percentage of residents' panel respondents strongly support or tend to support collecting general waste every three weeks (Q2)?✓40.8%40.8%
What is the 95% margin of error, in percentage points, on the share of residents' panel respondents who support the change (strongly or tend to)?✓4.7 points4.69 points
On the residents' panel, is the difference in the share supporting the change (strongly or tend to) between owner-occupiers and renters statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Among residents' panel respondents whose household includes a child under 3 or someone who uses continence or other hygiene products, what percentage chose 'every three weeks, with extra support for households that need it' as their preferred option (Q4)?✓55.6%55.6%
How many of the open-text answers are copies or near-copies of the template comment circulated by a local group?✗42209
How many of the open-text answers from residents' panel respondents mention hygiene: smell, flies, maggots or vermin, or nappies or hygiene waste sitting in the bin?✓109
As a percentage of all residents' panel respondents, how many oppose the change at Q2 and said they would support it if their household could have a larger bin at no extra charge (Q3)?✓11.1%10.2%
What percentage of residents' panel respondents said they would be very or fairly likely to recycle more if general waste were collected every three weeks (Q5)?✓40.5%40.5%
What percentage of the addresses invited by letter to the residents' panel produced a completed questionnaire?✓21.1%21.1%

Traps in the data 7 of 8 handled

  • ✓The open consultation is self-selected: anyone with the link could respond, Brookfield is 38% of it against 12% of households, and households with a young child or hygiene needs are over-represented. It shows 78% opposed, against 45% on the panel, which matches the district's age and tenure profile.
  • ✓209 open-consultation comments (16% of the consultation's responses) are copies or near-copies of a template circulated by Keep Brookfield Tidy.
  • ✓The council leader wants to show that residents support the change. Support is 41% on the panel (with 45% opposed) and 15% in the open consultation.
  • ✓Q3 was asked only of those who oppose: 22% of panel opponents would support the change with a larger bin, which is only 10% of all panel respondents; most opponents would still oppose.
  • ✗Wendle Vale (panel base 16) shows 62% support, far above any other ward.
  • ✓On the panel, owner-occupiers support the change more than renters (43% against 37%, bases 282 and 140), but the gap is not statistically significant at 95%.
  • ✓40% of the panel say they would be likely to recycle more: a stated intention about a hypothetical change.
  • ✓Only 422 of 2,000 addresses invited to the panel completed (about 21%); the panel is matched on age and tenure only, and those who replied may care more about bins than those who did not.

The analysis

Recommendation

Do not implement three-weekly general waste collections across the whole district. Instead, pilot the change in areas with lower opposition and lower concentrations of households with hygiene needs, with the 'extra support' option (larger bins or extra collections for nappies/hygiene waste) available to all households that need it. Monitor the pilot closely for bin overflow, fly-tipping, and hygiene issues before deciding on wider rollout.

What would change it

Evidence that the 'extra support' option (larger bins or extra collections for hygiene waste) reduces opposition among households with hygiene needs to below 50%, and that bin overflow/fly-tipping does not increase significantly in the pilot areas.

Findings

  1. Residents do not support three-weekly general waste collections. In the open consultation, 77.5% oppose the change, and even in the more representative residents' panel, 45.3% oppose while only 40.8% support.

    High confidenceBase 1740

    Open consultation: 77.5% oppose (1318 base). Residents' panel: 45.3% oppose, 40.8% support (422 base).

  2. Opposition is strongest among households with a child under 3 or someone using continence/hygiene products. In the residents' panel, 74.1% of these households oppose the change, compared to 41.0% of others.

    High confidenceBase 422

    Residents' panel: 74.1% oppose among households with child under 3 or hygiene products (54 base), 41.0% oppose among others (368 base).

  3. Offering a larger bin does not convert most opponents. Among residents' panel respondents who oppose the change, only 22.5% would support it if offered a larger bin, while 61.3% would still oppose.

    Medium confidenceBase 191

    Residents' panel: 22.5% would support with larger bin, 61.3% would still oppose (191 base).

  4. Households with hygiene needs overwhelmingly prefer the 'extra support' option. In the residents' panel, 55.6% of these households chose 'every three weeks, with extra support' as their preferred option, compared to 19.0% of others.

    High confidenceBase 422

    Residents' panel: 55.6% prefer 'extra support' among households with child under 3 or hygiene products (54 base), 19.0% among others (368 base).

  5. The open consultation was heavily influenced by a campaign. 42 of the 417 open-text answers are copies or near-copies of a template circulated by 'Keep Brookfield Tidy', and 38% of open-consultation respondents are from Brookfield ward (vs. 12% of the district).

    High confidenceBase 1318

    Open consultation: 42 campaign comments (417 base), 38.0% from Brookfield (1318 base). District: 12% of households in Brookfield.

  6. The residents' panel is more representative but still underrepresents households with hygiene needs. Only 12.8% of panel respondents have a child under 3 or hygiene products (vs. ~15% district estimate), and 21.1% of invited households responded.

    High confidenceBase 422

    Residents' panel: 7.1% child under 3, 5.9% hygiene products (422 base). District estimate: ~15%. Response rate: 21.1% (422/2000).

  7. Three-weekly collections may increase recycling. In the residents' panel, 40.5% said they would be very or fairly likely to recycle more, and many open-text answers cite this as a benefit.

    Medium confidenceBase 422

    Residents' panel: 40.5% likely to recycle more (422 base). Open-text: multiple respondents mention recycling more as a positive.

  8. Hygiene concerns are a major driver of opposition. In the residents' panel, 10 open-text answers mention hygiene issues (smell, flies, maggots, vermin, nappies), and 74.1% of households with hygiene needs oppose the change.

    High confidenceBase 422

    Residents' panel: 10 hygiene-related open-text answers (422 base), 74.1% oppose among households with hygiene needs (54 base).

Caveats

  • The open consultation is not representative: 38% of respondents are from Brookfield ward (vs. 12% of the district), and 42 comments are copies of a campaign template. It overstates opposition.
  • The residents' panel underrepresents households with hygiene needs (12.8% vs. ~15% district estimate) and has a 21.1% response rate, which may bias results.
  • Opposition is driven by practical concerns (bin capacity, hygiene) and perceived value for money, not just principle. These are harder to address with communication alone.
  • The margin of error on the residents' panel support figure is ±4.7 percentage points, so the true support could be as low as 36.1% or as high as 45.5%.
  • Households with hygiene needs are a small but critical group. Their opposition is very high (74.1%), and they strongly prefer the 'extra support' option (55.6%).

Next steps

  • experimentRun a pilot in 1-2 wards with lower opposition (e.g., Wendle Vale, Kettlewell) and offer the 'extra support' option to all households in the pilot area.
  • researchConduct a targeted survey of households with hygiene needs (child under 3 or continence products) to understand their specific concerns and whether the 'extra support' option would address them.
  • researchModel the financial and environmental impact of the 'extra support' option (larger bins, extra collections) to ensure it is cost-effective and delivers net benefits.
  • monitorMonitor bin overflow, fly-tipping, and hygiene complaints in the pilot areas and compare to control areas.
  • decisionCommunicate the pilot's purpose and the 'extra support' option clearly to all residents in the pilot areas, with a feedback mechanism.

Open-text themes it coded

Overflowing bins and fly-tipping 390Hygiene concerns (smell, flies, maggots, vermin, nappies) 62Value for money concerns (council tax, service cuts) 153Recycling more as a positive 78No impact or minimal impact 59

The survey it planned

2 screening questions and 10 questions, as the model wrote them.

  1. S1

    Are you aged 18 or over?

    One answer
    • Yes
    • No

    Continues if Yes

  2. S2

    Do you currently live in the Hollinbrook District Council area?

    One answer
    • Yes
    • No

    Continues if Yes

  3. Q1

    Hollinbrook District Council is considering changing how often general waste (the black bin) is collected. Currently, general waste is collected every two weeks. The council is proposing to change this to every three weeks to save money and encourage more recycling. Food waste would still be collected weekly, and recycling every two weeks. How much do you support or oppose this change?

    Scale

    Scale 1-5: Strongly oppose to Strongly support

    Judges: leading

    Presenting saving money and encouraging recycling as the reasons for the proposal gives respondents a positive, one-sided framing before asking about support.

    The preamble gives only the council's case for the change ('to save money and encourage more recycling'). It states as fact that the change will encourage recycling and mentions no downsides, which pushes respondents towards support.

  4. Q2

    Which of the following best describes your household?

    One answer
    • No children under 5 and no one uses continence or hygiene products (e.g., nappies, incontinence pads)
    • Household includes children under 5
    • Household includes someone who uses continence or hygiene products (e.g., nappies, incontinence pads)
    • Household includes both children under 5 and someone who uses continence or hygiene products
  5. Q3

    If general waste collections changed to every three weeks, how would this affect your household?

    Any that apply
    • We would have no problem managing our waste
    • We would struggle to manage our waste
    • We would need a larger general waste bin
    • We would need extra collections for nappies or hygiene waste
    • We would recycle more to reduce general waste
    • We would use other disposal methods (e.g., burning, fly-tipping)
    • Other (please specify)
  6. Q4

    The council is considering different options for the change. Which option do you prefer?

    One answer
    • Move to three-weekly general waste collections across the whole district
    • Move to three-weekly collections but provide extra support (e.g., larger bins, extra collections for nappies/hygiene waste) for households that need it
    • Pilot three-weekly collections in some areas first to see how it works
    • Do not change the current two-weekly general waste collection
  7. Q5

    If the council provided extra support such as a larger general waste bin or extra collections for nappies and hygiene waste, would this change your view on moving to three-weekly collections?

    One answer
    • Yes, I would support the change with extra support
    • No, I would still oppose the change
    • I am unsure

    Routing Shown only if Q4 is: Do not change the current two-weekly general waste collection

  8. Q6

    Do you think moving to three-weekly general waste collections would encourage you to recycle more?

    One answer
    • Yes, definitely
    • Yes, probably
    • No, probably not
    • No, definitely not
    • I already recycle as much as I can
  9. Q7

    How often do you currently put out your general waste (black bin) for collection?

    One answer
    • Every two weeks (when it is collected)
    • Less often than every two weeks
    • More often than every two weeks (e.g., I put it out even when it is not full)
    Judges: ambiguous

    It assumes respondents personally put out a black bin, excluding situations such as communal bins. The 'more often than every two weeks' option conflicts with the stated collection schedule, and its example about bin fullness does not explain greater frequency.

    Collection is every two weeks, so a bin cannot be put out more often than that. The example given for 'more often' (putting it out when not full) describes something else entirely. The options are incoherent, and respondents will interpret them differently.

  10. Q8

    Which ward do you live in?

    One answer
    • Ward A
    • Ward B
    • Ward C
    • Ward D
    • Ward E
    • Ward F
    • I don't know
  11. Q9

    What is your age group?

    One answer
    • 18-24
    • 25-34
    • 35-44
    • 45-54
    • 55-64
    • 65-74
    • 75+
    • Prefer not to say
  12. Q10

    What is your housing tenure?

    One answer
    • Owned (with or without a mortgage)
    • Rented (from the council or housing association)
    • Rented (from a private landlord)
    • Other
    • Prefer not to say
Sample plan

Two samples: (1) Open consultation: promoted via council website, social media, and residents' magazine, open to any adult resident in the district (no fixed target). (2) Residents' panel: 420 completes, recruited by letter to randomly selected addresses, matched to the district's age and tenure profile. Quotas for the panel: 70 completes per ward (6 wards), with age and tenure quotas to match district demographics.