Sign inTest your prompt

Benchmarks / SurveyBench / Should Corefit gyms stop opening 24 hours? / Gemini 4 Carbon

Measured by Spring Prompt

Gemini 4 Carbon: Should Corefit gyms stop opening 24 hours?

The decision: whether to close all 14 gyms between 11pm and 5am, close overnight only at the quietest gyms, or keep every gym open 24 hours. The brief and the data →

Verdict
✗ Not sound
Research score
84 of 100
Analysis rating
1,076
Head to head, this task
won 5 of 9

Why it is not sound

The survey

  • Answer bands that overlap or leave gaps
  • A double-barrelled question: Q8

The analysis

  • Reported a material finding the data does not support
  • Missed a trap: Only current members were surveyed: people who joined for 24-hour access and have since left, and people who would join only because of it, are missing, so the survey cannot show what 24-hour opening does for recruitment or the brand; and night-shift workers are a small group whose answers carry little weight in the totals even though they depend on the hours.

Findings the judges found unsupported

  • To the Marketing Director: unweighted cancellation intent (14.6%) overstates likely churn due to non-response and hypothetical-intent bias, but realistic network-wide churn risk still spans roughly 1,000 to 4,400 members—amplified by rolling contracts, nearby 24/7 rivals, and demands for a fee cut among daytime users.The claimed realistic churn range of 1,000–4,400 members is unsupported. Weighting hypothetical cancellation intentions does not turn them into a forecast, and no evidence establishes the lower bound. The later use of 4,400 lost memberships as a decision threshold repeats this error despite the caveats.

Scorecard

Numbers 11 of 11 right

QuestionIts answerTrue
What percentage of respondents reported at least one visit to a Corefit gym between 11pm and 5am in the last four weeks?✓14.9%14.9%
What percentage of all respondents said they would be likely (fairly or very) to cancel their membership if their home gym closed between 11pm and 5am?✓14.6%14.6%
Among respondents who reported at least one visit between 11pm and 5am in the last four weeks, what percentage said they would be likely (fairly or very) to cancel?✓45.8%45.8%
Weighted so that respondents with at least one visit between 11pm and 5am in the last four weeks count as 6% of members (the door-entry figure) and the rest as 94%, what percentage would be likely (fairly or very) to cancel?✓11.4%11.4%
If the weighted share likely to cancel were applied to all 38,600 members, about how many members would that be?✓4,4004,383
Among respondents who reported at least one visit between 11pm and 5am in the last four weeks, in which band of the Q1 table does the median number of visits fall? Give the lowest value of that band (for example 4 for '4 to 7').✓22
What is the 95% margin of error, in percentage points, on the unweighted share of respondents who reported at least one visit between 11pm and 5am in the last four weeks?✓2.3 points2.27 points
Is the difference between North West England and North East England respondents in the share likely (fairly or very) to cancel statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
How many of the open-text answers say that 11pm to 5am is the only time the respondent can train because they work shifts or nights?✓1616
How many of the open-text answers from respondents who visited between 11pm and 5am in the last four weeks say they would move to another gym?✓1919
What percentage of the members invited completed the survey?✓8%8%

Traps in the data 7 of 8 handled

  • ✓14.9% of respondents report at least one visit between 11pm and 5am in the last four weeks, against 6% of members in the door-entry records: members with a stake in the overnight hours were more likely to respond. Unweighted, 14.6% say they would be likely to cancel; weighted to the 6%, 11.4%.
  • ✓'Likely to cancel' answers (45.8% of overnight users, 14.6% overall) are stated intentions about a hypothetical closure, asked straight after members were invited to think about it.
  • ✓The mean number of overnight visits among overnight users (7.4) is pulled up by night-shift workers who come almost daily; the median is in the '2 to 3' band.
  • ✓The operations director believes almost nobody uses the gyms overnight. Partly true: 6% of members in the door-entry records (about 2,316 people). But night-shift workers (5.9% of respondents) mostly train between 11pm and 5am (77.8% of those who visited overnight usually arrive then) and are the most likely to say they would cancel.
  • ✓At Wakefield Calder Park (base 11), 54.5% of respondents used the gym overnight and 27.3% say they would be likely to cancel.
  • ✓North West England respondents (base 229) look more likely to cancel than North East England respondents (base 220): 19.2% against 13.6%, a gap that is not statistically significant at 95%.
  • ✓Only 950 of 11,900 invited members completed (about 8%); those who responded differ from those who did not, as the overnight-use skew shows.
  • ✗Only current members were surveyed: people who joined for 24-hour access and have since left, and people who would join only because of it, are missing, so the survey cannot show what 24-hour opening does for recruitment or the brand; and night-shift workers are a small group whose answers carry little weight in the totals even though they depend on the hours.

The analysis

Recommendation

Do not close all 14 gyms overnight, and do not split policy by region. Instead, keep gyms open 24 hours across the network by default and pursue selective overnight closure (via a controlled pilot at 2–3 genuinely quietest sites) only if per-gym door-entry logs and site P&Ls prove that overnight operating savings comfortably exceed lost membership revenue from local overnight users and nearby 24/7 competitor defection.

What would change it

Keep every gym open 24 hours with no site closures if per-site door-entry logs show overnight users are evenly spread (~4–6%+ at every gym) or overnight cost savings per site are smaller than losing ~70–100 members per gym; conversely, move to broader overnight closures if site P&Ls show high overnight security/utility/insurance costs (or acute safety incidents) that far exceed even ~4,400 lost memberships (~11.4% weighted churn) or if a multi-month closure pilot at 2–3 sites produces minimal net cancellations (<2–3% of site membership).

Findings

  1. To the Operations Director: overnight usage is small in headcount (6% of members in door-entry logs, ~2,316 members) rather than the 14.9% unweighted survey figure, but closing all 14 gyms overnight is high-risk because core overnight users visit intensively and have no alternative training window.

    High confidenceBase 950

    Door-entry records show 6% of all 38,600 members entered between 11pm and 5am in September 2026 versus 14.9% (142/950, ±2.3pp) in the survey, proving affected users were 2.5x as likely to respond. Among survey overnight users (n=142), mean 4-week frequency is 7.4 visits (median in the '2 to 3' band) and 35.9% usually arrive between 11pm and 5am. Among night-shift workers (n=56), 80.4% used the gym overnight and 57.1% visited 15+ times in four weeks; 16 open-text respondents state shift/night work makes 11pm–5am their only possible time to train.

  2. To the Marketing Director: unweighted cancellation intent (14.6%) overstates likely churn due to non-response and hypothetical-intent bias, but realistic network-wide churn risk still spans roughly 1,000 to 4,400 members—amplified by rolling contracts, nearby 24/7 rivals, and demands for a fee cut among daytime users.

    Medium confidenceBase 950

    Reweighting the sample to match the 6% door-entry overnight rate lowers 'fairly/very likely to cancel' (Q4 codes 4–5) from 14.6% (139/950) to 11.4% (~4,400 of 38,600 members), or 4.7% (~1,800 members) on 'very likely' alone (18.3% of the 6% overnight users plus 4.0% of the 94% non-overnight users). Even among members with zero overnight visits (n=808), 17.8% joined for 24-hour access and 9.2% say they would likely cancel; 52 of 239 open-text answers expect a monthly price reduction for losing six hours a day, 19 overnight users explicitly name or threaten switching to 24-hour competitors (PureGym, The Gym Group, Anytime Fitness), and 38.9% of members joined specifically for no fixed-term contract, making exit frictionless.

  3. To the Regional Manager: the gap in stated cancellation likelihood between North West England (19.2%) and North East England (13.6%) is not statistically significant and does not justify a regional policy split.

    High confidenceBase 449

    North West cancellation likelihood is 19.2% (n=229) versus 13.6% in the North East (n=220), a 5.6pp difference that fails significance at the 95% level (two-proportion z = 1.59, p = 0.11). Actual reported overnight usage in the survey is virtually identical across the two regions (14.0% North West vs 15.0% North East), and Q6 home-gym location mismatches in open-text responses further weaken regional comparisons.

  4. Survey sample sizes per gym are too small and noisy to select which individual gyms to close overnight; site-by-site selection must use door-entry logs rather than Q6 survey splits.

    High confidenceBase 950

    Individual gym bases at Q6 range from n=11 at Wakefield Calder Park (where 54.5% reported overnight visits, ±29.4pp) to n=61–81 elsewhere (±9pp to ±12pp margins of error). Furthermore, at least 16 open-text respondents describe training or living in a different city from their Q6 registered home gym (for example, registered to Leeds Kirkstall while paying and training in Glasgow or Preston), indicating cross-gym usage, stale registration records, or coding error.

  5. Overnight closure would create secondary friction in shoulder hours (10–11pm and 5–6am), while keeping gyms open unstaffed leaves an unresolved perceived-safety issue among daytime members.

    Medium confidenceBase 239

    In Q3 open-text responses (n=239), 31 respondents (almost all non-overnight users) worry that displaced overnight trainers will crowd quiet 10–11pm and 5–6am slots—relevant because 22.0% of all respondents usually arrive between 5am and 9am. Conversely, 42 open-text respondents actively welcome closing between 11pm and 5am because unstaffed overnight gyms feel unsafe or unsettling.

Caveats

  • Severe non-response bias on overnight usage: only 8.0% of invited members completed the survey (950/11,900), and 14.9% of respondents reported overnight visits versus 6.0% in September door-entry logs—meaning unweighted totals over-represent overnight users by 2.5x.
  • Stated cancellation intent overstates actual churn: Q4 measures hypothetical reaction immediately after priming overnight closure at Q1–Q3. In subscription businesses, actual cancellations—especially among the 9.2% of non-overnight users saying they would likely cancel—are typically a fraction of top-two-box intent, though rolling monthly contracts lower exit friction.
  • Home-gym (Q6) and segment data-integrity anomalies: at least 16 open-text answers name a local Corefit branch or city incompatible with their Q6 home gym (e.g., registered to Leeds Kirkstall, Salford Quays, or Sheffield Attercliffe while describing training in Glasgow, Edinburgh, Preston, or Gateshead), and several respondents coded as having visited between 11pm and 5am at Q1 explicitly write in Q3 that they never visit overnight (e.g., M034, M305, M735). Regional and gym-level survey breakdowns must therefore be treated with caution.
  • Small gym-level sample sizes: individual gym bases range from n=11 (Wakefield Calder Park) to n=81, making site-level survey percentages too imprecise for site-closure selection without door-entry records.
  • No operational cost or competitor-proximity data in the study: the survey cannot show whether utility, insurance, security, or maintenance savings from closing 11pm–5am outweigh lost membership fees at any given site.

Next steps

  • researchExtract 12-month door-entry data by gym and hour (11pm–5am, 10–11pm, 5–6am) for all 14 sites, alongside actual gym visited versus registered home gym, and map competitor 24-hour gyms within 2 miles of each site.
  • decisionBuild a per-gym breakeven model comparing actual 11pm–5am operating cost savings (utilities, remote monitoring, cleaning, insurance, maintenance) against realistic churn scenarios (e.g., losing 25%–50% of true overnight users plus 1%–3% of non-overnight members).
  • experimentIf 2–3 low-overnight-usage gyms without close 24/7 competitors clear the financial breakeven bar, run a 3-month overnight closure pilot at those sites only—allowing affected night-shift members multi-gym access where a sister Corefit is nearby (e.g., Leeds Kirkstall/Hunslet or Manchester Ancoats/Salford Quays) rather than cutting headline prices.
  • monitorAudit and communicate overnight safety measures (CCTV monitoring, panic alarms/lanyards, lighting, tailgating controls) at all gyms remaining open 24 hours.

Open-text themes it coded

Unaffected because never using the gym between 11pm and 5am 90Expect a lower monthly fee if opening hours are cut 52Welcome overnight closure due to safety concerns about unstaffed gyms 42Worry displaced overnight users will crowd quiet 10–11pm or 5–6am sessions 31Would cancel membership or switch to a rival 24-hour gym 20Night-shift work makes 11pm to 5am the only viable window to train 16

The survey it planned

1 screening question and 10 questions, as the model wrote them.

  1. S1

    Are you currently a paying member of Corefit?

    One answer
    • Yes, I have an active membership
    • No, my membership is currently frozen or paused
    • No, I have cancelled my membership
    • No, I have never been a Corefit member

    Continues if Yes, I have an active membership

  2. Q1

    Which Corefit gym do you use most often?

    One answer
    • North of England - Gym 1 [Insert Location Name]
    • North of England - Gym 2 [Insert Location Name]
    • North of England - Gym 3 [Insert Location Name]
    • North of England - Gym 4 [Insert Location Name]
    • North of England - Gym 5 [Insert Location Name]
    • North of England - Gym 6 [Insert Location Name]
    • North of England - Gym 7 [Insert Location Name]
    • North of England - Gym 8 [Insert Location Name]
    • Scotland - Gym 9 [Insert Location Name]
    • Scotland - Gym 10 [Insert Location Name]
    • Scotland - Gym 11 [Insert Location Name]
    • Scotland - Gym 12 [Insert Location Name]
    • Scotland - Gym 13 [Insert Location Name]
    • Scotland - Gym 14 [Insert Location Name]
  3. Q2

    When you decided to join Corefit, what were the main reasons you chose us? (Select up to 3)

    Any that apply
    • Low monthly price
    • Rolling monthly contract (no long-term tie-in)
    • Convenient location (near home, work, or study)
    • Open 24 hours a day
    • Quality and range of gym equipment
    • Cleanliness and gym environment
    • Classes or personal training options
    • Friends or family are members here
    • Other
  4. Q3

    Over the past 3 months, roughly how often have you visited a Corefit gym in total (at any time of day or night)?

    One answer
    • 4 or more times a week
    • 2 to 3 times a week
    • About once a week
    • 1 to 3 times a month
    • Less than once a month
    • I have not visited in the past 3 months
  5. Q4

    Over the past 3 months, at which times of the day have you entered or worked out at a Corefit gym? (Select all that apply)

    Any that apply
    • Early morning (5:00am – 7:59am)
    • Morning (8:00am – 11:59am)
    • Afternoon (12:00pm – 4:59pm)
    • Evening (5:00pm – 8:59pm)
    • Late evening (9:00pm – 10:59pm)
    • Overnight (11:00pm – 4:59am)

    Routing Shown only if Q3 is: 4 or more times a week; 2 to 3 times a week; About once a week; 1 to 3 times a month; Less than once a month

  6. Q5

    Specifically thinking about the overnight hours between 11:00pm and 5:00am, how often have you used a Corefit gym during these hours over the past 3 months?

    One answer
    • 3 or more times a week
    • 1 to 2 times a week
    • 2 to 3 times a month
    • About once a month
    • Once or twice in the past 3 months
    • Never in the past 3 months
  7. Q6

    If your Corefit gym changed its opening hours to 5:00am – 11:00pm every day (meaning it would be closed overnight from 11:00pm to 5:00am), how would this affect your workouts?

    One answer
    • It would not affect me at all; I never use or plan to use those hours
    • I could easily work out at other times between 5:00am and 11:00pm
    • I could work out at other times, though it would be less convenient for me
    • I would have to reduce how often I go to the gym
    • I would not be able to use the gym at all
  8. Q7

    If your Corefit gym closed overnight between 11:00pm and 5:00am (with monthly membership prices staying the same), what would you most likely do with your membership?

    One answer
    • Definitely keep my Corefit membership
    • Probably keep my Corefit membership
    • Not sure / it would depend on other gyms nearby
    • Probably cancel my Corefit membership
    • Definitely cancel my Corefit membership
  9. Q8

    Why would closing between 11:00pm and 5:00am make you consider cancelling, or what other gym options would you look for?

    Open text

    Routing Shown only if Q7 is: Not sure / it would depend on other gyms nearby; Probably cancel my Corefit membership; Definitely cancel my Corefit membership

    Judges: double-barrelled

    It asks two separate things in one open box: why the closure would make them consider cancelling, and what other gym options they would look for.

    It asks two distinct questions (why they would consider cancelling AND what other options they would look for) but provides a single open-text field.

  10. Q9

    Which of the following best describes your typical working or study schedule?

    One answer
    • Standard daytime hours (e.g., roughly 9am to 5pm)
    • Shift work, evening, or night shifts (e.g., healthcare, hospitality, logistics, emergency services)
    • Flexible, freelance, or irregular hours
    • Full-time or part-time student
    • Not currently working or studying (e.g., retired, full-time carer, between jobs)
  11. Q10

    Which age group do you belong to?

    One answer
    • 16–24
    • 25–34
    • 35–44
    • 45–54
    • 55 or over
    • Prefer not to say
Sample plan

CRITICAL PRECISION NOTE FOR THE BOARD: Achieving ±8 percentage points at 95% confidence (worst-case p=0.50) requires n ≈ 143 completes PER GYM (applying finite population correction for ~2,757 members/gym), which equals ~2,000 completes in total across 14 gyms. With the brief's constraint of ~1,000 total completes (using a stratified quota of n = 72 completes per gym, total n = 1,008), gym-level worst-case precision is ±11.4pp. HOWEVER, because actual overnight usage is near p = 0.06 (per door-entry logs), n = 72 per gym achieves ±5.4pp at 95% confidence for proportions ≤10% (meeting the ±8pp goal for overnight usage and churn estimates). SAMPLING & INVITATIONS: Use stratified random sampling from the membership database across all 14 home gyms, inviting ~700–1,000 randomly selected active members per gym in batches (expecting an 8–12% email response rate) until hitting strict quotas of n = 72 completes per gym (or expand budget/target to n = 145 per gym / 2,030 total if ±8pp is strictly required on 50/50 split questions like Q2). Link survey IDs to anonymised database flags (home gym, region, tenure, and September 11pm–5am door-entry flag) so overall chain results can be weighted by true gym member counts AND checked for non-response bias (e.g., ensuring overnight users do not disproportionately respond or abstain). All questions are worded neutrally without revealing cost pressures or stakeholder assumptions.