Benchmarks / SurveyBench / Should Corefit gyms stop opening 24 hours? / Gemini 3.8 Flash

Measured by Spring Prompt

Gemini 3.8 Flash: Should Corefit gyms stop opening 24 hours?

The decision: whether to close all 14 gyms between 11pm and 5am, close overnight only at the quietest gyms, or keep every gym open 24 hours. The brief and the data →

Verdict: ✗ Not sound
Research score: 78 of 100
Analysis rating: 898
Head to head, this task: won 7 of 11

Why it is not sound

The survey

Ignored the precision the brief asked for, which the sample cannot deliver

The analysis

Reported a material finding the data does not support
Missed a trap: Only current members were surveyed: people who joined for 24-hour access and have since left, and people who would join only because of it, are missing, so the survey cannot show what 24-hour opening does for recruitment or the brand; and night-shift workers are a small group whose answers carry little weight in the totals even though they depend on the hours.

Findings the judges found unsupported

When weighted by actual population door-entry data (6% overnight visitors), projected churn is approximately 11.4% across Corefit's membership.The weighting calculation supports approximately 11.4% saying they would be likely to cancel, not projected actual churn or a forecast loss of about 4,400 members. The later financial estimates also rely on this unsupported conversion from intention to behaviour.
Closure would cause operational displacement, creating severe overcrowding in the shoulder hours immediately before 11pm and after 5am.The comments establish concern about possible displacement, not that closure would cause severe overcrowding. The broad arrival-time bands do not establish existing shoulder-hour occupancy, available capacity or how overnight users would change their visits.
Closing overnight triggers substantial cancellation risk even among members who never visit at night, driven by perceived loss of value and price sensitivity.The 9.2% of non-visitors saying they are likely to cancel is a stated intention, and the data does not show this is 'substantial' risk. It also does not show the intent is driven by price sensitivity: the fee comments are requests for a discount, not statements of intent to cancel.

Scorecard

Numbers 11 of 11 right

Question		Its answer	True
What percentage of respondents reported at least one visit to a Corefit gym between 11pm and 5am in the last four weeks?	✓	14.9%	14.9%
What percentage of all respondents said they would be likely (fairly or very) to cancel their membership if their home gym closed between 11pm and 5am?	✓	14.6%	14.6%
Among respondents who reported at least one visit between 11pm and 5am in the last four weeks, what percentage said they would be likely (fairly or very) to cancel?	✓	45.8%	45.8%
Weighted so that respondents with at least one visit between 11pm and 5am in the last four weeks count as 6% of members (the door-entry figure) and the rest as 94%, what percentage would be likely (fairly or very) to cancel?	✓	11.4%	11.4%
If the weighted share likely to cancel were applied to all 38,600 members, about how many members would that be?	✓	4,399	4,383
Among respondents who reported at least one visit between 11pm and 5am in the last four weeks, in which band of the Q1 table does the median number of visits fall? Give the lowest value of that band (for example 4 for '4 to 7').	✓	2	2
What is the 95% margin of error, in percentage points, on the unweighted share of respondents who reported at least one visit between 11pm and 5am in the last four weeks?	✓	2.3 points	2.27 points
Is the difference between North West England and North East England respondents in the share likely (fairly or very) to cancel statistically significant at the 95% level? (1 for yes, 0 for no)	✓	No	No
How many of the open-text answers say that 11pm to 5am is the only time the respondent can train because they work shifts or nights?	✓	16	16
How many of the open-text answers from respondents who visited between 11pm and 5am in the last four weeks say they would move to another gym?	✓	19	19
What percentage of the members invited completed the survey?	✓	7.98%	8%

Traps in the data 7 of 8 handled

✓14.9% of respondents report at least one visit between 11pm and 5am in the last four weeks, against 6% of members in the door-entry records: members with a stake in the overnight hours were more likely to respond. Unweighted, 14.6% say they would be likely to cancel; weighted to the 6%, 11.4%.
✓'Likely to cancel' answers (45.8% of overnight users, 14.6% overall) are stated intentions about a hypothetical closure, asked straight after members were invited to think about it.
✓The mean number of overnight visits among overnight users (7.4) is pulled up by night-shift workers who come almost daily; the median is in the '2 to 3' band.
✓The operations director believes almost nobody uses the gyms overnight. Partly true: 6% of members in the door-entry records (about 2,316 people). But night-shift workers (5.9% of respondents) mostly train between 11pm and 5am (77.8% of those who visited overnight usually arrive then) and are the most likely to say they would cancel.
✓At Wakefield Calder Park (base 11), 54.5% of respondents used the gym overnight and 27.3% say they would be likely to cancel.
✓North West England respondents (base 229) look more likely to cancel than North East England respondents (base 220): 19.2% against 13.6%, a gap that is not statistically significant at 95%.
✓Only 950 of 11,900 invited members completed (about 8%); those who responded differ from those who did not, as the overnight-use skew shows.
✗Only current members were surveyed: people who joined for 24-hour access and have since left, and people who would join only because of it, are missing, so the survey cannot show what 24-hour opening does for recruitment or the brand; and night-shift workers are a small group whose answers carry little weight in the totals even though they depend on the hours.

The analysis

Recommendation

Do not implement a blanket overnight closure across all 14 gyms. Keep gyms open 24 hours in high-density urban locations with direct 24/7 competitor presence and significant night-shift populations (such as Manchester, Leeds, and Glasgow). If overnight operational savings must be realized, pilot an overnight closure only at quiet, low-competition suburban clubs while simultaneously testing a tiered pricing model (discounted off-peak/daytime rate vs full-price 24/7 access).

What would change it

Evidence showing that overnight operating costs (energy, security, insurance, maintenance) exceed the revenue generated by the projected 11.4% membership churn risk (~£1.0M annually), or club-level door telemetry proving specific locations average fewer than 3 unique entries per night with no nearby 24/7 budget competitors.

Findings

Overnight closure risks acute churn among night shift workers for whom overnight access is an absolute necessity.

High confidenceBase 56

80.4% of night shift workers (base: 56) visit between 11pm and 5am; 48.2% state they are likely to cancel if overnight hours are closed. 16 qualitative comments confirm these members cannot train at any other time.
Closing overnight triggers substantial cancellation risk even among members who never visit at night, driven by perceived loss of value and price sensitivity.

High confidenceBase 808

9.2% of respondents who made zero visits between 11pm and 5am (base: 808) say they are likely to cancel. Qualitative answers repeatedly demand fee reductions if hours are cut.
When weighted by actual population door-entry data (6% overnight visitors), projected churn is approximately 11.4% across Corefit's membership.

Medium confidenceBase 950

Applying the 6% true visitor rate to stated cancellation rates (45.8% for night visitors, 9.2% for non-visitors) yields a weighted churn likelihood of 11.4%, equating to ~4,400 members across the 38,600 member base.
The observed difference in cancellation intent between North West and North East England is not statistically significant.

High confidenceBase 449

North West cancellation intent is 19.2% (base: 229) vs North East 13.6% (base: 220), giving a 5.6 percentage point gap with z = 1.59 (p = 0.11), which is not significant at the 95% confidence level.
Closure would cause operational displacement, creating severe overcrowding in the shoulder hours immediately before 11pm and after 5am.

High confidenceBase 950

22.0% of all members arrive 5am-9am and 35.4% arrive 5pm-11pm. Numerous regular morning (5am-6am) and late-evening (10pm-11pm) users expressed anxiety that displaced overnight users will crowd their workout windows.
A notable minority of daytime members actively prefer overnight closures due to safety and supervision concerns regarding unstaffed hours.

Medium confidenceBase 239

Over 25 open-text respondents unprompted reported feeling unsafe, uncomfortable, or exposed knowing gyms were open without staff overnight.

Caveats

Overnight users are over-represented in the unweighted survey sample (14.9%) compared to actual electronic door-entry data (6.0%), reflecting self-selection bias.
Stated cancellation intent (hypothetical willingness to churn) typically overestimates actual behavioral churn by a factor of 1.5x to 2.5x.
Individual gym sample sizes are small (e.g., Wakefield Calder Park n=11, Bradford Centre n=61), meaning club-by-club differences carry wide margins of error.
The survey did not measure members' willingness to pay a premium for 24-hour access versus a discounted daytime-only tier.

Next steps

researchAudit hourly door-entry telemetry and local competitor opening hours for all 14 individual gyms over the past 6 months.
decisionRun a financial sensitivity model comparing overnight operating cost savings against a 5% to 11% churn loss.
experimentPilot overnight closure (11pm-5am) at 1-2 quietest suburban sites with low competitor presence, offering affected night users transfer or a small subscription discount.
monitorTrack shoulder-hour entry volumes (10pm-11pm and 5am-6am) and member satisfaction regarding gym safety and equipment availability.

Open-text themes it coded

Shift and night workers unable to train at any other time 16Demand for lower monthly membership fee if hours are reduced 52Fear of crowding and displacement into shoulder hours (10pm-11pm and 5am-6am) 30Relief or enhanced feelings of safety from closing unstaffed overnight hours 42Intention to switch to a 24-hour competitor among overnight visitors 19Completely unaffected due to daytime or regular evening attendance 90

The survey it planned

1 screening question and 10 questions, as the model wrote them.

S1
Are you currently an active member of a Corefit gym?
One answer
- Yes, I have an active monthly membership
- No, my membership is currently frozen or paused
- No, I have cancelled my membership / I am no longer a member
Continues if Yes, I have an active monthly membership
Q1
Which of the following were important reasons why you chose to join Corefit? (Please select all that apply)
Any that apply
- Convenient location / close to home or work
- Low membership price / value for money
- No long-term contract / rolling monthly flexibility
- 24-hour access / open overnight
- Quality and range of gym equipment
- Cleanliness and gym environment
- Recommended by friends or family
- Other
Q2
Which Corefit gym is your primary (home) gym?
One answer
- Aberdeen
- Dundee
- Edinburgh Central
- Edinburgh West
- Glasgow City Centre
- Glasgow South
- Leeds Central
- Leeds North
- Liverpool
- Manchester Central
- Manchester Salford
- Newcastle
- Sheffield
- York
Q3
Over the past 6 months, how often on average have you visited a Corefit gym during overnight hours (between 11:00 PM and 5:00 AM)?
One answer
- Never
- Rarely (less than once a month)
- Occasionally (1 to 3 times a month)
- Regularly (1 to 2 times a week)
- Frequently (3 or more times a week)
Q4
What are your primary reasons for visiting the gym between 11:00 PM and 5:00 AM? (Please select all that apply)
Any that apply
- Shift work or non-standard working hours
- The gym is quieter and equipment is available without waiting
- Fits better with family or caregiving commitments
- Personal preference / I focus better late at night or early morning
- Avoiding peak travel or commuting times
- Other
Routing Shown only if Q3 is: Rarely (less than once a month); Occasionally (1 to 3 times a month); Regularly (1 to 2 times a week); Frequently (3 or more times a week)
Q5
If Corefit were to adjust its opening hours to 5:00 AM to 11:00 PM (closing overnight between 11:00 PM and 5:00 AM), how would this affect your fitness routine?
One answer
- No impact at all – I never train during those hours anyway
- Minor impact – I could easily adjust my workout times
- Moderate impact – It would be inconvenient, but I could manage
- Major impact – It would severely disrupt my ability to work out
Q6
If your home gym closed overnight between 11:00 PM and 5:00 AM, what action would you most likely take regarding your membership?
One answer
- Keep my Corefit membership and continue working out as normal
- Keep my Corefit membership, but work out less often
- Cancel my Corefit membership and switch to another 24-hour gym
- Cancel my Corefit membership and exercise at home / outdoors
- Unsure / would wait and see
Q7
How important is having the option of 24-hour gym access to you, even if you do not use it regularly?
Scale

Scale 1-5: Not at all important to Extremely important
Q8
Which of the following best describes your typical daily work or study schedule?
One answer
- Standard daytime hours (e.g., 9am–5pm)
- Rotating or irregular shift work
- Regular night shifts
- Full-time or part-time student
- Self-employed / flexible working hours
- Not currently working / retired
- Prefer not to say
Q9
What is your age group?
One answer
- 16–24
- 25–34
- 35–44
- 45–54
- 55–64
- 65+
- Prefer not to say
Q10
Is there anything else you would like to share regarding Corefit's opening hours or your membership?
Open text

Sample plan

A stratified random sample drawn from Corefit's active member database (N=38,600 across 14 gyms, averaging ~2,750 members per gym). To satisfy the board's requirement of ±8 percentage points margin of error at a 95% confidence level for each individual gym, a minimum of 142 completed responses is required per gym (finite population corrected from N≈2,750). Total target completes: 1,988 (142 completes × 14 gyms), which also satisfies the overall sample requirement of ~1,000+ completes. Invitations will be sent via email with hard quotas of n=142 per gym, oversampling members flagged in gate records as having used the gym between 11pm and 5am to ensure robust sub-segment analysis, with post-stratification weighting applied back to true population proportions (6% overnight users, 94% non-overnight users) to eliminate non-response and selection bias.