Benchmarks / SurveyBench / Should Corefit gyms stop opening 24 hours? / Claude Haiku 5.5

Measured by Spring Prompt

Claude Haiku 5.5: Should Corefit gyms stop opening 24 hours?

The decision: whether to close all 14 gyms between 11pm and 5am, close overnight only at the quietest gyms, or keep every gym open 24 hours. The brief and the data →

Verdict
✗ Not sound
Research score
87 of 100
Analysis rating
1,412
Head to head, this task
won 7 of 9

Why it is not sound

The survey

  • Answer bands that overlap or leave gaps

The analysis

  • Reported a material finding the data does not support

Findings the judges found unsupported

  • Overnight use is a minority behaviour but is concentrated among night-shift workers, for whom 11pm to 5am is often the only training window. Closing all 14 gyms overnight would remove the only gym access for this group and is the main source of likely cancellations.Night-shift workers have the highest stated cancellation rate, but they are not the main source of likely cancellations in absolute numbers. About 27 of the 139 respondents likely to cancel work shifts including nights. Even all overnight users account for only 65 of those 139 responses, and their contribution falls further after weighting. Closure would remove the only training window for some night-shift workers, not necessarily the whole group.
  • Stated cancellation risk from closing overnight is concentrated in overnight users. Weighted to the door-entry share, about 11.4% of members (roughly 4,400 of 38,600) say they would be likely to cancel if their home gym closed 11pm-5am every night. This is the best available estimate of the membership loss from a 14-gym overnight closure, but it is a stated intention and not a measured behaviour.The weighted calculation supports an estimate of stated cancellation intention, not the claim that roughly 4,400 is the best available estimate of actual membership loss. The later caveat correctly distinguishes intention from behaviour, but calling the number an indicative ceiling is also unsupported: stated intentions do not establish an upper bound on actual losses.

Scorecard

Numbers 11 of 11 right

QuestionIts answerTrue
What percentage of respondents reported at least one visit to a Corefit gym between 11pm and 5am in the last four weeks?✓14.9%14.9%
What percentage of all respondents said they would be likely (fairly or very) to cancel their membership if their home gym closed between 11pm and 5am?✓14.6%14.6%
Among respondents who reported at least one visit between 11pm and 5am in the last four weeks, what percentage said they would be likely (fairly or very) to cancel?✓45.8%45.8%
Weighted so that respondents with at least one visit between 11pm and 5am in the last four weeks count as 6% of members (the door-entry figure) and the rest as 94%, what percentage would be likely (fairly or very) to cancel?✓11.4%11.4%
If the weighted share likely to cancel were applied to all 38,600 members, about how many members would that be?✓4,4004,383
Among respondents who reported at least one visit between 11pm and 5am in the last four weeks, in which band of the Q1 table does the median number of visits fall? Give the lowest value of that band (for example 4 for '4 to 7').✓22
What is the 95% margin of error, in percentage points, on the unweighted share of respondents who reported at least one visit between 11pm and 5am in the last four weeks?✓2.3 points2.27 points
Is the difference between North West England and North East England respondents in the share likely (fairly or very) to cancel statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
How many of the open-text answers say that 11pm to 5am is the only time the respondent can train because they work shifts or nights?✓1616
How many of the open-text answers from respondents who visited between 11pm and 5am in the last four weeks say they would move to another gym?✓1919
What percentage of the members invited completed the survey?✓8%8%

Traps in the data 8 of 8 handled

  • ✓14.9% of respondents report at least one visit between 11pm and 5am in the last four weeks, against 6% of members in the door-entry records: members with a stake in the overnight hours were more likely to respond. Unweighted, 14.6% say they would be likely to cancel; weighted to the 6%, 11.4%.
  • ✓'Likely to cancel' answers (45.8% of overnight users, 14.6% overall) are stated intentions about a hypothetical closure, asked straight after members were invited to think about it.
  • ✓The mean number of overnight visits among overnight users (7.4) is pulled up by night-shift workers who come almost daily; the median is in the '2 to 3' band.
  • ✓The operations director believes almost nobody uses the gyms overnight. Partly true: 6% of members in the door-entry records (about 2,316 people). But night-shift workers (5.9% of respondents) mostly train between 11pm and 5am (77.8% of those who visited overnight usually arrive then) and are the most likely to say they would cancel.
  • ✓At Wakefield Calder Park (base 11), 54.5% of respondents used the gym overnight and 27.3% say they would be likely to cancel.
  • ✓North West England respondents (base 229) look more likely to cancel than North East England respondents (base 220): 19.2% against 13.6%, a gap that is not statistically significant at 95%.
  • ✓Only 950 of 11,900 invited members completed (about 8%); those who responded differ from those who did not, as the overnight-use skew shows.
  • ✓Only current members were surveyed: people who joined for 24-hour access and have since left, and people who would join only because of it, are missing, so the survey cannot show what 24-hour opening does for recruitment or the brand; and night-shift workers are a small group whose answers carry little weight in the totals even though they depend on the hours.

The analysis

Recommendation

Do not close all 14 gyms from 11pm to 5am on the current evidence. Do not keep every gym open 24 hours without a plan either, because there is a real and concentrated retention risk to shift workers. Pending gym-level door-entry data, the most defensible option is to keep 24-hour opening at the gyms with material overnight use, which are likely to be the gyms where night-shift members train. Consider closing overnight only at the quietest gyms, and only as a tested pilot with a fee adjustment or a low-cost overnight access tier for shift workers. Any closure should be accompanied by a staffed-hours or safety plan, because the safety concern is a real reason some members would welcome it. Regional differences should not be used to set the policy, because the difference is not significant and overnight use is the same in both regions.

What would change it

(1) Gym-by-hour door-entry data showing overnight entries concentrated in a few gyms with low use, which would support closing only those gyms. (2) Evidence that overnight users have no practical alternative nearby, which would weaken the case for retaining 24-hour opening at those gyms. (3) A pilot showing that cancellations after an overnight closure, with or without a fee change, are well below the stated 11.4% rate. (4) Evidence that the fee reduction the market expects would cost more than the overnight revenue saved, which would argue against closing at all. (5) A non-member survey showing that 24-hour access drives acquisition strongly, which would argue for keeping all 14 gyms open.

Findings

  1. Overnight use is a minority behaviour but is concentrated among night-shift workers, for whom 11pm to 5am is often the only training window. Closing all 14 gyms overnight would remove the only gym access for this group and is the main source of likely cancellations.

    High confidenceBase 950

    14.9% of respondents (142/950) visited 11pm-5am in the last four weeks, against about 6% of members (about 2,316) on door-entry data. Among the 56 respondents on shifts that include nights, 80.4% visited overnight, and 48.2% said they would be likely to cancel. All 32 respondents who visited 15+ times are on shifts that include nights. Among all overnight users, 59.2% gave 24-hour opening as a reason for joining, against 17.8% of non-users.

  2. Stated cancellation risk from closing overnight is concentrated in overnight users. Weighted to the door-entry share, about 11.4% of members (roughly 4,400 of 38,600) say they would be likely to cancel if their home gym closed 11pm-5am every night. This is the best available estimate of the membership loss from a 14-gym overnight closure, but it is a stated intention and not a measured behaviour.

    Medium confidenceBase 950

    Likely to cancel: 45.8% of overnight users (65 of 142) and 9.2% of non-users (74 of 808). Weighted: 0.06 x 45.8 + 0.94 x 9.2 = 11.4%. Unweighted all-respondent figure is 14.6%, which overstates the true share because overnight users are over-represented in the sample. Projection: 11.4% x 38,600 = about 4,400.

  3. The North West versus North East cancellation gap is not statistically significant and should not drive a regional decision. Overnight-use rates are almost identical in the two regions (14.0% and 15.0%), so there is no evidence that North West members are a distinct case.

    High confidenceBase 449

    Likely to cancel: North West 19.2% (n=229) vs North East 13.6% (n=220). Difference 5.6 percentage points; standard error about 3.5 points; z about 1.6, p about 0.11, which is above the 0.05 threshold. Overnight-use rates: North West 14.0%, North East 15.0%.

  4. Price is the most common expectation attached to any reduction in hours. Many members who do not use the gym overnight still expect a lower fee if the hours are cut, so an overnight closure without a price adjustment would generate complaints and possibly cancellations across the membership, not just among overnight users.

    Medium confidenceBase 239

    About 52 of the 239 open-text answers ask for or expect a lower monthly fee, and most of these come from members who never visit overnight. Several name a specific amount (a fiver, £3 to £4, a tenner). Among overnight users, the open-text answers name switching to 24-hour competitors (PureGym, Anytime Fitness, The Gym Group) as the alternative.

  5. A substantial minority of members, many of them non-overnight users, see unstaffed overnight opening as a safety concern and would welcome closure. This is a real counterweight to the retention risk, though it is qualitative and self-selected.

    Low confidenceBase 239

    About 42 open-text answers express unease about the gym being unstaffed overnight or welcome a closure for that reason. Most of these are from members who never use the gym between 11pm and 5am.

  6. Members who train early morning or late evening fear that displaced overnight users will crowd their slots. This is an operational risk that the survey cannot quantify but that could erode the value of the quiet hours for daytime and early-morning members.

    Low confidenceBase 239

    About 31 open-text answers from members who avoid the overnight hours describe expected crowding at 10pm, around 5am, or at opening time. Several name specific slots (10pm, 5:00am squat racks).

  7. The operations director's premise is partly right. Overnight use is low across the membership, but 'almost nobody' is not accurate. About 2,300 members used the gyms overnight in September, and they are concentrated in a shift-worker group whose stated loss risk is high.

    High confidenceBase 950

    Door-entry share of about 6%, roughly 2,316 members. Q1 distribution among overnight users: median in the 2 to 3 band, mean 7.4 visits, 32 respondents at 15 or more visits, all of whom are shift workers.

Caveats

  • The sample over-represents overnight users: 14.9% of respondents visited 11pm-5am against about 6% of members on door-entry data. The unweighted all-respondent figures (14.6% likely to cancel) therefore overstate the membership-wide risk. The weighted figure (11.4%) corrects for this but assumes the door-entry share for September applies to the four weeks before the survey.
  • Cancellation intent is hypothetical and self-reported. Stated likelihood usually overstates actual cancellation, so the roughly 4,400 figure is an indicative ceiling for a 14-gym overnight closure, not a forecast.
  • Response rate is low (8.0% of the 11,900 invited; 950 of 38,600 members). Members with strong views on overnight access, in either direction, may be more likely to respond. The 95% margin of error of 2.3 points assumes a random sample and ignores non-response and design effects.
  • Only current members were surveyed. The marketing question about lost members is answered only for existing members. Losses of prospective members, and the value of 24-hour access as an acquisition feature (24% of all respondents and 59% of overnight users cited it), are not measured.
  • Subgroups are small. The shift-worker group contains 56 respondents, and gym-level bases are about 61 to 81 (Wakefield 11). Gym-level and shift-level findings are directional only.
  • The survey measures members' reported behaviour and intentions, not the gyms' actual busy-hour profile. It cannot identify which gyms are quietest, so it cannot assess the option of closing overnight only at the quietest gyms. The 2,316 overnight door-entry figure is membership-wide and is not broken down by gym or hour in the data provided.
  • Some open-text answers conflict with the coded Q1 data. For example, M034 and M305 are coded as overnight visitors but say they never use the gym overnight. These may be recall or coding errors and were treated as stated.
  • Open-text answers (239) are self-selected and were not coded by an independent second coder. The theme counts are indicative of what respondents raised, not of how common each view is across the membership.
  • Q3 asks about a hypothetical closure of the home gym only. Responses do not capture effects of a network-wide change, such as members transferring to a nearby Corefit that remains open.

Next steps

  • researchPull door-entry counts by gym and hour for September and October 2026, and compare overnight entries with the membership of night-shift members at each gym.
  • experimentRun a pilot closing 11pm to 5am at two or three of the lowest-overnight-use gyms, with a fee reduction in one arm and no fee change in the other. Track cancellations, transfers to other Corefit gyms, and visits at neighbouring gyms.
  • experimentTest a night-access option for shift workers (for example a low-cost overnight add-on or a verified shift-worker tier) at gyms that remain 24-hour.
  • researchRun a short survey of prospective members and lapsed members about 24-hour access and its role in joining.
  • decisionSet out a fee policy before any closure: if hours are cut, state the price change in advance and link it to the hours removed.
  • decisionDo not use regional differences to set policy. Record the North West and North East gap as not established, and decide per gym on usage and shift-worker data.
  • monitorMonitor overnight safety incidents, reports of unstaffed gyms, and crowding at 10pm and opening time at all gyms, whichever option is chosen.

Open-text themes it coded

Night-shift workers whose only training window is 11pm-5am 16Would leave Corefit (switch gym or cancel) if overnight hours closed 20Expect a lower monthly fee if opening hours are cut 50Welcome closure because unstaffed overnight gyms feel unsafe 42Concern that overnight users will crowd quiet slots (10pm or early morning) 31No effect: never uses the gym between 11pm and 5am 89

The survey it planned

2 screening questions and 10 questions, as the model wrote them.

  1. S1

    Are you currently a member of Corefit?

    One answer
    • Yes
    • No

    Continues if Yes

  2. S2

    Are you 18 or over?

    One answer
    • Yes
    • No

    Continues if Yes

  3. Q1

    In the last three months, how often have you used a Corefit gym between 11pm and 5am?

    One answer
    • Never
    • Less than once a month
    • 1 to 3 times a month
    • 1 to 2 times a week
    • 3 or more times a week
  4. Q2

    Suppose Corefit closed all its gyms between 11pm and 5am. What would you be most likely to do?

    One answer
    • Keep my membership and carry on as before outside those hours
    • Keep my membership but change when I work out
    • Cancel my membership
    • Move my membership to a gym that is open 24 hours
    • Not sure
  5. Q3

    If you could not use your Corefit gym between 11pm and 5am, how much would that affect your decision to keep your membership?

    Scale

    Scale 1-5: Not at all to A great deal

    Routing Shown only if Q1 is: Less than once a month; 1 to 3 times a month; 1 to 2 times a week; 3 or more times a week

  6. Q4

    Why do you usually use the gym between 11pm and 5am? Select all that apply.

    Any that apply
    • My working hours mean it is the only time I can go
    • It fits around my sleep or other routines
    • I cannot get to the gym at other times because of caring responsibilities
    • I prefer to train when it is quiet
    • Something else

    Routing Shown only if Q1 is: Less than once a month; 1 to 3 times a month; 1 to 2 times a week; 3 or more times a week

  7. Q5

    Please rank these reasons for choosing Corefit, from most important (1) to least important (6).

    Ranking
    • Low monthly price
    • Open 24 hours a day
    • Gym is close to my home or work
    • Less busy than other gyms
    • Rolling monthly contract with no tie-in
    • Equipment and facilities
    Judges: assumptive

    Requiring every listed reason to be ranked assumes they were applicable reasons for choosing Corefit. Respondents cannot distinguish reasons that played no part from reasons that mattered less.

    It forces respondents to rank all options, assuming all of them were reasons for choosing the gym.

  8. Q6

    How important was being open 24 hours a day in your decision to join Corefit?

    Scale

    Scale 1-5: Not important at all to Essential

  9. Q7

    Which of these best describes your work pattern?

    One answer
    • Daytime hours, usually Monday to Friday
    • Evening work
    • Shift work that includes some night shifts
    • Mainly night shifts
    • Irregular or variable hours
    • I do not work outside the home
    • Prefer not to say
  10. Q8

    What is your age?

    One answer
    • 18 to 24
    • 25 to 34
    • 35 to 44
    • 45 to 54
    • 55 to 64
    • 65 or over
    • Prefer not to say
  11. Q9

    How long have you been a Corefit member?

    One answer
    • Less than 6 months
    • 6 to 12 months
    • 1 to 2 years
    • More than 2 years
  12. Q10

    To show you are reading carefully, please select 'Agree' for this question.

    One answer
    • Strongly disagree
    • Disagree
    • Agree
    • Strongly agree
Sample plan

Population: the 38,600 current Corefit members across 14 gyms, each with a home gym and an email address on file. Invitations are sent by email with a unique link that pre-fills the member's home gym, so no gym question is asked. Use a stratified random sample. Stratum A (census): every member who entered a gym between 11pm and 5am in September 2026 (about 2,300 members, 6%). Stratum B: a random sample of all other members, drawn separately within each gym. Assuming about 20% response, this means roughly 2,300 invitations to Stratum A and about 8,300 to Stratum B, about 10,600 in total, for about 2,100 completes. Quotas: at least 151 completes per gym (the minimum for ±8 points at 95% confidence on a 50% estimate, 14 gyms x 151 = 2,114). Stratum A is not capped. Stop Stratum B invitations for a gym once its quota is met, and check each gym's completes weekly. Minimum precision: with 1,000 completes (about 71 per gym) the margin per gym is about ±11.6 points, so the board's ±8 target per gym needs about 2,120 completes. If budget caps at 1,000, report per-gym results as indicative and state the wider margin. Invitations should describe the survey as being about members' use of opening hours and should not reveal the closure options being considered. Under-18s are excluded by S2. Weight completes back to the membership by gym and by Stratum A/B membership before estimating totals.