GPT-6.1 Terra: Why Fernway customers cancel

The decision: whether to build one-tap skipping of any week (about four months of product work) or cut the price of the plans by 20%, to win back and keep customers. The brief and the data →

Verdict: ✗ Not sound
Research score: 95 of 100
Analysis rating: 1,273
Head to head, this task: won 4 of 10

Why it is not sound

The analysis

Missed a trap: The one-tap skipping question (Q3) came immediately before the price-cut question (Q4), so answers to Q4 may have been framed by Q3.

Scorecard

Numbers 11 of 11 right

Question		Its answer	True
What percentage of respondents gave price as their main reason for cancelling?	✓	21.6%	21.6%
Weighted to the tenure mix of all customers who cancelled, what percentage would give price as their main reason?	✓	26.3%	26.3%
Among respondents whose main reason was that it was hard to skip weeks, what percentage said they would be likely (fairly or very) to come back if any week could be skipped with one tap?	✓	58.6%	58.6%
How many of the open-text answers mention recipe variety or repetitive recipes?	✓	40	40
What is the 95% margin of error, in percentage points, on the unweighted share of respondents giving price as their main reason?	✓	3.2 points	3.19 points
Is the difference in the share satisfied (4-5) between Family and Couples respondents, across all respondents, statistically significant at the 95% level? (1 for yes, 0 for no)	✓	No	No
Weighted to the tenure mix of all 9,850 customers who cancelled, about how many of them would give 'hard to skip weeks' as their main reason?	✓	3,193	3,193
How many of the open-text answers from respondents who were customers for over 12 months mention recipe variety or repetitive recipes?	✓	25	25
Among respondents whose main reason was that it was hard to skip weeks, by how many percentage points does the share likely to come back with one-tap skipping exceed the share likely to come back with a 20% price cut?	✓	35.7 points	35.7 points
What percentage of the customers invited completed the survey?	✓	6.5%	6.5%
Among respondents whose main reason was price, what percentage said they would be likely (fairly or very) to come back if the plan cost 20% less?	✓	49.3%	49.3%

Traps in the data 7 of 8 handled

✓Northern Ireland respondents (base 12) cite delivery problems far more often.
✓Family-plan respondents look less satisfied overall, but are more satisfied than Couples within every tenure band: Family customers skew to long tenure, and long-tenure customers are less satisfied.
✓Long-tenure customers are 50% of respondents but 20% of cancellers; short-tenure customers, who cite price more, are under-represented.
✓'Likely to come back' answers are stated intentions about a hypothetical feature.
✓The chief executive believes customers leave mainly because of price.
✓Only 702 of 9,850 invited customers started the survey (about 7%); those who respond may differ from those who do not.
✓Only customers who cancelled were surveyed, so the survey cannot show what distinguishes leavers from customers who stay; the reasons are self-reported.
✗The one-tap skipping question (Q3) came immediately before the price-cut question (Q4), so answers to Q4 may have been framed by Q3.

The analysis

Recommendation

Provisionally prioritise one-tap skipping over a blanket 20% price cut, subject to a business case and behavioural testing before committing four months of work. Design around actual flexibility constraints, not just tap count. Do not ignore price-sensitive early leavers: test targeted offers separately. Investigate Northern Ireland delivery immediately and assess recipe variety as a separate retention opportunity.

What would change it

A credible test showing that improved skipping produces little incremental retained contribution, or that a price reduction produces greater incremental contribution after its cost, would change the recommendation. Evidence that skip deadlines, limits or operational constraints cannot be improved enough to meet customer needs would also prompt reconsideration.

Findings

Price matters, especially for early leavers, but the survey does not confirm the CEO's claim that it is the leading reason people leave.

Medium confidenceBase 640

Price was the main reason for 21.6% of 640 respondents, versus 35.5% for difficulty skipping. Weighting by the tenure distribution of all 9,850 cancellers gives approximately 26.3% for price and 32.4% for skipping. Price was cited by 31.2% of respondents with under three months' tenure, versus 13.7% with over 12 months.
One-tap skipping is the stronger candidate for a win-back investment, but stated likelihood is not proof of either return or retention.

Medium confidenceBase 227

Among the 227 respondents who cited difficulty skipping, 58.6% said they were likely to return with one-tap skipping, versus 22.9% with a 20% price cut: a 35.7 percentage-point difference. Across all 640 respondents, the respective figures were 36.4% and 25.8%.
The Family plan should not be treated as the problem solely because its overall satisfaction is lower.

Medium confidenceBase 640

Satisfied shares were 49.2% among 301 Family respondents and 53.4% among 339 Couples respondents; the 4.2-point difference is not statistically significant at the 95% level. Family satisfaction was higher within each tenure band. Long-tenure customers, who reported lower satisfaction overall, made up 67.1% of Family respondents and 31.0% of Couples respondents. The under-three-month Family subgroup had only 26 respondents.
Northern Ireland warrants an operational investigation, not a conclusion about the scale of a regional delivery problem.

Low confidenceBase 12

Eight of 12 Northern Ireland respondents, or 66.7%, named delivery as their main cancellation reason, compared with 50 of 628 respondents elsewhere, about 8.0%. R445 also described late or incomplete boxes. The Northern Ireland base is very small and survey respondents may not represent local cancellers.
Flexibility needs extend beyond a single skip button; recipe variety is also a distinct retention issue.

Medium confidenceBase 190

Among 190 open answers, 40 mention recipe variety or repetition, including 25 from customers with over 12 months' tenure. Flexibility answers request multiweek pauses, later deadlines and fewer meals as well as simpler skipping; for example, R217 describes a skip limit and R295 an early deadline.

Caveats

Only 640 of 9,850 invitees completed the survey. Long-tenure and Family customers are overrepresented; respondents may differ from nonrespondents even within the same tenure group. Tenure weighting corrects the known tenure mix only, not other selection bias.
The 3.2-point price margin of error is an approximate 95% normal-theory sampling margin under a simple random sample, without finite-population correction. This voluntary, low-response survey was not such a sample, so the figure does not capture nonresponse bias.
Q1 forces one main reason; price, flexibility and other factors can overlap. Tenure-weighted estimates and the approximately 3,193 skip-related cancellers assume respondents within each tenure band represent all cancellers in that band.
The return questions are hypothetical and were asked of the same people. The tables do not show paired responses, actual behaviour, a current-offer baseline or statistical uncertainty for the paired comparison.
The survey covers recent UK cancellers, not current subscribers or all prospective customers. It cannot establish the effect on retention, total demand or profitability.
Open-text coding is interpretive and themes can overlap. Counts describe the 190 people who answered Q5, not all customers. A request for easier skipping does not necessarily mean a one-tap control alone would meet the need.

Next steps

decisionBuild a comparative business case for skipping and a 20% price cut before committing to either.
experimentRandomise eligible current customers and recent cancellers to a feasible improved-skip experience, a 20% price offer and an appropriate control; use a lightweight pilot where possible.
researchResearch skip journeys and operational constraints, including deadlines, consecutive skips, multiweek pauses and meal-count flexibility.
researchAudit Northern Ireland delivery and fulfilment records, complaints and cancellation volumes against other regions, and contact affected customers.
researchAnalyse satisfaction and cancellation by tenure and plan using customer records, and investigate repeat-menu exposure.
monitorMonitor price-related early cancellations and test a targeted affordability offer rather than applying a universal cut immediately.

Open-text themes it coded

Recipe variety or repetition 40Easier skipping, pausing or less frequent boxes 49Fewer meals or adjustable meal counts 28Price, discounts or value for money 43Delivery or fulfilment reliability 18Packaging waste, recyclability or condition 19Larger or adjustable portion sizes 12

The survey it planned

1 screening question and 12 questions, as the model wrote them.

S1
Have you cancelled a Fernway subscription within the past 6 months?
One answer
- Yes
- No
- Not sure
Continues if Yes
Q1
How long had you been a Fernway customer when you most recently cancelled?
One answer
- Less than 3 months
- 3 to 12 months
- More than 12 months
- Not sure
Q2
Which Fernway plan did you have when you most recently cancelled?
One answer
- Couples plan, for two
- Family plan, for four
- Not sure
Q3
Are you currently subscribed to Fernway again?
One answer
- Yes
- No
- Not sure
Q4
Overall, how satisfied were you with Fernway while you were subscribed? Use a scale from 1 to 5.
Scale

Scale 1-5: 1 - Very dissatisfied to 5 - Very satisfied
Q5
Which of the following contributed to your most recent decision to cancel Fernway? Select all that apply. If you select 'Not sure', do not select another answer.
Any that apply
- The price was too high
- I wanted fewer meals or boxes
- It was difficult to skip a week or change a delivery
- I wanted to pause my subscription
- The delivery schedule or delivery experience did not suit me
- The recipes or meal choice did not suit me
- I was unhappy with ingredient quality or freshness
- The portions did not suit me
- The meals took too much time or effort
- My household, routine or circumstances changed
- I preferred another way of getting meals
- Another reason
- Not sure
Q6
What was the single most important reason you cancelled?
One answer
- The price was too high
- I wanted fewer meals or boxes
- It was difficult to skip a week or change a delivery
- I wanted to pause my subscription
- The delivery schedule or delivery experience did not suit me
- The recipes or meal choice did not suit me
- I was unhappy with ingredient quality or freshness
- The portions did not suit me
- The meals took too much time or effort
- My household, routine or circumstances changed
- I preferred another way of getting meals
- Another reason
- Not sure
Q7
The next questions are about hypothetical options, not offers. If you were not currently subscribed, how likely would you be to subscribe to Fernway again at the same price and with the same delivery and skipping rules you had when you cancelled? Use a scale from 1 to 5.
Scale

Scale 1-5: 1 - Very unlikely to 5 - Very likely
Q8
If you were not currently subscribed, how likely would you be to subscribe again if you could skip any upcoming week with one tap in the app, without the six-day advance requirement, but the price stayed the same as when you cancelled? Use a scale from 1 to 5.
Scale

Scale 1-5: 1 - Very unlikely to 5 - Very likely
Q9
If you were not currently subscribed, how likely would you be to subscribe again if your plan price were reduced by 20% on an ongoing basis, but the delivery and skipping rules stayed the same as when you cancelled? Use a scale from 1 to 5.
Scale

Scale 1-5: 1 - Very unlikely to 5 - Very likely
Q10
If you could choose just one of these hypothetical changes to Fernway, which would you choose?
One answer
- One-tap skipping of any upcoming week without the six-day advance requirement, with no price reduction
- An ongoing 20% reduction in the plan price, with no change to delivery or skipping rules
- I would choose neither
- Not sure
Q11
Which, if any, of these changes to how you receive Fernway would make it more appealing to you? Select all that apply. If you select 'None of these' or 'Not sure', do not select another answer.
Any that apply
- An easier way to skip a week
- Being able to pause the subscription
- Being able to receive fewer meals
- Being able to receive boxes less often
- More choice of delivery days or times
- None of these
- Not sure
Q12
Is there anything else you would like to tell us about why you cancelled or what might bring you back? Please do not include sensitive personal information. This question is optional.
Open text

Sample plan

Invite every customer in the database who cancelled in the past six months, using a neutral email invitation. Screen for cancellation within that period. Aim for about 600 completes; monitor responses in the three database tenure groups and send reminders to underrepresented groups without closing other groups. Check reported tenure against database tenure where possible and investigate discrepancies. Do not claim the requested precision is guaranteed: with 200 independent responses per group, the worst-case 95% margin of error is approximately ±6.9 percentage points. Approximately 385 responses per group, or 1,155 total, are needed for ±5 points under simple random sampling, before any finite-population adjustment. Report the achieved group sizes and confidence intervals; consider increasing completes if the board requires that precision. Nonresponse and selection bias are not captured by margins of error.