Particle-1: Why Fernway customers cancel

The decision: whether to build one-tap skipping of any week (about four months of product work) or cut the price of the plans by 20%, to win back and keep customers. The brief and the data →

Verdict: ✗ Not sound
Research score: 88 of 100
Analysis rating: 1,193
Head to head, this task: won 10 of 13

Why it is not sound

The analysis

Reported a material finding the data does not support
Missed a trap: Only customers who cancelled were surveyed, so the survey cannot show what distinguishes leavers from customers who stay; the reasons are self-reported.

Findings the judges found unsupported

Price leavers are a distinct, more short-tenure segment: a 20% cut could win back about half of them, but they are much less moved by one-tap skipping.Saying a 20% cut could win back about half of price leavers turns hypothetical stated likelihood into a quantified behavioural outcome. The data show that 49.3% say they would be likely to return, not that about half would actually return.

Scorecard

Numbers 11 of 11 right

Question		Its answer	True
What percentage of respondents gave price as their main reason for cancelling?	✓	21.6%	21.6%
Weighted to the tenure mix of all customers who cancelled, what percentage would give price as their main reason?	✓	26.3%	26.3%
Among respondents whose main reason was that it was hard to skip weeks, what percentage said they would be likely (fairly or very) to come back if any week could be skipped with one tap?	✓	58.6%	58.6%
How many of the open-text answers mention recipe variety or repetitive recipes?	✓	40	40
What is the 95% margin of error, in percentage points, on the unweighted share of respondents giving price as their main reason?	✓	3.2 points	3.19 points
Is the difference in the share satisfied (4-5) between Family and Couples respondents, across all respondents, statistically significant at the 95% level? (1 for yes, 0 for no)	✓	No	No
Weighted to the tenure mix of all 9,850 customers who cancelled, about how many of them would give 'hard to skip weeks' as their main reason?	✓	3,193	3,193
How many of the open-text answers from respondents who were customers for over 12 months mention recipe variety or repetitive recipes?	✓	25	25
Among respondents whose main reason was that it was hard to skip weeks, by how many percentage points does the share likely to come back with one-tap skipping exceed the share likely to come back with a 20% price cut?	✓	35.7 points	35.7 points
What percentage of the customers invited completed the survey?	✓	6.5%	6.5%
Among respondents whose main reason was price, what percentage said they would be likely (fairly or very) to come back if the plan cost 20% less?	✓	49.3%	49.3%

Traps in the data 7 of 8 handled

✓Northern Ireland respondents (base 12) cite delivery problems far more often.
✓Family-plan respondents look less satisfied overall, but are more satisfied than Couples within every tenure band: Family customers skew to long tenure, and long-tenure customers are less satisfied.
✓Long-tenure customers are 50% of respondents but 20% of cancellers; short-tenure customers, who cite price more, are under-represented.
✓'Likely to come back' answers are stated intentions about a hypothetical feature.
✓The chief executive believes customers leave mainly because of price.
✓Only 702 of 9,850 invited customers started the survey (about 7%); those who respond may differ from those who do not.
✗Only customers who cancelled were surveyed, so the survey cannot show what distinguishes leavers from customers who stay; the reasons are self-reported.
✓The one-tap skipping question (Q3) came immediately before the price-cut question (Q4), so answers to Q4 may have been framed by Q3.

The analysis

Recommendation

Do not cut all plan prices by 20% now. Build one-tap skipping and a proper pause/hold capability as the primary win-back and retention bet, because subscription-control friction is the top reason and has the strongest overall stated win-back. Treat price as a real but more targeted problem: run limited price offers for short-tenure and price-reason leavers rather than a blanket price cut, and investigate Northern Ireland delivery through operational data before making a regional claim.

What would change it

A randomised in-market test showing that a 20% broad price cut produces materially higher reactivation and retained contribution margin than one-tap skipping would change the recommendation. So would a skip pilot with low adoption or no retention lift, or evidence that the build cost/risk is substantially higher than assumed.

Findings

If the choice is one broad intervention, one-tap skipping has stronger stated win-back than a 20% price cut, especially among customers who left because skipping was hard.

High confidenceBase 640

Overall likely to come back: one-tap skip 36.4% vs 20% price cut 25.8% (base 640). Among hard-to-skip leavers: skip 58.6% vs price cut 22.9%, a 35.7 percentage-point gap (base 227).
The CEO's price thesis is real but incomplete: price is not the top cancellation reason, and weighting to the actual tenure mix raises it only to about 26.3%, still below subscription-control friction.

High confidenceBase 640

Price is 21.6% unweighted; weighted by population tenure mix 26.3%. Hard-to-skip is 35.5% unweighted and about 32.4% weighted. Recipes repetitive is 18.0%.
Price leavers are a distinct, more short-tenure segment: a 20% cut could win back about half of them, but they are much less moved by one-tap skipping.

Medium confidenceBase 138

Among price-main-reason leavers, 49.3% likely to return with 20% less vs 24.6% with one-tap skip (base 138). Price as main reason falls with tenure: 31.2% under 3 months, 27.2% 3-12 months, 13.7% over 12 months.
Long-tenure cancellers are driven more by repetitiveness and subscription administration than by price.

High confidenceBase 640

Repetitive recipes: 6.5% under 3 months, 14.4% 3-12 months, 25.4% over 12 months. Hard-to-skip: 26.1%, 36.9%, 38.8%. Price: 31.2%, 27.2%, 13.7%. Open text has 25 of 40 recipe-variety/repetition mentions from over-12-month customers.
The Family plan is not the clear product problem; its lower overall satisfaction is small, not statistically significant, and likely confounded by tenure mix.

High confidenceBase 640

Satisfied (4-5): Couples 53.4%, Family 49.2%, difference 4.2 pp, z about 1.06, p about 0.29, not significant. Within each tenure band Family is actually at least as satisfied as Couples: under 3 months 69.2% vs 66.1%; 3-12 months 61.6% vs 57.4%; over 12 months 42.1% vs 35.2%.
The Northern Ireland delivery signal is a red flag but not decision-grade on its own.

Low confidenceBase 12

Northern Ireland base is only 12; 66.7% cite delivery problems as main reason, about 8 respondents. The 95% confidence interval is very wide, roughly 40-93%.
The sample is not representative of the canceller population; unweighted tables overrepresent long-tenure and Family customers.

High confidenceBase 640

Sample vs population: over 12 months 48.0% vs 20%; under 3 months 21.6% vs 45%; Family 47.0% vs 34%; Couples 53.0% vs 66%. Weighting changes price from 21.6% to 26.3% and hard-to-skip to about 32.4%.
Low completion and hypothetical stated likelihood mean this survey should guide direction, not forecast actual win-back.

High confidenceBase 640

640 completed from 9,850 invited = 6.5% completion; 702 started. Q3 and Q4 are stated likelihood, not behaviour, and open text covers only 190 respondents.
Open text corroborates subscription-control and recipe-variety friction, and shows many customers had multiple reasons not captured by the single main-reason question.

Medium confidenceBase 190

40 of 190 open-text answers mention recipe variety/repetition; many more mention pause, skip, holidays, delivery reliability, packaging, or portion size.

Caveats

Respondents are only cancelling customers; the survey says nothing about current-customer retention or prospective customers.
Q3 and Q4 measure stated likelihood, not actual return behaviour, and Q3 precedes Q4, so hypothetical bias and order effects are possible.
Completion is only 6.5% of invitees and the achieved sample is skewed versus the canceller population: long-tenure and Family customers are overrepresented.
The tables are unweighted; weighting changes price from 21.6% to about 26.3% and is important for any population estimate.
The Northern Ireland base is only 12, so the 66.7% delivery-problem figure is not reliable enough for a regional operational conclusion without delivery data.
Open-text answers come from 190 of 640 respondents and are self-selected; theme counts indicate mentions, not prevalence.
Single main-reason coding hides multi-cause churn; many open-text answers mention two or more reasons.
No cost, margin, build-risk, or price-elasticity data are in the survey, so it cannot by itself calculate the financial return of either option.

Next steps

decisionApprove one-tap skip plus pause/hold and flexible frequency as the primary roadmap item.
experimentRun a targeted price-offer experiment for short-tenure and price-reason leavers instead of a blanket 20% cut.
researchAnalyse operational delivery data by region, especially Northern Ireland: on-time rate, missing items, complaints, refunds, and courier performance.
experimentTest menu-variety improvements for long-tenure and Family customers: more frequent new recipes, broader cuisines, fish/vegetarian rotation, and seasonal menus.
researchRe-run a representative, weighted canceller survey or add a behavioural/conjoint study if budget decisions depend on precise willingness to pay.
monitorMonitor skip-feature adoption and cancellation reasons after launch.

Open-text themes it coded

Flexibility / skipping / pausing / frequency control 50Recipe variety / repetition / menu innovation 40Price / value / affordability / discounts 42Fewer meals / flexible quantity / smaller plan 30Portion size / hunger 12Delivery reliability / timing / missing items / courier 19Packaging / recycling / waste 19Life circumstances / no-fault cancellation 6

The survey it planned

3 screening questions and 12 questions, as the model wrote them.

S1
Our records show that you cancelled a Fernway subscription in the last six months. Is that correct?
One answer
- Yes
- No
- Not sure
Continues if Yes
S2
Are you aged 18 or over and currently resident in the UK?
One answer
- Yes
- No
Continues if Yes
S3
Did you personally make, or share in making, the decision to cancel your Fernway subscription?
One answer
- Yes
- No
Continues if Yes
Q1
What was the main reason you cancelled your Fernway subscription?
One answer
- Price or cost
- Not enough flexibility to skip, pause, or choose fewer meals
- Menu or recipe variety
- Food quality
- Delivery or packaging issues
- Portion sizes
- Customer service
- Dietary or allergy needs
- Changed household or lifestyle
- Other
Q2
Before you cancelled, how easy or difficult was it to skip a delivery in the Fernway app?
One answer
- Very easy
- Somewhat easy
- Neither easy nor difficult
- Somewhat difficult
- Very difficult
- I did not try to skip a delivery
- Don't know
Q3
Overall, how satisfied were you with Fernway while you were subscribed?
Scale

Scale 1-5: Very dissatisfied to Very satisfied
Q4
How satisfied were you with the value for money of your Fernway subscription?
Scale

Scale 1-5: Very dissatisfied to Very satisfied
Q5
Please rank the following changes from most to least likely to have kept you subscribed to Fernway. Put the most likely change first.
Ranking
- A 20% lower price
- One-tap skipping of any week
- Being able to pause for longer periods
- An option for fewer meals per week
- More recipe variety
- Better food quality
Q6
If Fernway could offer only one of these, which would be more likely to bring you back?
One answer
- A 20% lower price
- One-tap skipping of any week
- Neither would bring me back
- Not sure
Q7
How likely is it that you would have stayed with Fernway if you could skip any week with one tap?
Scale

Scale 1-5: Not at all likely to Extremely likely
Q8
How likely is it that you would have stayed with Fernway if the price had been 20% lower?
Scale

Scale 1-5: Not at all likely to Extremely likely
Q9
How likely would you be to rejoin Fernway now if one-tap skipping of any week were available?
Scale

Scale 1-5: Not at all likely to Extremely likely
Q10
How likely would you be to rejoin Fernway now if the price were 20% lower?
Scale

Scale 1-5: Not at all likely to Extremely likely
Q11
Are you currently using another meal-kit subscription?
One answer
- Yes
- No
- Not sure
Q12
What is the one thing Fernway could change that would most likely make you consider rejoining?
Open text

Sample plan

Invite every UK customer in the customer database who cancelled a Fernway subscription in the six-month period by email, and screen with S1-S3. Target about 600 completes. Use database tenure to stratify into under 3 months, 3-12 months, and over 12 months, with a quota of about 200 completes per tenure group (600 total) to support group comparisons; send reminders and stop reminders for a tenure group when its quota is met. Important precision note: with 200 completes per tenure group, the 95% confidence interval for a proportion near 50% is about ±6.9 percentage points per group, not ±5. The brief's ±5 requirement needs about 385 completes per tenure group, or about 1,155 total at p=0.5; about 600 completes cannot meet ±5 for all three tenure groups. If the board requires ±5 per group, increase the total to about 1,155 completes or relax the margin. Report achieved n and confidence intervals by tenure group. If the achieved sample is not proportional to the cancelled population, apply design weights for overall estimates but keep unweighted group n for precision. Use complete responses for core questions; exclude partials unless all core questions are answered.