Benchmarks / SurveyBench / Why Fernway customers cancel / Muse Wonder
Muse Wonder: Why Fernway customers cancel
The decision: whether to build one-tap skipping of any week (about four months of product work) or cut the price of the plans by 20%, to win back and keep customers. The brief and the data →
- Verdict
- ✗ Not sound
- Research score
- 95 of 100
- Analysis rating
- 1,460
- Head to head, this task
- won 10 of 16
Why it is not sound
The survey
- Ignored the precision the brief asked for, which the sample cannot deliver
- A double-barrelled question: Q3
- A double-barrelled question: Q4
Scorecard
Numbers 11 of 11 right
| Question | Its answer | True | |
|---|---|---|---|
| What percentage of respondents gave price as their main reason for cancelling? | ✓ | 21.6% | 21.6% |
| Weighted to the tenure mix of all customers who cancelled, what percentage would give price as their main reason? | ✓ | 26.3% | 26.3% |
| Among respondents whose main reason was that it was hard to skip weeks, what percentage said they would be likely (fairly or very) to come back if any week could be skipped with one tap? | ✓ | 58.6% | 58.6% |
| How many of the open-text answers mention recipe variety or repetitive recipes? | ✓ | 40 | 40 |
| What is the 95% margin of error, in percentage points, on the unweighted share of respondents giving price as their main reason? | ✓ | 3.2 points | 3.19 points |
| Is the difference in the share satisfied (4-5) between Family and Couples respondents, across all respondents, statistically significant at the 95% level? (1 for yes, 0 for no) | ✓ | No | No |
| Weighted to the tenure mix of all 9,850 customers who cancelled, about how many of them would give 'hard to skip weeks' as their main reason? | ✓ | 3,193 | 3,193 |
| How many of the open-text answers from respondents who were customers for over 12 months mention recipe variety or repetitive recipes? | ✓ | 25 | 25 |
| Among respondents whose main reason was that it was hard to skip weeks, by how many percentage points does the share likely to come back with one-tap skipping exceed the share likely to come back with a 20% price cut? | ✓ | 35.7 points | 35.7 points |
| What percentage of the customers invited completed the survey? | ✓ | 6.5% | 6.5% |
| Among respondents whose main reason was price, what percentage said they would be likely (fairly or very) to come back if the plan cost 20% less? | ✓ | 49.3% | 49.3% |
Traps in the data 8 of 8 handled
- ✓Northern Ireland respondents (base 12) cite delivery problems far more often.
- ✓Family-plan respondents look less satisfied overall, but are more satisfied than Couples within every tenure band: Family customers skew to long tenure, and long-tenure customers are less satisfied.
- ✓Long-tenure customers are 50% of respondents but 20% of cancellers; short-tenure customers, who cite price more, are under-represented.
- ✓'Likely to come back' answers are stated intentions about a hypothetical feature.
- ✓The chief executive believes customers leave mainly because of price.
- ✓Only 702 of 9,850 invited customers started the survey (about 7%); those who respond may differ from those who do not.
- ✓Only customers who cancelled were surveyed, so the survey cannot show what distinguishes leavers from customers who stay; the reasons are self-reported.
- ✓The one-tap skipping question (Q3) came immediately before the price-cut question (Q4), so answers to Q4 may have been framed by Q3.
The analysis
Prioritise the one-tap skip build over a blanket 20% price cut, but scope it as broader flexibility after rapid validation: one-tap skip/pause, later skip deadline, multi-week pause and, if feasible, variable/fewer meals per week. Do not implement a universal 20% cut on this evidence; instead test targeted price fairness/loyalty or introductory-price-transition offers for price-sensitive segments. Treat recipe variety as a separate retention workstream for long-tenure customers.
Evidence that actual behavioural response reverses the survey ranking: e.g. a randomised win-back or in-product experiment showing a 20% cut produces materially more retained/rejoined customers per pound of margin lost than one-tap skipping; CRM/ops data showing hard-to-skip leavers rarely return in practice; usability or engineering evidence that one-tap skip cannot be delivered as assumed; or representative weighted/behavioural data showing price is the dominant causal driver once non-response is corrected.
Findings
-
Build one-tap skipping before a blanket 20% price cut: hard to skip is the largest stated reason for leaving and the intervention with the larger stated win-back response.
Q1: Hard to skip 35.5% unweighted, 32.4% tenure-weighted vs Price 21.6% unweighted, 26.3% weighted. Q3 net likely to return with one-tap skip 36.4% vs Q4 25.8% with 20% less. Among hard-to-skip leavers 58.6% would likely return with one-tap skip vs 22.9% with a price cut, a 35.7pp gap.
-
CEO question: No, price is important but is not the main confirmed reason people leave. It is second, and a price cut mainly appeals to the minority who left for price.
Price is 21.6% unweighted, 26.3% after tenure weighting (95% margin approx +/-3.2pp unweighted). Among price leavers, 49.3% say likely to return at 20% less, but price leavers are only 138 respondents; among all others the price-cut likely-return share is 9.7-22.9%. Open text also contains many price-rise, introductory-price and loyalty complaints, suggesting targeted price fairness may matter more than a universal cut.
-
Head of Product question: There is no evidence the Family plan itself is the satisfaction problem. The apparent Family gap is a tenure-mix artefact.
Net satisfied: Couples 53.4% (n=339) vs Family 49.2% (n=301), difference 4.2pp, not significant at 95% (approx +/-7.8pp for the difference). Within tenure, Family is equal or higher: Under 3m 69.2% vs 66.1%, 3-12m 61.6% vs 57.4%, Over 12m 42.1% vs 35.2%. Family respondents are disproportionately long-tenure, and long-tenure satisfaction is much lower overall.
-
Satisfaction and reasons deteriorate/change with tenure: long-tenure customers are less satisfied and more likely to cite repetitiveness and skipping difficulty, while early leavers are more price- and delivery-sensitive.
Net satisfied falls from 66.7% under 3 months and 59.0% at 3-12 months to 39.7% over 12 months. Recipes repetitive rises 6.5% to 14.4% to 25.4%; hard to skip rises 26.1% to 36.9% to 38.8%; price falls 31.2% to 27.2% to 13.7%. This is consistent with novelty wearing off and flexibility needs accumulating, not proof of causation.
-
Operations Director question: Northern Ireland shows a strong delivery warning signal, but the survey base is far too small to diagnose a Northern Ireland delivery problem.
Delivery problems as main reason: Northern Ireland 66.7% but base n=12, i.e. 8 respondents, vs England 8.1% (n=546), Scotland 13.0% (n=46), Wales 0.0% (n=36). The NI estimate has a very wide interval; operational delivery data by region must be checked before acting.
-
Open text reveals a related unmet need not directly tested in Q3/Q4: reducing or varying the number of meals, not only skipping whole weeks.
190 open-text answers include repeated requests for two-meal, one-meal, fortnightly and variable-quantity options, e.g. R007, R011, R059, R098, R138, R206, R270, R328, R361, R634. This may be part of the same flexibility problem as skipping and could increase the value, or change the design, of the flexibility work.
-
Recipe variety is a material retention issue, especially for long-tenure customers, and neither candidate decision directly fixes it.
Recipes got repetitive is 18.0% overall as main reason and 25.4% over 12 months. In open text, 40 of 190 answers mention variety/repetitiveness, 25 of them from over-12-month customers. Among repetitive-recipe leavers, neither one-tap skip (28.7% likely) nor 20% price cut (19.1% likely) recovers a majority.
-
Stated likelihood to return is not a forecast of actual return. Both options leave most leavers unlikely or neutral, so neither should be sold internally as recovering most cancellers.
Even the stronger option, one-tap skip, has only 36.4% net likely overall, with 41-42% unlikely and 21.2% neutral. For price cut the equivalent is 25.8% likely and 51.2% unlikely. Q3/Q4 are hypothetical, unpriced against real behaviour and subject to intention-behaviour gap.
Caveats
- Only 640 of 9,850 invited customers completed the survey (6.5%), so non-response bias is a major risk; responders may be more dissatisfied, more engaged or have stronger reasons than typical cancellers.
- The sample is badly out of line with the canceller population: over-12-month customers are 48.0% of responses but 20% of cancellers, under-3-month are 21.6% vs 45%; Family is 47.0% vs 34%. Unweighted headlines therefore understate price and overstate hard-to-skip/repetitiveness. Tenure weighting corrects tenure only, not other non-response bias.
- All results are unweighted column percentages from cross-sectional self-report after cancellation; stated main reason may be post-hoc, simplified to one reason, and stated return intent is not observed behaviour.
- Q3 and Q4 are not a clean experiment: respondents answered both hypotheticals, wording and order effects are possible, and neither option's profitability, cannibalisation or effect on existing customers is measured.
- Small bases must not be over-read: Northern Ireland n=12, Wales n=36, Scotland n=46; several Q1 reason subgroups are n=28-58; open text is only 190 of 640 respondents and is self-selected.
- Open-text coding counts explicit mentions only; an answer can express multiple themes, and absence of a mention does not mean the issue did not matter. Several answers conflate related ideas such as skipping, pausing, fortnightly delivery and fewer meals.
- The survey covers people who cancelled March-August 2025 and were emailed in September; it says nothing directly about current customers at risk, never-customers, or win-back economics.
Next steps
- decisionDecision: choose flexibility build as the primary investment and reject/postpone blanket 20% price cut, with explicit margin guardrails.
- experimentRun a randomised win-back experiment to past cancellers: one-tap-skip message/offer vs 20%-off offer vs control, segmented by original reason, tenure and plan.
- researchPrototype and usability-test one-tap skip plus pause/hold, late deadline, multi-week skip and variable meal count with current and cancelled customers.
- researchAnalyse operational delivery, missing-item and complaint data by nation/region, especially Northern Ireland, and link to cancellation.
- monitorRe-weight and if needed top-up survey analysis by tenure, plan and reason; track current customers' skip attempts, failed skips and cancellations after skip friction.
- experimentTest targeted price interventions - loyalty discount, price-rise protection, extended introductory pricing - rather than a universal cut.
Open-text themes it coded
Easy skip, pause or holiday hold 46Price, value, price rises or loyalty discounts 42Recipe variety or repetitive recipes 40Fewer, smaller or variable number of meals per week 28Delivery reliability or missing items 16Portion sizes too small or adjustable portions 12Packaging waste or non-recyclable packaging 19Nothing - life circumstances changed 6
The survey it planned
3 screening questions and 12 questions, as the model wrote them.
-
S1
Which of the following best describes you?
One answer- I cancelled a Fernway subscription in the last 6 months
- I currently have a Fernway subscription and have not cancelled it
- I paused a Fernway subscription but did not cancel it
- I have never had a Fernway subscription
- Someone else in my household cancelled a Fernway subscription, but I did not
Continues if I cancelled a Fernway subscription in the last 6 months
-
S2
Where do you currently live?
One answer- England
- Scotland
- Wales
- Northern Ireland
- Outside the United Kingdom
Continues if England; Scotland; Wales; Northern Ireland
-
S3
Are you aged 18 or over?
One answer- Yes
- No
Continues if Yes
-
Q1
Overall, how satisfied or dissatisfied were you with Fernway while you were subscribed?
ScaleScale 1-5: Very dissatisfied to Very satisfied
-
Q2
In your own words, why did you cancel your Fernway subscription? Please be as specific as you can.
Open text -
Q3
Which of the following, if any, were reasons why you cancelled your Fernway subscription? Please select all that apply.
Any that apply- It was too expensive for my budget
- I did not think it was good value for money
- I could not easily skip weeks I did not want
- I could not easily pause my subscription
- The plans did not offer a smaller number of meals per week that suited me
- I did not enjoy the recipes, or there was not enough choice
- The quality or freshness of the ingredients was not good enough
- The portion sizes did not suit my household
- The recipes took too much time or effort to cook
- There was too much packaging or food waste
- Deliveries were unreliable or at inconvenient times
- My household circumstances changed
- I only wanted to use an introductory offer
- I found another way of eating that suited me better
- Another reason not listed here
- None of these
Judges: double-barrelledSeveral options combine distinct cancellation reasons into one answer: recipe enjoyment versus choice, packaging versus food waste, and delivery reliability versus inconvenient timing. Selecting an option does not identify which reason applied.
Several options combine two distinct reasons (e.g., 'packaging or food waste', 'unreliable or at inconvenient times', 'did not enjoy the recipes, or there was not enough choice').
-
Q4
And which one of these was the single most important reason why you cancelled your Fernway subscription? Please select one answer.
One answer- It was too expensive for my budget
- I did not think it was good value for money
- I could not easily skip weeks I did not want
- I could not easily pause my subscription
- The plans did not offer a smaller number of meals per week that suited me
- I did not enjoy the recipes, or there was not enough choice
- The quality or freshness of the ingredients was not good enough
- The portion sizes did not suit my household
- The recipes took too much time or effort to cook
- There was too much packaging or food waste
- Deliveries were unreliable or at inconvenient times
- My household circumstances changed
- I only wanted to use an introductory offer
- I found another way of eating that suited me better
- Another reason not listed here
- None of these
Judges: double-barrelledSeveral options combine distinct reasons, so the answer cannot identify a single most important reason—for example, recipe enjoyment versus choice or delivery reliability versus inconvenient timing.
Uses the same double-barrelled options as Q3.
-
Q5
To check that you are reading the questions carefully, please select ‘Blue’ from the list below.
One answer- Red
- Blue
- Green
- Yellow
-
Q6
If Fernway stayed exactly as it was when you cancelled, how likely or unlikely would you be to rejoin in the next 6 months?
ScaleScale 1-5: Very unlikely to Very likely
-
Q7
Please rank the following possible changes from 1 (most likely to make you consider rejoining Fernway) to 5 (least likely to make you consider rejoining Fernway). Please use each number once.
Ranking- Being able to skip any week with one tap in the app, without changing your delivery at least six days ahead
- A 20% reduction in the price of the Couples and Family plans
- Being able to pause your subscription for several weeks at a time and restart it when you choose
- Being able to order fewer meals per week than the current plans offer
- A wider choice of recipes each week
-
Q8
If the only change Fernway made was to let you skip any week with one tap in the app, how likely or unlikely would you be to rejoin in the next 6 months?
ScaleScale 1-5: Very unlikely to Very likely
-
Q9
If the only change Fernway made was to reduce the price of its plans by 20%, how likely or unlikely would you be to rejoin in the next 6 months?
ScaleScale 1-5: Very unlikely to Very likely
-
Q10
If Fernway could make only one of the following changes, which one, if any, would make you most likely to rejoin? Please select one answer.
One answer- Let you skip any week with one tap in the app
- Reduce the price of its plans by 20%
- I would need both of these changes to consider rejoining
- Neither of these changes would make me likely to rejoin
- I would only consider rejoining if something else changed
-
Q11
For about how long were you a Fernway customer before you cancelled?
One answer- Less than 3 months
- 3 to 12 months
- More than 12 months
- I don’t remember
-
Q12
Which Fernway plan were you on most recently before you cancelled?
One answer- Couples plan (recipe box for two)
- Family plan (recipe box for four)
- I was on both at different times
- I don’t remember
Sample plan
Email-invite every customer in the Fernway database who cancelled in the last six months; do not sample down, because the database already defines the eligible UK population. Do not stop at about 600 completes: 600 completes split across three tenure groups (about 200 per group) would give a worst-case margin of about ±6.9 percentage points per group at 95% confidence, which misses the board requirement. For ±5 percentage points at 95% confidence at the worst case (50%), each tenure group needs at least 385 valid completes (1.96² × 0.25 ÷ 0.05² = 384.2), so require at least 385 and target 400 valid completes in each of the under-3-months, 3–12-months and over-12-months groups, i.e. at least 1,155 and a target of about 1,200 valid completes in total after excluding respondents who fail Q5. Use tenure recorded in the customer database at invitation to monitor group fills against Q11, send reminders to all non-respondents first and then targeted reminders to any tenure group below quota, and keep fieldwork open until every group reaches 385 or the invited population is exhausted; if a group remains below 385, report its achieved margin rather than claiming ±5 points. Report each tenure group separately and do not let the largest group compensate for a shortfall in another.