Mercury 3: Why Fernway customers cancel

The decision: whether to build one-tap skipping of any week (about four months of product work) or cut the price of the plans by 20%, to win back and keep customers. The brief and the data →

Verdict: ✗ Not sound
Research score: 42 of 100
Analysis rating: 752
Head to head, this task: won 1 of 13

Why it is not sound

The survey

A double-barrelled question: Q3

The analysis

Got at least one set number wrong
Made a recommendation that does not follow from the data
Missed a trap: Long-tenure customers are 50% of respondents but 20% of cancellers; short-tenure customers, who cite price more, are under-represented.
Missed a trap: The chief executive believes customers leave mainly because of price.
Missed a trap: Only 702 of 9,850 invited customers started the survey (about 7%); those who respond may differ from those who do not.
Missed a trap: Only customers who cancelled were surveyed, so the survey cannot show what distinguishes leavers from customers who stay; the reasons are self-reported.
Missed a trap: The one-tap skipping question (Q3) came immediately before the price-cut question (Q4), so answers to Q4 may have been framed by Q3.

Scorecard

Numbers 0 of 11 right

Question		Its answer	True
What percentage of respondents gave price as their main reason for cancelling?	✗	not given	21.6%
Weighted to the tenure mix of all customers who cancelled, what percentage would give price as their main reason?	✗	not given	26.3%
Among respondents whose main reason was that it was hard to skip weeks, what percentage said they would be likely (fairly or very) to come back if any week could be skipped with one tap?	✗	not given	58.6%
How many of the open-text answers mention recipe variety or repetitive recipes?	✗	not given	40
What is the 95% margin of error, in percentage points, on the unweighted share of respondents giving price as their main reason?	✗	not given	3.19 points
Is the difference in the share satisfied (4-5) between Family and Couples respondents, across all respondents, statistically significant at the 95% level? (1 for yes, 0 for no)	✗	not given	No
Weighted to the tenure mix of all 9,850 customers who cancelled, about how many of them would give 'hard to skip weeks' as their main reason?	✗	not given	3,193
How many of the open-text answers from respondents who were customers for over 12 months mention recipe variety or repetitive recipes?	✗	not given	25
Among respondents whose main reason was that it was hard to skip weeks, by how many percentage points does the share likely to come back with one-tap skipping exceed the share likely to come back with a 20% price cut?	✗	not given	35.7 points
What percentage of the customers invited completed the survey?	✗	not given	6.5%
Among respondents whose main reason was price, what percentage said they would be likely (fairly or very) to come back if the plan cost 20% less?	✗	not given	49.3%

Traps in the data 3 of 8 handled

✓Northern Ireland respondents (base 12) cite delivery problems far more often.
✓Family-plan respondents look less satisfied overall, but are more satisfied than Couples within every tenure band: Family customers skew to long tenure, and long-tenure customers are less satisfied.
✗Long-tenure customers are 50% of respondents but 20% of cancellers; short-tenure customers, who cite price more, are under-represented.
✓'Likely to come back' answers are stated intentions about a hypothetical feature.
✗The chief executive believes customers leave mainly because of price.
✗Only 702 of 9,850 invited customers started the survey (about 7%); those who respond may differ from those who do not.
✗Only customers who cancelled were surveyed, so the survey cannot show what distinguishes leavers from customers who stay; the reasons are self-reported.
✗The one-tap skipping question (Q3) came immediately before the price-cut question (Q4), so answers to Q4 may have been framed by Q3.

The analysis

Recommendation

Findings

The survey it planned

3 screening questions and 9 questions, as the model wrote them.

S1
Are you aged 18 or over?
One answer
- Yes
- No
Continues if Yes
S2
Did your Fernway subscription end in the last 6 months?
One answer
- Yes
- No
Continues if Yes
S3
Do you currently have an active Fernway subscription?
One answer
- Yes
- No
Continues if No
Q1
When you cancelled your Fernway subscription, which one of the following was the main reason?
One answer
- The price was too high / it was not good value for money
- I wanted more flexibility to skip weeks or pause deliveries
- The portion sizes were wrong (too much or too little food)
- I did not like the recipes / the menu became repetitive
- Cooking time or effort was too much
- Delivery issues (late, missed, or damaged boxes)
- I found a cheaper or better alternative
- My circumstances changed (e.g., finances, work, health, holidays)
- Another reason
Q2
Overall, how satisfied were you with Fernway while you were a subscriber?
Scale

Scale 1-5: Very dissatisfied to Very satisfied
Q3
In the Fernway app, how easy or difficult did you find it to skip or change a delivery?
One answer
- Very easy
- Easy
- Neither easy nor difficult
- Difficult
- Very difficult
- I never tried to skip or change a delivery
Judges: double-barrelled
Skipping and changing a delivery can have different levels of difficulty, but respondents can give only one ease rating.
Skipping a delivery and changing a delivery are different actions; one could be easy while the other is difficult.
Q4
When you decided to cancel, which mattered more to you: the price or the flexibility to skip or pause deliveries?
One answer
- The price mattered more
- The flexibility to skip or pause mattered more
- Both mattered equally
- Neither mattered much
- Don't know
Q5
If you had to choose between two possible Fernway options, which would have been more valuable to you as a subscriber?
One answer
- A 20% lower price
- One-tap skipping of any week in the app
- Both would have been equally valuable
- Neither would have made a difference
Q6
Would you consider resubscribing to Fernway in the future?
One answer
- Definitely yes
- Probably yes
- Unsure
- Probably not
- Definitely not
Q7
If Fernway were to offer one of the following changes, which would most encourage you to resubscribe?
One answer
- A 20% lower price
- One-tap skipping of any week
- The ability to pause deliveries for several weeks
- Fewer meals per week at a lower price
- More recipe variety
- Nothing would encourage me to resubscribe
Q8
How appealing would a Fernway option with fewer meals per week (for example, two meals instead of three) be to you?
Scale

Scale 1-5: Not at all appealing to Extremely appealing
Q9
Is there anything else you would like to tell us about your experience with Fernway or why you cancelled?
Open text

Sample plan

Invite all customers in the Fernway customer database who cancelled in the last six months. Target 600 completed surveys, with soft quotas of up to 200 per tenure group (under 3 months, 3-12 months, over 12 months), using CRM tenure data. If a tenure group is smaller, invite all of them and note the resulting precision. Precision note: +/-5 percentage points at 95% confidence for each tenure group requires ~385 completes per group (~1,155 total), so a 600-complete cap cannot guarantee that precision. We should either raise the total to about 1,155 or agree to report wider confidence intervals (e.g., +/-6.9pp for 200 per group).