GPT-6.1 Sol: Why Fernway customers cancel

The decision: whether to build one-tap skipping of any week (about four months of product work) or cut the price of the plans by 20%, to win back and keep customers. The brief and the data →

Verdict: ✓ Sound research
Research score: 100 of 100
Analysis rating: 1,650
Head to head, this task: won 9 of 9

Scorecard

Numbers 11 of 11 right

Question		Its answer	True
What percentage of respondents gave price as their main reason for cancelling?	✓	21.6%	21.6%
Weighted to the tenure mix of all customers who cancelled, what percentage would give price as their main reason?	✓	26.3%	26.3%
Among respondents whose main reason was that it was hard to skip weeks, what percentage said they would be likely (fairly or very) to come back if any week could be skipped with one tap?	✓	58.6%	58.6%
How many of the open-text answers mention recipe variety or repetitive recipes?	✓	40	40
What is the 95% margin of error, in percentage points, on the unweighted share of respondents giving price as their main reason?	✓	3.19 points	3.19 points
Is the difference in the share satisfied (4-5) between Family and Couples respondents, across all respondents, statistically significant at the 95% level? (1 for yes, 0 for no)	✓	No	No
Weighted to the tenure mix of all 9,850 customers who cancelled, about how many of them would give 'hard to skip weeks' as their main reason?	✓	3,193	3,193
How many of the open-text answers from respondents who were customers for over 12 months mention recipe variety or repetitive recipes?	✓	25	25
Among respondents whose main reason was that it was hard to skip weeks, by how many percentage points does the share likely to come back with one-tap skipping exceed the share likely to come back with a 20% price cut?	✓	35.7 points	35.7 points
What percentage of the customers invited completed the survey?	✓	6.5%	6.5%
Among respondents whose main reason was price, what percentage said they would be likely (fairly or very) to come back if the plan cost 20% less?	✓	49.3%	49.3%

Traps in the data 8 of 8 handled

✓Northern Ireland respondents (base 12) cite delivery problems far more often.
✓Family-plan respondents look less satisfied overall, but are more satisfied than Couples within every tenure band: Family customers skew to long tenure, and long-tenure customers are less satisfied.
✓Long-tenure customers are 50% of respondents but 20% of cancellers; short-tenure customers, who cite price more, are under-represented.
✓'Likely to come back' answers are stated intentions about a hypothetical feature.
✓The chief executive believes customers leave mainly because of price.
✓Only 702 of 9,850 invited customers started the survey (about 7%); those who respond may differ from those who do not.
✓Only customers who cancelled were surveyed, so the survey cannot show what distinguishes leavers from customers who stay; the reasons are self-reported.
✓The one-tap skipping question (Q3) came immediately before the price-cut question (Q4), so answers to Q4 may have been framed by Q3.

The analysis

Recommendation

Provisionally favour one-tap skipping over a blanket 20% price cut, but gate the four-month commitment on a rapid behavioural pilot and an economic case. Skipping remains the leading cancellation reason after tenure adjustment and has stronger stated win-back appeal, particularly among people reporting skipping difficulties. Do not translate those intentions into a revenue forecast. First verify whether a simpler button would solve the actual barriers or whether deadlines, pause limits and delivery cadence are decisive. Model the cost of development and the reduction in paid boxes caused by easier skipping against retained contribution margin; compare this with the margin sacrificed by a permanent price reduction. Test targeted price offers for price-sensitive leavers rather than discounting every plan immediately. In parallel, investigate Northern Ireland delivery performance and recipe repetition. Do not redesign or discount the Family plan solely because its aggregate satisfaction is lower.

What would change it

A representative, tenure-and-plan-calibrated analysis showing substantially stronger price demand would weaken the skipping recommendation, but observed behaviour should decide it. Change to price reduction if a controlled test shows that 20% lower pricing produces greater incremental contribution profit over a common time horizon than easier skipping, after accounting for discounts to customers who would otherwise pay full price, repeat purchase, cannibalisation and implementation costs. Also reconsider the specified one-tap build if research shows that most skipping-related cancellations arise from quantity, frequency or operational cut-offs that the proposed feature cannot change, or if a pilot fails to improve sustained paid activity enough to justify four months of work.

Findings

Chief executive: price matters, especially for newer customers, but the survey does not confirm that price is the dominant reason people leave. A blanket 20% reduction is not supported by this evidence.

Medium confidenceBase 640

Price was the main reason for 21.6% of 640 respondents, versus 35.5% for difficulty skipping. Adjusting to the population tenure mix raises price to 26.3%, but skipping still leads at 32.4%. Price was selected by 31.2% of customers with under three months' tenure, compared with 13.7% of those with over 12 months. Among 138 price leavers, 49.3% said they would likely return with a 20% price cut; this supports testing targeted price offers, not assuming the same effect for everyone.
One-tap skipping is the stronger provisional investment, but stated interest is not a forecast of recovered customers or profit.

Medium confidenceBase 640

Across 640 respondents, 36.4% said they would likely return with one-tap skipping, versus 25.8% with a 20% price cut. Among the 227 respondents who left because skipping was difficult, the corresponding figures were 58.6% and 22.9%, a 35.7-point advantage for skipping. Tenure-weighted cancellation reasons suggest approximately 3,193 of the 9,850 cancellers would identify skipping as their main reason; this is an extrapolated segment size, not an expected win-back count.
Head of product: the aggregate satisfaction figures do not establish that the Family plan is the problem. Its lower overall score is entangled with its much longer-tenured respondent mix.

High confidenceBase 640

Satisfied shares were 49.2% for Family respondents (n=301) and 53.4% for Couples respondents (n=339). The approximately 4.2-point difference is not significant at 95% under a conventional independent-proportions test; the approximate confidence interval for Family minus Couples is -12.0 to +3.6 points. Family satisfaction is higher within every tenure band: 69.2% versus 66.1%, 61.6% versus 57.4%, and 42.1% versus 35.2%. Over-12-month customers make up 67.1% of Family respondents but only 31.0% of Couples respondents. This reversal argues against blaming the plan from the aggregate alone.
Operations director: Northern Ireland has a strong delivery warning that merits immediate investigation, but the survey cannot establish the regional delivery failure rate.

Medium confidenceBase 12

Eight of 12 Northern Ireland respondents identified delivery as their main cancellation reason, or 66.7%, compared with 50 of 628 respondents elsewhere, or 8.0%. R445 also describes late deliveries and missing items. The regional base is very small and includes only responding cancellers; nevertheless, eight reports should not be dismissed as merely a small-base artefact.
Build around control over deliveries, not just a shorter click path. A one-tap button alone may leave important sources of inflexibility unresolved.

Medium confidenceBase 190

Among 190 open-text respondents, 49 mentioned skipping, pausing or less frequent deliveries, and 28 mentioned fewer meals or variable weekly quantities. These themes overlap. Requests include later cut-offs (R295, R433), skipping several weeks together (R220), preserving settings during a pause (R049), and fewer meals per box (R337). These are related but distinct needs that the single 'hard to skip weeks' answer does not separate.
Recipe repetition is a separate retention problem, particularly visible among longer-tenured respondents; neither proposed intervention directly fixes it.

Medium confidenceBase 640

Recipe repetition was the main cancellation reason for 18.0% of all respondents and 25.4% of the 307 respondents with over 12 months' tenure. Forty open-text answers mentioned recipe variety or repetition; 25 of those came from over-12-month customers. Family respondents selected repetition more often than Couples respondents, 23.6% versus 13.0%, but tenure composition may contribute to that difference.

Caveats

Only 640 of 9,850 invited cancellers completed, a 6.5% completion rate. This is a self-selected respondent sample, not a probability sample. Nonresponse bias may be materially larger than conventional sampling error.
The sample substantially overrepresents long-tenured customers: 48.0% had over 12 months' tenure versus 20% in the population. Under-three-month customers are underrepresented at 21.6% versus 45%. Family customers are also overrepresented at 47.0% versus 34%.
The weighted answers use tenure only: price = 0.45×31.2 + 0.35×27.2 + 0.20×13.7 = 26.3%; skipping = 0.45×26.1 + 0.35×36.9 + 0.20×38.8 = 32.42%. Multiplying the latter by 9,850 gives approximately 3,193. Published percentages are rounded, so these are approximate.
Tenure adjustment does not correct all nonresponse bias or plan imbalance. Respondent-level data are needed to calibrate to both known population margins. The supplied tables do not permit tenure-weighted comparisons of Q3 and Q4 without additional assumptions.
The 3.19-point price margin of error uses the conventional 1.96×sqrt(p×(1-p)/640) calculation, with p=138/640 and no finite-population correction. It is a nominal sampling-error benchmark, not a defensible bound on total error for this self-selected survey. The Family significance test has the same sampling-model limitation.
The survey covers former customers only. It cannot directly estimate retention effects among current customers, acquisition effects, actual reactivation or incremental profit.
Q3 and Q4 measure hypothetical intentions, not behaviour. There is no no-change baseline; both options were shown to the same people in a fixed order. Their audiences can overlap, and the aggregate tables do not provide paired responses for analysing individual preferences.
Q1 requires one main reason and supplies the response categories. It misses interacting causes, and its skipping category may combine usability, deadlines, pause limits and an unsuitable weekly cadence.
Only 190 respondents supplied open text, 29.7% of completes. Themes are nonexclusive, manually coded expressions of needs, not population prevalence estimates. The broader flexibility theme includes fortnightly delivery requests as well as explicit skips and pauses.
Some open-text descriptions of household size or boxes do not match the reported plan label, and reported meal minimums vary. These may reflect changing circumstances, recall or configuration differences; verify against account records rather than silently reassigning respondents.
Northern Ireland has only 12 respondents, and no regional customer or delivery-volume population benchmarks are supplied. Its 66.7% figure is a share of responding cancellers' main reasons, not the percentage of Northern Ireland deliveries that failed.
Lower satisfaction among longer-tenured cancellers does not show that tenure causes dissatisfaction. Different cohorts, experiences and selection into cancellation could explain the association.

Next steps

decisionMake a provisional choice for skipping, with a staged funding gate rather than an unconditional four-month build; prepare comparable unit-economic scenarios for both options.
researchObtain respondent-level data, calibrate to the known tenure and plan margins, and examine Q3/Q4 preferences and cancellation reasons within tenure and plan.
researchValidate skipping needs using account records and usability sessions with early and long-tenured leavers, plus current customers considering cancellation.
experimentRun a randomised, operationally feasible pilot comparing easier skipping, a 20% price offer and a no-change control; test former and current customers separately.
researchAudit Northern Ireland delivery and fulfilment records immediately, segmented by courier, route and delivery date; contact the affected respondents where permitted.
experimentMonitor recipe repetition and test a broader rotation for longer-tenured customers, comparing plans within tenure bands.

Open-text themes it coded

Recipe variety or repetitive recipes 40Price, affordability, value for money or loyalty discounts 43Skipping, pausing or less frequent deliveries 49Fewer meals per box or variable weekly quantities 28Delivery reliability, missing ingredients or damaged deliveries 18Packaging waste, recyclability or packaging quality 18Larger or adjustable portion sizes 12Life changes or circumstances with no requested product remedy 6

The survey it planned

1 screening question and 12 questions, as the model wrote them.

S1
This survey takes about 5–6 minutes. We want to understand your experience of Fernway, including what worked and what did not. Have you cancelled a Fernway subscription in the last six months? Please answer yes even if you have subscribed again since then.
One answer
- Yes
- No
- I’m not sure
Continues if Yes
Q1
Thinking about your most recent cancellation, how long had you been a Fernway subscriber immediately before you cancelled? Include weeks when you skipped a delivery, but not any earlier subscriptions that you had already cancelled.
One answer
- Under 3 months
- 3–12 months
- Over 12 months
- I don’t remember
Q2
Which plan were you on immediately before that cancellation?
One answer
- Couples plan: recipe boxes for two
- Family plan: recipe boxes for four
- I don’t remember
Q3
Overall, how satisfied or dissatisfied were you with Fernway during that subscription?
One answer
- Very dissatisfied
- Somewhat dissatisfied
- Neither satisfied nor dissatisfied
- Somewhat satisfied
- Very satisfied
- I don’t remember
Q4
In your own words, what led you to cancel? A short answer is fine. You can leave this question blank.
Open text
Q5
Which of the following contributed to your decision to cancel? Select all that apply.
Any that apply
- The regular price was too high for what I received
- My household needed to reduce spending
- A discount or introductory offer ended
- I could not skip deliveries at short enough notice
- I wanted to pause deliveries for a longer period
- I wanted fewer meals or less frequent deliveries
- The recipes did not offer enough choice or variety
- The meals did not suit our tastes or dietary needs
- I was unhappy with ingredient quality or freshness
- The portions were not right for my household
- Preparing the meals took too much time or effort
- I had problems with deliveries
- I had problems with customer service
- My household circumstances or routine changed
- I preferred another meal kit, shopping for myself or another way of getting meals
- I no longer needed a meal-kit subscription
- Another reason
- I don’t remember
Q6
Which ONE reason had the greatest influence on your decision to cancel?
One answer
- The regular price was too high for what I received
- My household needed to reduce spending
- A discount or introductory offer ended
- I could not skip deliveries at short enough notice
- I wanted to pause deliveries for a longer period
- I wanted fewer meals or less frequent deliveries
- The recipes did not offer enough choice or variety
- The meals did not suit our tastes or dietary needs
- I was unhappy with ingredient quality or freshness
- The portions were not right for my household
- Preparing the meals took too much time or effort
- I had problems with deliveries
- I had problems with customer service
- My household circumstances or routine changed
- I preferred another meal kit, shopping for myself or another way of getting meals
- I no longer needed a meal-kit subscription
- Another reason
- No single reason mattered most
- I don’t remember
Q7
Which ONE of these changes, if any, would make you most likely to consider Fernway today? Assume everything else stayed the same.
One answer
- A permanent 20% reduction in the regular price
- Skipping any upcoming delivery week with one tap in the app, without the current requirement to make the change at least six days ahead
- Being able to pause deliveries for a longer period and choose when to restart
- Being able to order fewer meals or receive deliveries less often
- Better ingredient quality or freshness
- More recipe choice, including meals that suit my tastes or dietary needs
- More reliable deliveries
- Another change
- None of these changes would make me consider Fernway
- I’m not sure
Q8
Have you subscribed to Fernway again since the cancellation you have been telling us about?
One answer
- Yes, and I currently have a Fernway subscription
- Yes, but I have cancelled again
- No
- I’m not sure
Q9
Imagine you were choosing a meal-kit subscription today. If Fernway offered the same regular price and product you had before cancelling, with skips still requiring at least six days’ notice, how likely would you be to choose Fernway in the next four weeks?
One answer
- Very unlikely
- Somewhat unlikely
- Neither likely nor unlikely
- Somewhat likely
- Very likely
- I’m not sure
Q10
Now consider this alternative on its own: Fernway’s regular price is permanently reduced by 20%, so you pay £80 for every £100 you would previously have paid. Skips still require at least six days’ notice, and everything else stays the same. How likely would you be to choose Fernway in the next four weeks?
One answer
- Very unlikely
- Somewhat unlikely
- Neither likely nor unlikely
- Somewhat likely
- Very likely
- I’m not sure
Q11
Consider this different alternative on its own, not combined with the price reduction: Fernway keeps the same regular price you had before cancelling, but you can skip any upcoming delivery week with one tap in the app, without the current requirement to make the change at least six days ahead. Everything else stays the same. How likely would you be to choose Fernway in the next four weeks?
One answer
- Very unlikely
- Somewhat unlikely
- Neither likely nor unlikely
- Somewhat likely
- Very likely
- I’m not sure
Q12
If these were the two available offers, which would you prefer? Everything else about Fernway would be the same.
One answer
- A permanent 20% lower regular price, with skips still requiring at least six days’ notice
- The previous regular price, with one-tap skipping of any upcoming delivery week and no six-day notice requirement
- Both offers would be equally appealing
- Neither offer would appeal to me
- I’m not sure

Sample plan

Invite every unique customer in Fernway’s database who cancelled in the last six months, including those who subsequently resubscribed. Fernway serves only the UK, so no additional country screener is needed. Use unique invitation links, prevent duplicate submissions and send up to two neutral reminders to nonrespondents. Keep the invitation neutral: ask about customers’ experiences, not whether Fernway is too expensive. Link responses securely to subscription dates, plan, cancellation date and resubscription status; use database tenure for quotas and primary analysis, with Q1 as a check. Define tenure as the continuous subscription spell ending in the relevant cancellation, including skipped weeks. Aim for approximately 600 completes, targeting 200 per tenure group: under 3 months, 3–12 months and over 12 months. Monitor response by tenure, plan and cancellation month, and direct reminders towards underrepresented groups. The requested precision is incompatible with 600 completes under standard large-population assumptions: 200 per group gives approximately ±6.9 percentage points at 95% confidence for a proportion near 50%, before weighting or other design effects. Achieving ±5 points requires at least 385 completes per group, or 1,155 total; seek approval to increase the target or explicitly relax the precision requirement. Check known population sizes for any applicable finite-population correction, but do not assume it resolves the shortfall. These sampling margins also do not account for nonresponse bias. Weight aggregate estimates to the eligible cancellation population’s tenure distribution, with additional calibration to plan and cancellation month where feasible; report effective sample sizes. Pilot the programmed questionnaire with 15–20 eligible respondents and confirm a median completion time below six minutes before full launch. Keep Q4 optional. Make 'I don’t remember' exclusive in Q5. Randomize substantive reasons in Q5 and retain the same order in Q6; keep residual answers last. In Q6, display only reasons selected in Q5 plus 'No single reason mattered most' and 'I don’t remember'; if Q5 is unanswered, display the full list. Randomize substantive changes in Q7 and counterbalance the positions of the two substantive offers in Q12, keeping residual answers last. Retain the stated question order and record displayed option orders.