Benchmarks / SurveyBench / Why Fernway customers cancel / Gemini 4 Neon

Measured by Spring Prompt

Gemini 4 Neon: Why Fernway customers cancel

The decision: whether to build one-tap skipping of any week (about four months of product work) or cut the price of the plans by 20%, to win back and keep customers. The brief and the data →

Verdict
✗ Not sound
Research score
28 of 100
Analysis rating
629
Head to head, this task
won 0 of 6

Why it is not sound

The survey

  • A double-barrelled question: Q8
  • A double-barrelled question: Q12

The analysis

  • Produced no usable analysis

Scorecard

The analysis

This model produced no usable analysis for this task.

The survey it planned

2 screening questions and 12 questions, as the model wrote them.

  1. S1

    Which of the following best describes your current status with Fernway?

    One answer
    • I cancelled my Fernway subscription in the last 6 months and am not currently subscribed
    • I cancelled my Fernway subscription more than 6 months ago
    • I currently have an active Fernway subscription
    • I have never had a Fernway subscription

    Continues if I cancelled my Fernway subscription in the last 6 months and am not currently subscribed

  2. S2

    How involved were you in the decision to cancel your Fernway subscription?

    One answer
    • I made the decision myself
    • I made the decision jointly with someone else in my household
    • Someone else in my household made the decision without my involvement

    Continues if I made the decision myself; I made the decision jointly with someone else in my household

  3. Q1

    Roughly how long were you subscribed to Fernway before you cancelled?

    One answer
    • Less than 3 months
    • 3 to 12 months
    • More than 12 months
    • Not sure
  4. Q2

    Which Fernway plan were you on most recently before you cancelled?

    One answer
    • Couples plan (for 2 people)
    • Family plan (for 4 people)
    • I switched between plans
    • Not sure
  5. Q3

    While you were subscribed, how satisfied or dissatisfied were you with Fernway's meal kits overall (including the recipes, ingredients, and deliveries)?

    Scale

    Scale 1-5: Very dissatisfied to Very satisfied

  6. Q4

    In your own words, what was the main reason you decided to cancel your Fernway subscription?

    Open text
  7. Q5

    Which of the following reasons contributed to your decision to cancel Fernway? Please select all that apply.

    Any that apply
    • The recipe variety or menu options did not suit me
    • Issues with ingredient freshness, quality, or missing items
    • Recipes took too long or were too complicated to cook
    • Delivery issues (such as late, missed, or damaged boxes)
    • Skipping a week was too inconvenient or the 6-day deadline was too early
    • I wanted to pause my subscription or order less often than weekly
    • There were too many meals per week or too much food waste
    • The weekly price was too high for my budget
    • It did not feel like good value for money
    • My household circumstances or routine changed (for example, moving, travel, or dietary changes)
    • I switched to another meal-kit service or regular supermarket shopping
    • Other reason
  8. Q6

    Thinking about your decision to cancel, which single reason was the most important? (If you had only one reason, please select it below.)

    One answer
    • The recipe variety or menu options did not suit me
    • Issues with ingredient freshness, quality, or missing items
    • Recipes took too long or were too complicated to cook
    • Delivery issues (such as late, missed, or damaged boxes)
    • Skipping a week was too inconvenient or the 6-day deadline was too early
    • I wanted to pause my subscription or order less often than weekly
    • There were too many meals per week or too much food waste
    • The weekly price was too high for my budget
    • It did not feel like good value for money
    • My household circumstances or routine changed (for example, moving, travel, or dietary changes)
    • I switched to another meal-kit service or regular supermarket shopping
    • Other reason
    • No single reason stood out
  9. Q7

    While you were subscribed, how often did you receive a Fernway box you did not really want because you missed the 6-day deadline to skip?

    One answer
    • Never — I always managed to skip in time when I wanted to
    • Never — I did not need to skip deliveries
    • Once
    • 2 or 3 times
    • 4 or more times
    • I did not know I could skip a week
  10. Q8

    If you were choosing a meal-kit subscription today, please rank the following features from 1 (most important to you) to 5 (least important to you).

    Ranking
    • Lower weekly price
    • Easy one-tap skipping of any week
    • Ability to pause or switch to fortnightly deliveries
    • Option to order fewer meals per week
    • Wider recipe choice and consistently fresh ingredients
    Judges: double-barrelled

    A single rank combines wider recipe choice with consistently fresh ingredients, although their importance may differ. Pausing and switching to fortnightly deliveries are also combined in one ranked option.

    The item 'Wider recipe choice and consistently fresh ingredients' joins two separate features into one item that gets a single rank. A respondent who values one but not the other cannot rank it accurately.

  11. Q9

    To help us check the quality of survey responses, please select 'Every two weeks' below.

    One answer
    • Every week
    • Every two weeks
    • Once a month
    • Less than once a month
  12. Q10

    How likely or unlikely would you be to resubscribe to Fernway in the next 6 months if the price of your plan were reduced by 20%?

    Scale

    Scale 1-5: Very unlikely to Very likely

  13. Q11

    How likely or unlikely would you be to resubscribe to Fernway in the next 6 months if Fernway offered one-tap skipping of any week?

    Scale

    Scale 1-5: Very unlikely to Very likely

  14. Q12

    If you had to pick just one change that would be most likely to win you back to Fernway, which would it be?

    One answer
    • A 20% reduction in the price of my plan
    • One-tap skipping of any week
    • Ability to pause my subscription or order fortnightly
    • Option to order fewer meals per week
    • Wider recipe choice or improved ingredient quality
    • More flexible delivery days or times
    • Nothing — I am unlikely to resubscribe to Fernway
    Judges: double-barrelled

    Despite asking for one change, an answer combines wider recipe choice with improved ingredient quality, so selecting it does not identify which change would win the respondent back. Pausing and fortnightly ordering are likewise combined.

    Options group distinct concepts (e.g., 'Wider recipe choice or improved ingredient quality', 'Ability to pause my subscription or order fortnightly').

Sample plan

Invite all UK customers in the Fernway CRM database who cancelled a subscription within the past 6 months (excluding employees/test accounts, reactivated subscribers, and email opt-outs) via personalised survey links that append CRM metadata (exact tenure, plan type, and cancellation month) while keeping responses confidential. Send up to two reminder emails over a 14-day field window and offer a neutral incentive (e.g., entry into a £50 supermarket or general retail voucher draw rather than Fernway credit, to avoid biasing the sample toward customers willing to return). Randomise non-anchor answer option order in Q5, Q6, Q8, and Q12, and counterbalance the presentation order of Q10 and Q11 (50/50 split) to eliminate order bias. Quotas: target equal strata across the three tenure groups (under 3 months, 3-12 months, over 12 months) while monitoring natural fallout across Couples and Family plans; apply post-stratification weights to match the true CRM tenure and plan distribution when reporting overall totals. Precision flag for the board: achieving +/-5 percentage points at 95% confidence (worst-case p=0.5) within each tenure group requires approximately 384 completes per group (~1,152 total, assuming a large population). With the budgeted 600 completes (~200 per tenure quota), margin of error is +/-4.0 percentage points overall but approximately +/-6.9 percentage points per tenure subgroup unless finite population correction (FPC) applies because a tenure tier has a small total churned base (e.g., N<=400). Before launch, check exact 6-month churn counts per tenure tier and either raise the sample target toward ~1,150 completes (if database size and response rate allow) or align the board on ~+/-7 percentage points per subgroup at n=600.