Sign inTest your prompt

Benchmarks / SurveyBench / Why Pennywell customers don't use savings pots / Gemini 3.8 Flash

Measured by Spring Prompt

Gemini 3.8 Flash: Why Pennywell customers don't use savings pots

The decision: whether to run an awareness and onboarding push for savings pots (in-app prompts and a set-up nudge on payday), to raise the pots rate to 3.85% AER, or both. The brief and the data →

Verdict
✗ Not sound
Research score
73 of 100
Analysis rating
898
Head to head, this task
won 9 of 10

Why it is not sound

The survey

  • Ignored the precision the brief asked for, which the sample cannot deliver
  • Answer bands that overlap or leave gaps
  • A double-barrelled question: Q1

The analysis

  • Missed a trap: The chief financial officer believes customers don't use pots because the rate is uncompetitive.

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What percentage of respondents said they knew, before the survey, that they could open savings pots (Yes at Q1)?✓39.4%39.4%
Among respondents who were asked Q3, what percentage chose 'The interest rate is too low'?✓29.7%29.7%
As a percentage of all respondents, how many knew about pots and chose 'The interest rate is too low' at Q3?✓11.7%11.7%
Weighted to the age mix of all customers who have never opened a pot, what percentage of them would know about pots and choose 'The interest rate is too low' at Q3?✓8.8%8.8%
Weighted to the age mix of all 361,000 customers who have never opened a pot, about how many of them did not know they could open savings pots (No or Not sure at Q1)?✓239,702239,702
Among respondents who reported an amount held in savings elsewhere (excluding None, Don't know and Prefer not to say), what percentage hold less than £5,000?✓55.2%55.2%
What percentage of all respondents said they would be likely (fairly or very) to start saving in a pot if the rate rose to 3.85% AER?✓29.9%29.9%
Among respondents asked Q3, is the difference in the share choosing 'The interest rate is too low' between those aged 35-54 and those aged 55-74 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
What is the 95% margin of error, in percentage points, on the share of respondents asked Q3 who chose 'The interest rate is too low'?✓5.4 points5.39 points
How many of the open-text answers say the respondent did not know about savings pots or what they are, or want Pennywell to tell them about pots?✓126126
How many of the open-text answers from respondents aged 18-34 say they did not know about savings pots or what they are, or want Pennywell to tell them about pots?✓4646
What percentage of the customers invited completed the survey?✓6.03%6%

Traps in the data 7 of 8 handled

  • ✓Q3 was asked only of the 276 respondents who knew about pots. 'The interest rate is too low' leads there (30%), but that is 12% of all respondents (about 9% weighted), while 61% of all respondents did not know pots existed (No or Not sure).
  • ✓Customers aged 55 or over are 15% of non-users but 31% of respondents; they know about pots more and cite the rate more, so unweighted figures overstate awareness and the rate reason (weighted, awareness falls from 39% to 34%).
  • ✓The mean amount held elsewhere (£14,513) is pulled up by a few large balances; the median is in the £1,000 to less than £5,000 band and 55% of those reporting an amount hold less than £5,000. Many hold nothing elsewhere.
  • ✓30% say they would be likely to start saving in a pot at 3.85% (17% at the current 3.1%); these are stated intentions about a hypothetical rate.
  • ✓Respondents aged 75 or over (base 20) show 65% holding £20,000 or more elsewhere, against 13% overall.
  • ✓Only 700 of 11,600 invited customers completed (about 6%); those who respond to an email about savings may be more engaged with saving than those who do not.
  • ✗The chief financial officer believes customers don't use pots because the rate is uncompetitive.
  • ✓Among those asked Q3, the rate reason is 10 points higher for 55-74s (38%, base 120) than for 35-54s (27%, base 102), but the difference is not significant at 95%.

The analysis

Recommendation

Execute the awareness and onboarding push (in-app feature discovery, home screen shortcut, streamlined 1-tap creation, and payday set-up nudges) immediately, while keeping the savings pot rate at 3.1% AER. Do not raise the rate to 3.85% AER at this stage, and do not dedicate campaign resources to customers aged 75+.

What would change it

Evidence that would change this recommendation: (1) An in-app payday prompt A/B test shows negligible conversion or high opt-out/dismissal rates; (2) Market-wide rate movements trigger current account deposit attrition to competitors; or (3) A pricing elasticity study demonstrates that a 3.85% rate attracts substantial sticky net-new liquidity whose net interest margin exceeds the 75 bps cost across the pot book.

Findings

  1. Lack of awareness is the primary bottleneck preventing savings pot adoption, far outweighing the interest rate.

    High confidenceBase 700

    60.6% of survey respondents (50.7% No, 9.9% Not sure) were unaware of pots before the survey. When weighted to the actual customer population, 66.4% (~240,000 customers) do not know pots exist. Over half of all open-text comments (126 of 230) explicitly cite lack of knowledge or demand basic explainers.

  2. The CFO's premise that 'rate is the top reason' is an artifact of survey routing and overrepresents a minority.

    High confidenceBase 276

    Q3 was only shown to the 276 respondents who already knew pots existed. Within that group, 29.7% cited rate, representing just 11.7% of all respondents and only 8.8% when weighted to the customer population mix. For the overwhelming majority of non-users, rate cannot be the barrier because they do not know the feature exists.

  3. In-app payday prompts drive higher intent than an expensive 75 bps rate rise, particularly among unaware and younger segments.

    High confidenceBase 700

    Payday nudges at 3.1% AER achieved 33.7% net likelihood overall (and 38.5% among 18-34s; 42.5% among unaware customers), compared to 29.9% overall (and 27.1% among 18-34s; 24.2% among unaware) for a rate hike to 3.85% AER.

  4. External savings balances are heavily skewed and cannot be captured simply via a rate increase.

    High confidenceBase 700

    While mean external savings among reporting respondents was £14,513, 35.7% of all respondents have £0 savings elsewhere, and 55.2% of those with savings hold less than £5,000 (median band £1k-£5k). Furthermore, respondents cite strong institutional inertia, trust, and habit with legacy providers (e.g., Leeds Building Society, Nationwide, Halifax).

  5. Targeting customers aged 75+ for a savings campaign is commercially unviable despite high per-capita balances.

    High confidenceBase 20

    Customers aged 75+ represent only 2.0% of the non-pot customer base (~7,200 people). In the survey, their base was tiny (n=20), and their net likelihood to start saving under either intervention was low (25.0%).

  6. Usability, discoverability, and withdrawal reassurance are critical friction points in the app onboarding flow.

    Medium confidenceBase 700

    Qualitative feedback reveals customers cannot locate pots ('buried under menus'), find the multi-step set-up intimidating or confusing, and express anxiety regarding whether withdrawals are truly instant in emergencies (7.6% cited withdrawal speed at Q3).

Caveats

  • The survey sample suffers from non-response skew: older customers (55-74) were overrepresented (28.6% of sample vs 13.0% in population), while younger customers (18-34) were underrepresented (31.1% vs 46.0%).
  • Survey completion rate was 6.03% (700/11,600), typical for email surveys but subject to self-selection bias towards more engaged or financially sensitive users.
  • Stated intent (Q5-Q7) consistently overstates real-world financial behavior; claimed intent to transfer savings does not directly translate to funded accounts.
  • The 75+ sample size is very small (n=20 overall, n=7 for Q3), meaning findings for this cohort carry high sampling variability and wide confidence intervals.
  • External balance data was self-reported and top-coded at £50,000+; the £14,513 mean reflects only respondents holding savings and uses arbitrary midpoint estimation.

Next steps

  • experimentDeploy an in-app payday savings pot nudge and home screen promotional card to a randomized 50% holdout experiment.
  • decisionRedesign the pot onboarding UX to simplify the setup steps and prominently reassure users that withdrawals are instant 24/7.
  • researchConduct financial modeling on deposit elasticity and net interest margin impact of a rate rise across existing vs incremental balances.
  • monitorMonitor monthly pot awareness and competitive savings market rates across active current account cohorts.

Open-text themes it coded

Awareness & Information Deficit 125Interest Rate Unattractiveness 45App UI & Discoverability Friction 31Setup Process Complexity 12Lack of Disposable Income / Affordability 23Competitor Trust & Account Inertia 15Access Anxiety & Withdrawal Speed Concerns 10General Disinterest in Saving 15

The survey it planned

2 screening questions and 10 questions, as the model wrote them.

  1. S1

    What is your age?

    One answer
    • Under 18
    • 18-34
    • 35-54
    • 55-74
    • 75 or over

    Continues if 18-34; 35-54; 55-74; 75 or over

  2. S2

    Have you ever opened a Savings Pot within your Pennywell app?

    One answer
    • Yes, I currently have one or have had one in the past
    • No, I have never opened one
    • I am not sure

    Continues if No, I have never opened one

  3. Q1

    Before taking this survey, did you know that Pennywell offers Savings Pots earning 3.1% AER inside the app?

    One answer
    • Yes, I was aware
    • No, I did not know this existed
    Judges: double-barrelled

    Combines awareness that pots exist with awareness of their current interest rate. Someone who knew about pots but not the rate has no clear answer, so the responses do not reliably measure prior awareness of pots.

    Asks in one answer whether the respondent knew pots exist and whether they knew the 3.1% AER rate. Someone who knew about pots but not the rate cannot answer accurately, which muddies the awareness measure for O1.

  4. Q2

    Why have you not opened a Pennywell Savings Pot? (Select all that apply)

    Any that apply
    • The interest rate (3.1% AER) is too low compared to other banks
    • I didn't know how to set one up or where to find it in the app
    • I meant to set one up, but forgot or haven't gotten around to it
    • I prefer to keep all my money in my main current account balance
    • I already have savings accounts elsewhere that I prefer using
    • I do not currently have spare money to put into savings
    • Other reason

    Routing Shown only if Q1 is: Yes, I was aware

  5. Q3

    How likely would you be to open a Savings Pot at the current rate (3.1% AER) if Pennywell prompted you with a quick one-tap setup on payday?

    Scale

    Scale 1-5: 1 - Extremely unlikely to 5 - Extremely likely

  6. Q4

    If Pennywell increased the Savings Pot interest rate from 3.1% AER to 3.85% AER, how likely would you be to open one?

    Scale

    Scale 1-5: 1 - Extremely unlikely to 5 - Extremely likely

  7. Q5

    If Pennywell could only introduce one of the following, which single improvement would most encourage you to open a Savings Pot?

    One answer
    • Increasing the interest rate to 3.85% AER
    • An automated feature that moves spare cash into savings on payday
    • Clearer prompts and guides inside the app showing where and how to create a pot
    • None of these would encourage me to open a pot
  8. Q6

    Do you currently hold any savings in an easy-access account with another bank or financial provider?

    One answer
    • Yes
    • No
    • Prefer not to say
  9. Q7

    Approximately how much total money do you currently hold across easy-access savings accounts at other institutions?

    One answer
    • Under £1,000
    • £1,000 to £4,999
    • £5,000 to £9,999
    • £10,000 to £24,999
    • £25,000 to £49,999
    • £50,000 or more
    • Prefer not to say

    Routing Shown only if Q6 is: Yes

  10. Q8

    What interest rate are you earning on the majority of your savings held elsewhere?

    One answer
    • Under 3.0% AER
    • 3.0% to 3.5% AER
    • 3.51% to 4.0% AER
    • More than 4.0% AER
    • I don't know / Not sure

    Routing Shown only if Q6 is: Yes

  11. Q9

    How much of your savings held elsewhere would you consider moving to Pennywell if pots paid 3.85% AER?

    One answer
    • All of it
    • Most of it (more than half)
    • Some of it (less than half)
    • None of it
    • Not sure

    Routing Shown only if Q6 is: Yes

  12. Q10

    What is the single biggest thing Pennywell could do to make you use its savings features?

    Open text
Sample plan

Target population: Active Pennywell customers aged 18+ who have never opened a savings pot according to database logs. Sampling method: Stratified random sampling via email invitation. Sample size and precision requirements: To achieve a margin of error of ±5 percentage points at 95% confidence for each of the four required age groups (18-34, 35-54, 55-74, 75+), standard sampling theory requires approximately 385 completed surveys per stratum, totaling ~1,540 completes overall. If overall volume is constrained to ~700 completes (n ≈ 175 per age bracket), margin of error per age group widens to approximately ±7.4 percentage points; the sample plan recommends oversampling to n=385 per subgroup (1,540 total) to strictly fulfill the board's ±5% precision requirement, or alternatively an equal quota of 175 completes per group if capped strictly at ~700.