Benchmarks / SurveyBench / Why Pennywell customers don't use savings pots / Claude Opus 5.5

Measured by Spring Prompt

Claude Opus 5.5: Why Pennywell customers don't use savings pots

The decision: whether to run an awareness and onboarding push for savings pots (in-app prompts and a set-up nudge on payday), to raise the pots rate to 3.85% AER, or both. The brief and the data →

Verdict
✓ Sound research
Research score
100 of 100
Analysis rating
1,646
Head to head, this task
won 10 of 11

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What percentage of respondents said they knew, before the survey, that they could open savings pots (Yes at Q1)?✓39.4%39.4%
Among respondents who were asked Q3, what percentage chose 'The interest rate is too low'?✓29.7%29.7%
As a percentage of all respondents, how many knew about pots and chose 'The interest rate is too low' at Q3?✓11.7%11.7%
Weighted to the age mix of all customers who have never opened a pot, what percentage of them would know about pots and choose 'The interest rate is too low' at Q3?✓8.8%8.8%
Weighted to the age mix of all 361,000 customers who have never opened a pot, about how many of them did not know they could open savings pots (No or Not sure at Q1)?✓240,000239,702
Among respondents who reported an amount held in savings elsewhere (excluding None, Don't know and Prefer not to say), what percentage hold less than £5,000?✓55.2%55.2%
What percentage of all respondents said they would be likely (fairly or very) to start saving in a pot if the rate rose to 3.85% AER?✓29.9%29.9%
Among respondents asked Q3, is the difference in the share choosing 'The interest rate is too low' between those aged 35-54 and those aged 55-74 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
What is the 95% margin of error, in percentage points, on the share of respondents asked Q3 who chose 'The interest rate is too low'?✓5.4 points5.39 points
How many of the open-text answers say the respondent did not know about savings pots or what they are, or want Pennywell to tell them about pots?✓126126
How many of the open-text answers from respondents aged 18-34 say they did not know about savings pots or what they are, or want Pennywell to tell them about pots?✓4646
What percentage of the customers invited completed the survey?✓6%6%

Traps in the data 8 of 8 handled

  • ✓Q3 was asked only of the 276 respondents who knew about pots. 'The interest rate is too low' leads there (30%), but that is 12% of all respondents (about 9% weighted), while 61% of all respondents did not know pots existed (No or Not sure).
  • ✓Customers aged 55 or over are 15% of non-users but 31% of respondents; they know about pots more and cite the rate more, so unweighted figures overstate awareness and the rate reason (weighted, awareness falls from 39% to 34%).
  • ✓The mean amount held elsewhere (£14,513) is pulled up by a few large balances; the median is in the £1,000 to less than £5,000 band and 55% of those reporting an amount hold less than £5,000. Many hold nothing elsewhere.
  • ✓30% say they would be likely to start saving in a pot at 3.85% (17% at the current 3.1%); these are stated intentions about a hypothetical rate.
  • ✓Respondents aged 75 or over (base 20) show 65% holding £20,000 or more elsewhere, against 13% overall.
  • ✓Only 700 of 11,600 invited customers completed (about 6%); those who respond to an email about savings may be more engaged with saving than those who do not.
  • ✓The chief financial officer believes customers don't use pots because the rate is uncompetitive.
  • ✓Among those asked Q3, the rate reason is 10 points higher for 55-74s (38%, base 120) than for 35-54s (27%, base 102), but the difference is not significant at 95%.

The analysis

Recommendation

Run the awareness and onboarding push first, at the current 3.1% rate. Fix discoverability and set-up alongside it: a home-screen entry point, fewer set-up steps, and plain-English messaging about instant access. Do not raise the rate to 3.85% on the strength of this survey. Only about 9% of non-users (weighted) cite the rate as their main reason, and those who do are largely people loyal to other providers who name thresholds above 3.85%. A rate rise also reprices existing pot balances at a cost not measured here. Launch the push as a randomised test with a holdout. Run a separate, controlled rate test on aware customers before committing to a book-wide rise. Do not build a savings campaign around the over-75s. They are 2% of non-users, measured on 20 respondents, and no more responsive to either lever. Do not size the opportunity from the £14,500 mean: the typical non-user holds under £5,000 elsewhere, and the age-weighted mean is roughly £7,000 including those with none.

What would change it

Behavioural results should decide this, not stated intent. Three results would shift the recommendation toward a rate rise or a combined approach. First, the awareness and nudge test lifts pot openings and funded balances only marginally versus holdout (for example, under 2 points more customers with funded pots after 90 days). Second, a controlled rate test on aware customers shows a large, durable inflow per customer that exceeds the cost of repricing existing pot balances. Third, competitor and churn data show pot users leaving for better rates. Conversely, if openings rise under the nudge but balances stay tiny, the push may need a rate component to build meaningful deposits.

Findings

  1. The main barrier is that most non-users do not know pots exist, not the rate. Weighted to the customer base, about two-thirds of non-users (around 240,000 people) did not know or were unsure that pots exist. Awareness is lowest among under-35s, who make up 46% of non-users.

    High confidenceBase 700

    Q1: 39.4% Yes, 50.7% No, 9.9% Not sure (base 700). No or Not sure by age: 18-34 78.5% (base 218), 35-54 61.1% (262), 55-74 40.0% (200), 75+ 65.0% (20). Weighted to the population age mix this is 66.4% of 361,000, about 240,000 customers. In the open text, 126 of 230 answers say the person did not know about pots or wants Pennywell to explain them (46 of these are from 18-34s).

  2. 'The rate is too low' is the top reason only among the minority who already knew about pots. Across all non-users, about 1 in 11 gives it as their main reason.

    High confidenceBase 276

    Q3 was asked only of the 276 who knew about pots: 29.7% chose rate (±5.4 points). That is 82 people, or 11.7% of all 700. Weighted to the population age mix it is 8.8%, about 32,000 customers. The other reasons the aware group gave add up to 70%: savings held elsewhere 19.6%, no money to save 13.4%, haven't got round to it 11.6%, unsure how to set up 10.9%, worried about access 7.6%.

  3. A payday set-up nudge at the current 3.1% rate draws at least as much stated interest as the rate rise. It draws much more among the unaware majority. The rate rise mainly appeals to people who already know about pots.

    Medium confidenceBase 700

    Net likely (4-5), base 700: baseline at 3.1% 17.4%, nudge 33.7%, rate rise to 3.85% 29.9%. Weighted to the population age mix: 17.4% / 35.1% / 28.9%. Unaware (No, base 355): nudge 42.5% vs rate 24.2%. Not sure (base 69): 47.8% vs 30.4%. Aware (base 276): nudge 18.8% vs rate 37.0%. By age, the nudge leads among 18-34s (38.5% vs 27.1%, base 218).

  4. Many customers who do know about pots cannot find them in the app or give up during set-up. A product fix to placement and set-up flow belongs in the onboarding push.

    Medium confidenceBase 230

    29 open-text answers describe pots as buried or hard to find, and 12 describe abandoning or being put off by fiddly set-up. Several come from 'Yes' respondents who tried and gave up (e.g. P039, P048, P141, P661). Q3: 10.9% of the aware group were unsure how to set one up (base 276).

  5. The '£14,500 average held elsewhere' overstates the money available. It excludes the 36% with no savings, is driven by a top band valued at an assumed £75,000, and comes from a sample that over-represents older customers. The typical non-user holds under £5,000 elsewhere.

    Medium confidenceBase 413

    Q4: 35.7% hold none; the median is in the £1,000-£4,999 band; 55.2% of the 413 who gave an amount hold under £5,000. Re-weighted to the population age mix and counting 'None' as £0, the band-midpoint mean is roughly £7,000 per non-user, about half the quoted figure. 55-74s are 28.6% of the sample but 13% of the population.

  6. Rate-sensitive customers mostly hold savings elsewhere out of habit and trust. A 0.75-point rise may not be enough to move them: several name thresholds of 4-5% or 'top of the best-buy tables'.

    Low confidenceBase 230

    45 open-text answers mention the rate. 17 mention loyalty to another provider, and 12 of those also mention the rate. Examples: P093 wants 5%, P178 at least 4%, P449 a full point above Nationwide, P499 best-buy level. Q3: 19.6% keep savings elsewhere (base 276).

  7. Customers aged 75 or over are too few, and measured on too small a base, to support a dedicated campaign. They are also no more responsive than others to either lever.

    Low confidenceBase 20

    Base 20 at Q1/Q4-Q7 and 7 at Q3. They are 2% of non-users, about 7,200 customers. Q4: 65% hold £20,000 or more (13 people). Net likely: nudge 25.0%, rate 25.0%, the joint-lowest of any age group.

  8. The rate-reason gap between 35-54s and 55-74s is not reliable evidence that older customers are more rate-driven.

    High confidenceBase 222

    Q3 rate choice: 27.5% (base 102) vs 37.5% (base 120). z ≈ 1.58, not significant at 95%.

  9. Roughly 1 in 10 of the open-text answers rule out pots because the person has no money to save, and a similar share are simply not interested. No lever will move these customers.

    Medium confidenceBase 230

    23 open-text answers cite no spare money and 15 say they are not interested in saving this way. Q3: 13.4% no money left over (base 276). Q4: 35.7% hold no savings elsewhere (base 700).

Caveats

  • Only 6.0% of those invited completed (700 of 11,600). Responders may be more engaged or more interested in savings than non-responders. Q2 itself told unaware respondents about pots, so the survey is partly an awareness intervention.
  • The sample is unweighted and skews old: 55-74s are 28.6% of the sample but 13% of non-users, and 18-34s are 31.1% vs 46%. Unweighted totals overstate rate sensitivity and balances held elsewhere and understate unawareness. Weighted figures are given where they matter.
  • Q3, the source of 'rate is the top reason', was asked only of the 276 who already knew about pots. It says nothing about the roughly two-thirds who did not know pots exist.
  • Q5-Q7 are stated intentions in a fixed order, with no rotation and no control. Stated likelihood typically overstates real take-up. The nudge and rate questions each followed the baseline, so order and contrast effects cannot be ruled out. Neither question asked how much people would deposit.
  • The survey cannot estimate deposits from a rate rise. Q4 measures total savings held elsewhere, not willingness to move it, and the mean depends on band midpoints, including an assumed £75,000 for the open top band.
  • The cost of a rate rise is not covered. Raising the pots rate to 3.85% also reprices balances of existing pot users, who were outside this survey. That cost needs finance modelling.
  • The 75+ group has bases of 20 (Q1/Q4-Q7) and 7 (Q3). Any figure for them is indicative only.
  • Some unaware respondents also asked for a better rate before they knew what pots pay. Their rate comments may reflect general expectations rather than a view of 3.1%.
  • There are minor data-quality flags. Some self-descriptions conflict with the age band (e.g. P089 '35-54, retired', P699 '35-54, student'), and respondents who preferred not to give an age were assigned the band on their account record.

Next steps

  • experimentRun a randomised test of in-app awareness prompts plus a payday set-up nudge at 3.1%, against a holdout of never-pot customers, stratified by age.
  • decisionRedesign pot discoverability and set-up: a home-screen entry, fewer steps, clear labels, and an instant-access message. Usability-test it with 8-10 customers before launch.
  • researchModel the full cost of a rise to 3.85%, including repricing of existing pot balances, against plausible inflows using actual pot-user balances and the re-weighted Q4 distribution.
  • experimentIf the business case warrants it, run a controlled rate test (e.g. a time-limited bonus rate) for a random subset of customers already aware of pots.
  • monitorTrack awareness of pots among non-users quarterly in a short pulse survey, weighted to the customer age mix, and track pot openings by age band in account data.
  • decisionDrop the over-75 campaign idea for now. If older-saver interest persists, test it within the main experiment's age strata rather than building a separate campaign.

Open-text themes it coded

Did not know pots exist or what they are / wants Pennywell to explain 126Interest rate too low / needs a better rate 45Pots hard to find in the app 29Set-up process confusing or too many steps 12Savings kept with another provider out of habit or trust 17No spare money to save 23Worry about speed of access to money 10Not interested in saving this way 15

The survey it planned

2 screening questions and 10 questions, as the model wrote them.

  1. S1

    Which of these age groups are you in?

    One answer
    • Under 18
    • 18-34
    • 35-54
    • 55-74
    • 75 or over
    • Prefer not to say

    Continues if 18-34; 35-54; 55-74; 75 or over

  2. S2

    Do you currently have a Pennywell current account?

    One answer
    • Yes
    • No
    • Not sure

    Continues if Yes

  3. Q1

    Pennywell offers savings pots. These are instant-access savings accounts that you can open inside the Pennywell app. Before today, did you know that Pennywell offers savings pots?

    One answer
    • Yes, and I have looked at them in the app
    • Yes, I had heard of them but have never looked at them
    • No, this is the first I have heard of them
    • Not sure
  4. Q2

    Why have you not opened a savings pot so far? Please select all that apply.

    Any that apply
    • I haven't got round to it or hadn't thought about it
    • I wasn't sure how to set one up
    • Setting one up looked like too much effort
    • I didn't understand what pots are or how they work
    • The interest rate isn't high enough
    • I already keep my savings somewhere else
    • I don't have money left over to save
    • I prefer to keep my savings away from my everyday bank
    • I'm not confident my savings would be safe with Pennywell
    • Another reason
    • No particular reason

    Routing Shown only if Q1 is: Yes, and I have looked at them in the app; Yes, I had heard of them but have never looked at them

  5. Q3

    And which one of these is the main reason you have not opened a savings pot?

    One answer
    • I haven't got round to it or hadn't thought about it
    • I wasn't sure how to set one up
    • Setting one up looked like too much effort
    • I didn't understand what pots are or how they work
    • The interest rate isn't high enough
    • I already keep my savings somewhere else
    • I don't have money left over to save
    • I prefer to keep my savings away from my everyday bank
    • I'm not confident my savings would be safe with Pennywell
    • Another reason
    • No particular reason

    Routing Shown only if Q1 is: Yes, and I have looked at them in the app; Yes, I had heard of them but have never looked at them

  6. Q4

    Savings pots currently pay 3.1% AER (variable). You can add or take out money at any time. How likely or unlikely are you to open a savings pot in the next 3 months?

    Scale

    Scale 1-5: Very unlikely to Very likely

  7. Q5

    Suppose the rate stayed at 3.1% AER, and on payday the app showed you a prompt that let you set up a pot and move money into it in a couple of taps. How likely or unlikely would you be to open a savings pot in the next 3 months?

    Scale

    Scale 1-5: Very unlikely to Very likely

  8. Q6

    Suppose instead that savings pots paid 3.85% AER (variable), with no other changes. How likely or unlikely would you be to open a savings pot in the next 3 months?

    Scale

    Scale 1-5: Very unlikely to Very likely

  9. Q7

    To check that the survey is displaying properly, please select 'Often' from the list below.

    One answer
    • Never
    • Rarely
    • Sometimes
    • Often
    • Always
  10. Q8

    Which one of these, if any, would do most to encourage you to open a savings pot?

    One answer
    • A higher interest rate on pots
    • Seeing pots suggested to me in the app
    • A reminder on payday to set some money aside
    • Being able to set up a pot in one or two taps
    • Clearer information on how pots work and how my money is protected
    • Something else
    • None of these, I'm not interested in opening a pot
  11. Q9

    Roughly how much do you have in total in savings accounts and cash ISAs with banks or providers other than Pennywell? Please don't include pensions, investments, property or money in current accounts.

    One answer
    • None
    • Under £1,000
    • £1,000 to £4,999
    • £5,000 to £9,999
    • £10,000 to £24,999
    • £25,000 to £49,999
    • £50,000 to £99,999
    • £100,000 or more
    • Prefer not to say
  12. Q10

    If you opened a Pennywell savings pot, how much of those savings would you consider moving into it?

    One answer
    • None of it
    • Less than a quarter
    • About a quarter to a half
    • About half to three-quarters
    • More than three-quarters
    • Not sure

    Routing Shown only if Q9 is: Under £1,000; £1,000 to £4,999; £5,000 to £9,999; £10,000 to £24,999; £25,000 to £49,999; £50,000 to £99,999; £100,000 or more

Sample plan

POPULATION: Pennywell customers aged 18 or over who have never opened a savings pot (about 361,000, which is 86% of 420,000).

SAMPLING: Draw a random sample from the account database, stratified by date-of-birth age band (18-34, 35-54, 55-74, 75 or over). Re-check pot status just before sending and exclude anyone who has opened a pot since the sample was drawn. Append database age to each record to cross-check S1.

INVITATIONS: Send email invitations in waves until each quota fills. Assume a response rate of roughly 5-10%, so invite about 10-20 times the quota in each band. The 75 or over band is likely to be small in an app-only bank and may respond less, so oversample it heavily. If it cannot reach quota, report it with its actual margin of error rather than merging it into another band.

PRECISION CONFLICT (the board must decide before fieldwork): ±5 points at 95% confidence (worst case, p = 50%) needs about 385 completes per age band, so about 1,540 in total. The planned 700 completes cannot deliver this. Spread as quotas of 175 per band, 700 gives about ±7.4 points per band, and about ±3.7 points for the total.

RECOMMENDATION: Either fund about 1,540 completes (385 per band, roughly 7 minutes is not needed because the survey stays at about 4 minutes), or accept ±7.4 points per band with 700 completes (175 per band).

WEIGHTING: Total-sample figures will be weighted back to the true age profile of never-openers, because equal quotas over-represent smaller bands.

EXCLUSIONS: Remove respondents who fail the Q7 attention check, finish in under about 60 seconds, or report an age band that conflicts with the database.

INCENTIVE AND TIMING: Offer a small incentive (for example, a prize draw). The survey takes about 3-4 minutes. In the invitation, call it a short survey about saving, not about pots or rates, to avoid biasing who responds.