Benchmarks / SurveyBench / Why Pennywell customers don't use savings pots / Muse Wonder

Measured by Spring Prompt

Muse Wonder: Why Pennywell customers don't use savings pots

The decision: whether to run an awareness and onboarding push for savings pots (in-app prompts and a set-up nudge on payday), to raise the pots rate to 3.85% AER, or both. The brief and the data →

Verdict
✗ Not sound
Research score
93 of 100
Analysis rating
1,460
Head to head, this task
won 10 of 15

Why it is not sound

The survey

  • A research objective no question serves
  • A double-barrelled question: Q10

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What percentage of respondents said they knew, before the survey, that they could open savings pots (Yes at Q1)?✓39.4%39.4%
Among respondents who were asked Q3, what percentage chose 'The interest rate is too low'?✓29.7%29.7%
As a percentage of all respondents, how many knew about pots and chose 'The interest rate is too low' at Q3?✓11.7%11.7%
Weighted to the age mix of all customers who have never opened a pot, what percentage of them would know about pots and choose 'The interest rate is too low' at Q3?✓8.8%8.8%
Weighted to the age mix of all 361,000 customers who have never opened a pot, about how many of them did not know they could open savings pots (No or Not sure at Q1)?✓239,679239,702
Among respondents who reported an amount held in savings elsewhere (excluding None, Don't know and Prefer not to say), what percentage hold less than £5,000?✓55.3%55.2%
What percentage of all respondents said they would be likely (fairly or very) to start saving in a pot if the rate rose to 3.85% AER?✓29.9%29.9%
Among respondents asked Q3, is the difference in the share choosing 'The interest rate is too low' between those aged 35-54 and those aged 55-74 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
What is the 95% margin of error, in percentage points, on the share of respondents asked Q3 who chose 'The interest rate is too low'?✓5.4 points5.39 points
How many of the open-text answers say the respondent did not know about savings pots or what they are, or want Pennywell to tell them about pots?✓126126
How many of the open-text answers from respondents aged 18-34 say they did not know about savings pots or what they are, or want Pennywell to tell them about pots?✓4746
What percentage of the customers invited completed the survey?✓6%6%

Traps in the data 8 of 8 handled

  • ✓Q3 was asked only of the 276 respondents who knew about pots. 'The interest rate is too low' leads there (30%), but that is 12% of all respondents (about 9% weighted), while 61% of all respondents did not know pots existed (No or Not sure).
  • ✓Customers aged 55 or over are 15% of non-users but 31% of respondents; they know about pots more and cite the rate more, so unweighted figures overstate awareness and the rate reason (weighted, awareness falls from 39% to 34%).
  • ✓The mean amount held elsewhere (£14,513) is pulled up by a few large balances; the median is in the £1,000 to less than £5,000 band and 55% of those reporting an amount hold less than £5,000. Many hold nothing elsewhere.
  • ✓30% say they would be likely to start saving in a pot at 3.85% (17% at the current 3.1%); these are stated intentions about a hypothetical rate.
  • ✓Respondents aged 75 or over (base 20) show 65% holding £20,000 or more elsewhere, against 13% overall.
  • ✓Only 700 of 11,600 invited customers completed (about 6%); those who respond to an email about savings may be more engaged with saving than those who do not.
  • ✓The chief financial officer believes customers don't use pots because the rate is uncompetitive.
  • ✓Among those asked Q3, the rate reason is 10 points higher for 55-74s (38%, base 120) than for 35-54s (27%, base 102), but the difference is not significant at 95%.

The analysis

Recommendation

Run the awareness and onboarding push now — do not raise the rate to 3.85% on this evidence alone, and do not build a 75+ campaign. Make pots visible and explained in-app (home/accounts surface, plain-English what/how/withdrawal-speed reassurance, short guided set-up), add the payday set-up nudge, and behaviourally test it. Defer a permanent rate rise pending a controlled test among already-aware customers, where the rate signal is strongest (Q7 likely 37.0% vs baseline 18.1%), and a proper inflow/cost model using actual balances likely to move rather than the £14,513 conditional mean. Sequencing matters: awareness/onboarding addresses about 240,000 unaware non-users and the nudge's biggest uplift; an untargeted rate rise would mainly subsidise people who cannot act because they do not know, cannot find, or cannot set up a pot.

What would change it

Evidence that would change this: a randomised behavioural trial showing the 3.85% rate (alone or added to onboarding) produces materially more funded pots and net new balances than onboarding+nudge at 3.1%, after repricing costs; validated account-level data showing large moveable balances among reachable, aware non-users; or onboarding testing showing awareness/friction is not the binding constraint in practice. Conversely, a trial showing the nudge lifts openings but balances remain trivial would argue against both pushes beyond cheap prompts.

Findings

  1. No, CFO: the rate does not settle it. 'Rate too low' is the top main reason only among the minority who already knew about pots; it is not the main barrier for most non-users.

    High confidenceBase 700

    Q3 base is only the 276 respondents (39.4% of 700) who said Yes at Q1. Among them 29.7% chose rate too low (95% margin of error ±5.4pp). That is 11.7% of all 700 respondents, and only 8.8% when weighted to the population age mix, because younger customers are both more numerous in the population and less likely to know about pots or cite rate.

  2. Lack of awareness is the largest addressable barrier, affecting roughly two-thirds of the non-user population.

    High confidenceBase 700

    Only 39.4% of respondents knew about pots before the survey; 50.7% said No and 9.9% Not sure. Weighted to the population age mix (46% 18-34, 39% 35-54, 13% 55-74, 2% 75+), 66.4% would be unaware, about 239,700 of 361,000 non-users. In open text, 126 of 230 answers say the respondent did not know about pots/what they are or want Pennywell to tell them, including 47 from 18-34s.

  3. The payday set-up nudge looks at least as promising as the rate rise in stated intent, and works best where the rate argument cannot yet bite: among people who did not know about pots.

    Medium confidenceBase 700

    Net likely to start saving: 17.4% at 3.1% (Q5), 33.7% with payday suggestion at same rate (Q6, +16.3pp), 29.9% at 3.85% (Q7, +12.5pp). Among Q1=No, Q6 likely is 42.5% vs 17.2% baseline, while Q7 is only 24.2%. Among Q1=Yes, the pattern reverses: Q7 37.0% vs Q6 18.8%, suggesting awareness/onboarding first, rate second for existing knowers.

  4. The CFO's £14,500 average cannot be multiplied across non-users to estimate inflows from a rate rise.

    High confidenceBase 413

    £14,513 is the mean only among the 413 respondents who reported an amount elsewhere, excluding 35.7% with None plus Don't know/Prefer not to say. It is built from band midpoints with £50,000+ counted as £75,000, so is sensitive to a small top band. The median reporter is in £1,000-<£5,000, and 55.3% of reporters hold <£5,000 elsewhere. Stated likely at 3.85% is only 29.9%, stated intent overstates behaviour, and many savers cite habit/loyalty to their existing provider, so neither full balances nor full transfer can be assumed.

  5. Do not build a campaign around customers aged 75 or over on this evidence.

    High confidenceBase 20

    Head of marketing's observation rests on 20 respondents aged 75+, of whom only 7 were asked Q3. In Q4, 65.0% of these 20 report £20,000+ elsewhere, but this age group is only 2% of the 361,000 non-user population (about 7,220 people), is the least likely in several intent cuts (Q6 and Q7 net likely both 25.0%, base 20), and 60.0% did not know about pots. The sample overall is age-skewed (55-74 are 28.6% of respondents vs 13% of population; 18-34 are 31.1% vs 46%), inflating unweighted balance and awareness figures.

  6. Age difference in citing rate is not proven: 55-74s naming rate more than 35-54s is within sampling noise.

    Medium confidenceBase 222

    Among Q3 respondents, rate too low: 37.5% of 120 aged 55-74 vs 27.5% of 102 aged 35-54. Difference 10.0pp, z≈1.6, not significant at 95%. Do not segment the rate decision by this contrast.

  7. After awareness, friction and trust are secondary barriers: finding/setting up pots, withdrawal speed reassurance, having no spare money, and habit with another provider.

    Medium confidenceBase 276

    Open text repeatedly cites buried menus/fiddly multi-step set-up, wanting instant withdrawal reassurance, no money left over, and savings habitually held at building societies/old banks. In Q3 (aware only), after rate 29.7%: keep savings elsewhere 19.6%, no money left over 13.4%, haven't got round to it 11.6%, wasn't sure how to set up 10.9%, worried about access speed 7.6%. A rate rise alone does not fix set-up, access confidence or absence of spare cash.

Caveats

  • Very low response: 700 completes from 11,600 invited (6.0%), 812 starters; non-respondents may differ systematically, and no weighting was applied in the tables supplied.
  • Sample is age-skewed versus the 361,000 non-user population: 55-74 over-represented (28.6% vs 13%) and 18-34 under-represented (31.1% vs 46%), biasing unweighted awareness, reasons and balances upward; 75+ rests on only 20 respondents (7 for Q3).
  • Q3 was asked only of the 276 who already knew about pots, so its percentages must never be read as shares of all non-users; 'main reason' is forced single choice and hides secondary reasons.
  • Q5-Q7 are sequential hypothetical stated intentions from the same respondents, not observed behaviour; order/anchoring effects are possible, no combined awareness+nudge+rate condition was tested, and intentions typically overstate actual opening and funding.
  • Balances are self-reported in bands; the £14,513 mean uses midpoints and £75,000 for £50,000+, excludes None/Don't know/Prefer not to say, and is skewed by a small high-balance tail — it is not a per-customer addressable balance.
  • Open text was answered by 230 of 700, self-selecting and likely more motivated or more frustrated; theme counts are indicative of salience among commenters, not population prevalence.
  • The survey cannot estimate revenue/cost: it contains no balances Pennywell would have to reprice for existing pot holders, no amount each switcher would actually move, no timing, and no competitor-rate context beyond respondent perceptions.

Next steps

  • decisionLaunch in-app awareness/onboarding: prominent pots entry point, 60-second explainer, guided set-up, explicit 'instant access, back in seconds' reassurance, payday set-up suggestion
  • experimentRandomised A/B test: control vs onboarding+nudge at 3.1%, with separate aware-segment arm testing 3.85% rate messaging/offer where operationally feasible
  • researchBuild CFO inflow model from account and market data: distribution of external balances by segment (not conditional mean), likely share moved, repricing cost on existing pots book, sensitivity to top-band assumption
  • researchDo not target 75+ as a campaign segment; instead include older customers in mainstream onboarding and run qualitative usability work with a boosted, properly recruited older sample if needed
  • experimentFix and instrument set-up funnel: reduce steps, clearer labelling/location, save-and-resume, withdrawal-speed proof point at point of deposit
  • monitorRe-field a short weighted tracker after launch with behaviour linkage consent, reporting weighted and unweighted results and bases by age

Open-text themes it coded

Did not know about pots / want Pennywell to explain them 126Wants a higher / more competitive rate 45Hard to find in app / set-up is fiddly or unclear 39Needs reassurance withdrawals are instant / no delays 10No spare money to save 23Savings habitually kept elsewhere / loyalty or trust in existing provider 15Not interested in saving this way / nothing would persuade 15

The survey it planned

3 screening questions and 10 questions, as the model wrote them.

  1. S1

    What is your age?

    One answer
    • Under 18
    • 18-34
    • 35-54
    • 55-74
    • 75 or over

    Continues if 18-34; 35-54; 55-74; 75 or over

  2. S2

    Do you currently have a Pennywell current account?

    One answer
    • Yes
    • No

    Continues if Yes

  3. S3

    Have you ever opened a Pennywell Savings Pot, including one that you later closed?

    One answer
    • Yes
    • No
    • Not sure

    Continues if No

  4. Q1

    Before receiving this survey, did you know that Pennywell offers Savings Pots, which are instant-access savings accounts inside the Pennywell app?

    One answer
    • Yes
    • No
    • Not sure
  5. Q2

    You said you knew about Pennywell Savings Pots but have never opened one. Why have you not opened one? Please select all that apply.

    Any that apply
    • I have no spare money to put into savings
    • I already save elsewhere and am happy with that arrangement
    • The interest rate is not attractive enough
    • I did not know how to open one
    • I do not understand how Savings Pots work
    • I have not got around to it
    • I forgot about them
    • I prefer to keep all my money in my current account
    • I am concerned about being able to access my money when I need it
    • I have concerns about safety or security
    • Another reason
    • Don't know

    Routing Shown only if Q1 is: Yes

  6. Q3

    Not counting any Pennywell Savings Pots, approximately how much do you currently hold in savings elsewhere in total? Please include cash ISAs, instant-access and other savings accounts, and any other savings, but exclude pensions, investments and property.

    One answer
    • £0 - I have no savings elsewhere
    • £1 to £500
    • £501 to £2,500
    • £2,501 to £10,000
    • £10,001 to £25,000
    • More than £25,000
    • Prefer not to say
  7. Q4

    If you opened a Pennywell Savings Pot, roughly how much of the savings you hold elsewhere would you consider moving into it?

    One answer
    • None of it
    • Less than a quarter
    • About a quarter to a half
    • More than half but not all of it
    • All of it
    • Don't know

    Routing Shown only if Q3 is: £1 to £500; £501 to £2,500; £2,501 to £10,000; £10,001 to £25,000; More than £25,000

  8. Q5

    Savings Pots are separate instant-access savings accounts within the Pennywell app. You can open one in the app, add or withdraw money at any time, and they currently pay 3.1% AER interest. How likely are you to open a Savings Pot in the next 3 months?

    Scale

    Scale 1-5: Very unlikely to Very likely

  9. Q6

    Please rank the following changes or forms of help from 1 to 5, where 1 would make you most likely to open a Savings Pot and 5 would make you least likely.

    Ranking
    • The interest rate increasing to 3.85% AER
    • A clear explanation of what Savings Pots are and how they work
    • A reminder on payday
    • Step-by-step help to set up a pot in the app
    • The option to move money into a pot automatically on payday
  10. Q7

    If the Savings Pots interest rate were 3.85% AER instead of 3.1% AER, and everything else stayed the same, how likely would you be to open a Savings Pot in the next 3 months?

    Scale

    Scale 1-5: Very unlikely to Very likely

  11. Q8

    If Pennywell showed you a prompt on payday that let you open and put money into a Savings Pot in a few taps, while the interest rate stayed at 3.1% AER, how likely would you be to open a Savings Pot in the next 3 months?

    Scale

    Scale 1-5: Very unlikely to Very likely

  12. Q9

    To check that you are reading the questions carefully, please select 'Disagree' below.

    One answer
    • Strongly disagree
    • Disagree
    • Neither agree nor disagree
    • Agree
    • Strongly agree
  13. Q10

    In your own words, what is the single main reason you have not opened a Pennywell Savings Pot, or what would most persuade you to open one?

    Open text
    Judges: double-barrelled

    It asks for both the main reason for not opening a pot and what would most persuade the respondent to open one, in a single answer. Responses will mix barriers and motivators and cannot be cleanly separated.

    It asks about two distinct things (the reason for not opening a pot and what would persuade them to open one) but only allows one open-ended answer.

Sample plan

Draw a stratified random sample by age band from the account database of Pennywell current-account customers aged 18 or over who have never opened a Savings Pot. The approximately 700 completes originally envisaged is insufficient for the required precision: at 95% confidence and the worst-case proportion of 50%, +/-5 percentage points requires 385 completes per age band. Set quotas of 400 completes in each of the 18-34, 35-54, 55-74 and 75 or over bands, giving 1,600 completes in total (approximately +/-4.9 points per band). Invite customers by email in random waves within each age stratum, oversampling older strata as needed for lower response, until each quota is filled; do not fill quotas from other age bands. Weight overall results back to the age distribution of all never-openers, but report age-band results unweighted. Screen out anyone failing S1-S3 and exclude Q9 attention-check failures from the main analysis, replacing them within the same age quota.