Benchmarks / SurveyBench / Why Pennywell customers don't use savings pots / Gemini 4 Hydrogen
Gemini 4 Hydrogen: Why Pennywell customers don't use savings pots
The decision: whether to run an awareness and onboarding push for savings pots (in-app prompts and a set-up nudge on payday), to raise the pots rate to 3.85% AER, or both. The brief and the data →
- Verdict
- ✗ Not sound
- Research score
- 74 of 100
- Analysis rating
- 1,088
- Head to head, this task
- won 2 of 9
Why it is not sound
The analysis
- Made a recommendation that does not follow from the data
- Missed a trap: Only 700 of 11,600 invited customers completed (about 6%); those who respond to an email about savings may be more engaged with saving than those who do not.
Scorecard
Numbers 12 of 12 right
| Question | Its answer | True | |
|---|---|---|---|
| What percentage of respondents said they knew, before the survey, that they could open savings pots (Yes at Q1)? | ✓ | 39.4% | 39.4% |
| Among respondents who were asked Q3, what percentage chose 'The interest rate is too low'? | ✓ | 29.7% | 29.7% |
| As a percentage of all respondents, how many knew about pots and chose 'The interest rate is too low' at Q3? | ✓ | 11.7% | 11.7% |
| Weighted to the age mix of all customers who have never opened a pot, what percentage of them would know about pots and choose 'The interest rate is too low' at Q3? | ✓ | 8.8% | 8.8% |
| Weighted to the age mix of all 361,000 customers who have never opened a pot, about how many of them did not know they could open savings pots (No or Not sure at Q1)? | ✓ | 239,702 | 239,702 |
| Among respondents who reported an amount held in savings elsewhere (excluding None, Don't know and Prefer not to say), what percentage hold less than £5,000? | ✓ | 55.2% | 55.2% |
| What percentage of all respondents said they would be likely (fairly or very) to start saving in a pot if the rate rose to 3.85% AER? | ✓ | 29.9% | 29.9% |
| Among respondents asked Q3, is the difference in the share choosing 'The interest rate is too low' between those aged 35-54 and those aged 55-74 statistically significant at the 95% level? (1 for yes, 0 for no) | ✓ | No | No |
| What is the 95% margin of error, in percentage points, on the share of respondents asked Q3 who chose 'The interest rate is too low'? | ✓ | 5.4 points | 5.39 points |
| How many of the open-text answers say the respondent did not know about savings pots or what they are, or want Pennywell to tell them about pots? | ✓ | 126 | 126 |
| How many of the open-text answers from respondents aged 18-34 say they did not know about savings pots or what they are, or want Pennywell to tell them about pots? | ✓ | 46 | 46 |
| What percentage of the customers invited completed the survey? | ✓ | 6% | 6% |
Traps in the data 7 of 8 handled
- ✓Q3 was asked only of the 276 respondents who knew about pots. 'The interest rate is too low' leads there (30%), but that is 12% of all respondents (about 9% weighted), while 61% of all respondents did not know pots existed (No or Not sure).
- ✓Customers aged 55 or over are 15% of non-users but 31% of respondents; they know about pots more and cite the rate more, so unweighted figures overstate awareness and the rate reason (weighted, awareness falls from 39% to 34%).
- ✓The mean amount held elsewhere (£14,513) is pulled up by a few large balances; the median is in the £1,000 to less than £5,000 band and 55% of those reporting an amount hold less than £5,000. Many hold nothing elsewhere.
- ✓30% say they would be likely to start saving in a pot at 3.85% (17% at the current 3.1%); these are stated intentions about a hypothetical rate.
- ✓Respondents aged 75 or over (base 20) show 65% holding £20,000 or more elsewhere, against 13% overall.
- ✗Only 700 of 11,600 invited customers completed (about 6%); those who respond to an email about savings may be more engaged with saving than those who do not.
- ✓The chief financial officer believes customers don't use pots because the rate is uncompetitive.
- ✓Among those asked Q3, the rate reason is 10 points higher for 55-74s (38%, base 120) than for 35-54s (27%, base 102), but the difference is not significant at 95%.
The analysis
Findings
-
Low awareness—not the 3.1% AER rate—is the primary barrier across non-pot customers; the CFO's impression that rate settles the decision rests on a Q3 routing artefact.
Only 39.4% unweighted (33.6% age-weighted) knew pots existed (Q1), leaving 66.4% age-weighted (~239,702 of 361,000 non-users) unaware. Q3 was asked only of the 276 aware respondents; while 29.7% (±5.4 pp) of that subgroup cited rate, that represents just 11.7% of the total sample (82/700) and only 8.8% once weighted to the population age mix. Age differences in citing rate at Q3 between 35-54 (27.5%, n=102) and 55-74 (37.5%, n=120) are not statistically significant (z=1.59, p=0.11).
-
A payday set-up nudge at 3.1% AER outperforms a 3.85% AER rate rise overall and more than doubles stated intent among unaware and younger customers.
At 3.1% baseline (Q5), 17.4% are likely to start saving. A payday nudge at 3.1% (Q6) raises stated likelihood to 33.7% (+16.3 pp overall), driven by Q1 'No' (42.5% vs 17.2% at Q5), Q1 'Not sure' (47.8% vs 15.9%), and 18-34s (38.5% vs 17.9%). Raising the rate to 3.85% (Q7) reaches 29.9% overall (+12.5 pp) and helps mainly already-aware users (37.0% vs 18.8% on Q6) and 55-74s (33.5% vs 30.0%), while moving Q1 'No' only to 24.2%.
-
Even when customers know about or look for pots, poor app findability and a multi-step setup flow cause abandonment, and some doubt withdrawal speed.
In Q2 open-text (n=230), 126 respondents (54.8%, including 46 aged 18-34) asked to be told what pots are and how they work, 40 (17.4%, across both aware and unaware segments) said pots are buried in menus or the setup process is confusing/fiddly, and 10 (4.3%, alongside 7.6% of Q3 respondents) worried about instant access at weekends or emergencies.
-
We cannot forecast deposit inflows from the £14,513 average external balance; balances are skewed, overstated by sample age mix, and cannot be converted from stated intent alone.
The £14,513 mean excludes the 35.7% holding 'None' (n=250) and 5.3% DK/PNS (n=37); including 'None' lowers the sample mean to ~£9,041, and among the 413 savers elsewhere the median lies in the £1,000-£4,999 band with 55.2% holding under £5,000. Unweighted data also over-represents 55-74s (28.6% vs 13% population) and under-represents 18-34s (31.1% vs 46%, where 45.9% hold None). Q4 includes Cash ISAs (tax-wrapped) while pots are taxable, and 19.6% of aware non-users (plus 17 Q2 comments) cite loyalty/habit with legacy banks.
-
Do not build a savings campaign around customers aged 75 or over: the segment is tiny, under-sampled, and the least responsive to either the nudge or the 3.85% rate.
Although 65.0% of the 75+ sample report £20,000+ elsewhere, the base is only n=20 (and n=7 at Q3), giving margins of error around ±20 pp. In the 361,000 non-pot population, 75+ accounts for just 2% (~7,220 customers) compared with 85% aged 18-54 (~306,850) and 13% aged 55-74 (~46,930). Moreover, 75+ shows the lowest Q6/Q7 intent (25.0% likely for both vs 20.0% baseline), and 42.9% of Q3 75+ respondents keep savings elsewhere.
-
Around a sixth of open-text respondents face hard affordability constraints or disinterest that neither messaging nor a 3.85% rate will overcome.
Among all respondents, 35.7% hold no savings elsewhere (45.9% of 18-34s) and 13.4% of Q3 respondents say they have no money left over to save; in Q2, 23 respondents cite zero spare money (bills, rent, debt, low income) and 15 say they are simply not interested in saving in pots.
Open-text themes it coded
Unaware of savings pots and requesting an explanation 126Interest rate perceived as too low or uncompetitive 19
The survey it planned
2 screening questions and 10 questions, as the model wrote them.
-
S1
Which age group are you in?
One answer- Under 18
- 18-34
- 35-54
- 55-74
- 75 or over
Continues if 18-34; 35-54; 55-74; 75 or over
-
S2
Do you currently have an active Pennywell current account?
One answer- Yes, it is my main bank account
- Yes, I use it regularly alongside another bank account
- Yes, I have one, though I rarely use it
- No, I no longer have a Pennywell account
- Not sure
Continues if Yes, it is my main bank account; Yes, I use it regularly alongside another bank account; Yes, I have one, though I rarely use it
-
Q1
Before taking this survey, did you know that Pennywell offers instant-access savings pots inside the app?
One answer- Yes, I knew about them and have looked at them in the app
- Yes, I knew they existed, though I have not looked at them
- No, I did not know Pennywell offered savings pots
- Not sure
- I already have (or have had) a Pennywell savings pot
-
Q2
Why have you not opened a Pennywell savings pot? (Select all that apply)
Any that apply- I have not got around to it or have not thought about it
- I already keep my savings with another bank or building society
- The interest rate (3.1% AER) is not high enough compared with other accounts
- I did not know what interest rate it pays or how it works
- I do not have spare money to put into savings at the moment
- I prefer keeping my savings separate from my everyday spending app
- Setting one up looked too complicated or time-consuming
- I prefer fixed-term savings or investments rather than instant access
- Other reason
Routing Shown only if Q1 is: Yes, I knew about them and have looked at them in the app; Yes, I knew they existed, though I have not looked at them
-
Q3
If you selected more than one reason above, which ONE is the main reason you have not opened a Pennywell savings pot? (If you only selected one, please confirm it below.)
One answer- I have not got around to it or have not thought about it
- I already keep my savings with another bank or building society
- The interest rate (3.1% AER) is not high enough compared with other accounts
- I did not know what interest rate it pays or how it works
- I do not have spare money to put into savings at the moment
- I prefer keeping my savings separate from my everyday spending app
- Setting one up looked too complicated or time-consuming
- I prefer fixed-term savings or investments rather than instant access
- Other reason
Routing Shown only if Q1 is: Yes, I knew about them and have looked at them in the app; Yes, I knew they existed, though I have not looked at them
-
Q4
Pennywell savings pots are instant-access savings accounts inside the app that currently pay 3.1% AER. Thinking about this, how likely would you be to open a Pennywell savings pot in the next 3 months?
ScaleScale 1-5: Not at all likely to Extremely likely
-
Q5
Still at the current rate of 3.1% AER, how likely would you be to open a Pennywell savings pot if you saw a prompt in the app on payday that let you set one up in a couple of taps?
ScaleScale 1-5: Not at all likely to Extremely likely
-
Q6
If Pennywell increased the savings pot interest rate from 3.1% AER to 3.85% AER (with the current in-app set-up, without a payday prompt), how likely would you be to open one in the next 3 months?
ScaleScale 1-5: Not at all likely to Extremely likely
-
Q7
To help us check data quality, please select 'Mostly agree' from the options below.
One answer- Strongly disagree
- Mostly disagree
- Neither agree nor disagree
- Mostly agree
- Strongly agree
-
Q8
If you were considering opening a Pennywell savings pot, which ONE of the following would make the biggest difference to you?
One answer- Increasing the interest rate from 3.1% AER to 3.85% AER
- An in-app prompt on payday to set up a pot in a couple of taps
- Clearer information in the app about how savings pots work
- None of these would make me open a Pennywell savings pot
-
Q9
Roughly how much do you currently hold in savings accounts outside of Pennywell? (Please include instant-access, notice, or Cash ISA savings, and exclude pensions, property, and stocks or shares investments.)
One answer- £0 (I do not currently have savings outside Pennywell)
- £1 to £999
- £1,000 to £4,999
- £5,000 to £14,999
- £15,000 to £49,999
- £50,000 or more
- Prefer not to say
-
Q10
If Pennywell offered a savings pot that suited your needs, roughly how much of those outside savings would you realistically consider moving into it?
One answer- £0 (I would keep existing savings where they are and only add new money)
- £1 to £999
- £1,000 to £4,999
- £5,000 to £14,999
- £15,000 or more
- Not sure
- Prefer not to say
Routing Shown only if Q9 is: £1 to £999; £1,000 to £4,999; £5,000 to £14,999; £15,000 to £49,999; £50,000 or more
Judges: assumptiveThe £0 option assumes respondents would add new money. Someone who would neither move existing savings nor add new money has no accurately worded £0 response.
Assumes the respondent does not already have a savings pot.
Sample plan
Target population: Pennywell adult customers (aged 18+) who have never opened a savings pot (~361,200 customers, 86% of the 420,000 customer base). Sampling method: Stratified random sample drawn from the Pennywell account database across the four required age bands (18-34, 35-54, 55-74, 75 or over), invited by email with one reminder after 3-5 days. Precision and sample size trade-off: Achieving +/- 5 percentage points at 95% confidence within each of the four age bands requires n=385 completes per band (about 1,540 completes total, subject to finite population correction if the 75 or over customer base in the database is small). Under the current 700-complete constraint, set equal quotas of 175 completes per age band (18-34, 35-54, 55-74, 75 or over), yielding +/- 7.4 percentage points margin of error per age band at 95% confidence and +/- 3.7 percentage points overall; recommend expanding to ~1,540 completes if +/- 5 pp per band is mandatory for the board. Assuming a 5-8% email completion rate (and lower base size or response rate in 75+), send invitations in waves starting with ~10,000-14,000 emails (or ~25,000 if targeting 1,540 completes) and dynamically top up slower-filling age quotas. Apply post-stratification weights by the true database age distribution when reporting overall totals (O1-O3).