Benchmarks / SurveyBench / Why Pennywell customers don't use savings pots / GPT-6.1 Terra

Measured by Spring Prompt

GPT-6.1 Terra: Why Pennywell customers don't use savings pots

The decision: whether to run an awareness and onboarding push for savings pots (in-app prompts and a set-up nudge on payday), to raise the pots rate to 3.85% AER, or both. The brief and the data →

Verdict
✓ Sound research
Research score
96 of 100
Analysis rating
1,273
Head to head, this task
won 7 of 11

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What percentage of respondents said they knew, before the survey, that they could open savings pots (Yes at Q1)?✓39.4%39.4%
Among respondents who were asked Q3, what percentage chose 'The interest rate is too low'?✓29.7%29.7%
As a percentage of all respondents, how many knew about pots and chose 'The interest rate is too low' at Q3?✓11.7%11.7%
Weighted to the age mix of all customers who have never opened a pot, what percentage of them would know about pots and choose 'The interest rate is too low' at Q3?✓8.8%8.8%
Weighted to the age mix of all 361,000 customers who have never opened a pot, about how many of them did not know they could open savings pots (No or Not sure at Q1)?✓239,700239,702
Among respondents who reported an amount held in savings elsewhere (excluding None, Don't know and Prefer not to say), what percentage hold less than £5,000?✓55.2%55.2%
What percentage of all respondents said they would be likely (fairly or very) to start saving in a pot if the rate rose to 3.85% AER?✓29.9%29.9%
Among respondents asked Q3, is the difference in the share choosing 'The interest rate is too low' between those aged 35-54 and those aged 55-74 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
What is the 95% margin of error, in percentage points, on the share of respondents asked Q3 who chose 'The interest rate is too low'?✓5.4 points5.39 points
How many of the open-text answers say the respondent did not know about savings pots or what they are, or want Pennywell to tell them about pots?✓125126
How many of the open-text answers from respondents aged 18-34 say they did not know about savings pots or what they are, or want Pennywell to tell them about pots?✓4446
What percentage of the customers invited completed the survey?✓6%6%

Traps in the data 8 of 8 handled

  • ✓Q3 was asked only of the 276 respondents who knew about pots. 'The interest rate is too low' leads there (30%), but that is 12% of all respondents (about 9% weighted), while 61% of all respondents did not know pots existed (No or Not sure).
  • ✓Customers aged 55 or over are 15% of non-users but 31% of respondents; they know about pots more and cite the rate more, so unweighted figures overstate awareness and the rate reason (weighted, awareness falls from 39% to 34%).
  • ✓The mean amount held elsewhere (£14,513) is pulled up by a few large balances; the median is in the £1,000 to less than £5,000 band and 55% of those reporting an amount hold less than £5,000. Many hold nothing elsewhere.
  • ✓30% say they would be likely to start saving in a pot at 3.85% (17% at the current 3.1%); these are stated intentions about a hypothetical rate.
  • ✓Respondents aged 75 or over (base 20) show 65% holding £20,000 or more elsewhere, against 13% overall.
  • ✓Only 700 of 11,600 invited customers completed (about 6%); those who respond to an email about savings may be more engaged with saving than those who do not.
  • ✓The chief financial officer believes customers don't use pots because the rate is uncompetitive.
  • ✓Among those asked Q3, the rate reason is 10 points higher for 55-74s (38%, base 120) than for 35-54s (27%, base 102), but the difference is not significant at 95%.

The analysis

Recommendation

Proceed with a controlled awareness and onboarding push at 3.1%, including an optional pay-in nudge, rather than immediately raising the rate for everyone. Test 3.85% separately and in combination with the push if operationally and legally feasible. Decide on a wider rate rise only after measuring incremental net balances and the full economics. Do not make customers aged 75 or over the centre of the campaign on the present evidence.

What would change it

A controlled test showing that 3.85%, alone or with onboarding, delivers durable incremental net balances and customer value that exceed the extra interest and implementation costs would support a rate rise. Conversely, weak adoption, low incremental balances or adverse customer outcomes from the push would argue against scaling it. Robust evidence of a particularly responsive older segment could justify a tailored campaign.

Findings

  1. The rate finding does not settle the decision to raise it. Awareness and onboarding deserve a test at the current rate, alongside a controlled test of the rate increase.

    Medium confidenceBase 700

    Rate was the leading Q3 answer among the 276 respondents who already knew about pots: 29.7%, or 82 people. That is 11.7% of all 700 respondents, and approximately 8.8% after weighting by customer age. Q3 was not asked of those unaware or unsure, so this is not the proportion of all customers for whom rate is a barrier.

  2. A substantial awareness gap makes an explanation and easy route into setup a promising first intervention, but the population estimate is indicative rather than a forecast of reachable customers.

    Medium confidenceBase 700

    At Q1, 50.7% said No and 9.9% were unsure, out of 700. Applying age-specific results to the 361,000 non-pot customers gives approximately 239,700 unaware or unsure. The age-weighted estimate does not correct for other differences between respondents and non-respondents.

  3. Both interventions warrant testing; the survey cannot establish which will produce more actual adoption or whether combining them adds value.

    Medium confidenceBase 700

    Among 700 respondents, 17.4% said likely at 3.1%, 33.7% with a pay-in nudge at 3.1%, and 29.9% at 3.85%. These are stated intentions in sequential questions, not observed adoption or independent treatment effects. Among 355 previously unaware respondents, the corresponding figures were 17.2%, 42.5% and 24.2%.

  4. The CFO cannot infer how much a rate rise would bring in from the £14,513 figure.

    High confidenceBase 700

    The mean applies to the 413 respondents reporting some savings elsewhere, not all 700: 35.7% reported none and 5.3% did not know or preferred not to say. The mean uses band midpoints and assumes £75,000 for the open-ended top band. Among those 413, 55.2% held less than £5,000. Neither the amount that would move nor its source is measured.

  5. Do not build the main campaign around customers aged 75 or over on this evidence alone.

    Low confidenceBase 20

    Thirteen of 20 respondents aged 75 or over reported £20,000 or more elsewhere, but this small subgroup is volatile and represents 2% of the non-pot customer population. Only seven were asked Q3. Large external balances do not establish willingness to move them.

  6. The apparent age difference in rate objections among those already aware is not conclusive.

    Medium confidenceBase 222

    Among those asked Q3, 28 of 102 aged 35–54 chose rate, compared with 45 of 120 aged 55–74: 27.5% versus 37.5%. A two-sided comparison is not significant at the 95% level; it does not prove the groups are identical.

  7. Make the explanation and setup path easy to find, and explain access to funds accurately; some customers will still lack spare money or prefer another provider.

    Medium confidenceBase 230

    Of 230 open-text answers, 125 mention lack of awareness or wanting Pennywell to explain pots. Respondents also describe difficulty finding or completing setup, withdrawal concerns, limited spare money and loyalty to existing providers. These are self-selected comments, not population prevalence estimates.

Caveats

  • The survey was randomly invited but only 700 of 11,600 invitees completed it. Non-response may bias results; the tables are unweighted, and age weighting alone cannot remove all such bias.
  • The 95% margin of error of approximately 5.4 percentage points for Q3 rate is a conventional sampling approximation for 82 of 276, not an allowance for non-response or survey-design bias.
  • Q3 was restricted to those already aware and forced one main reason. It cannot rank barriers across all non-pot customers or reveal multiple simultaneous barriers.
  • Q5–Q7 were asked sequentially of the same respondents. Stated likelihood is not behaviour; order, survey information and hypothetical framing may influence answers. The combined rate-and-nudge proposition was not tested.
  • The Q4 mean is conditional on reporting some external savings, depends on an arbitrary value for the open-ended top band and is not the amount available or willing to move. It should not be multiplied by all 361,000 customers.
  • Open-text coding is interpretive, answers may express multiple themes, and the 230 people who answered are not necessarily representative of all respondents.
  • Age subgroup figures, especially the 20 respondents aged 75 or over and seven of them asked Q3, are too imprecise for a dedicated campaign decision.

Next steps

  • decisionApprove a limited, clearly explained awareness and onboarding test at the existing rate. Make pots easy to find, show the setup path and explain instant access accurately; make the pay-in suggestion optional.
  • experimentWhere feasible, randomise eligible non-pot customers into a 2×2 test: current versus 3.85% rate, crossed with push versus no push. Address rate eligibility, fairness and operational constraints before launch.
  • researchBuild the financial case using observed balances and a range of scenarios, including the cost of paying a higher rate on existing pot balances and on money that would have arrived anyway.
  • researchRun usability research across ages and awareness levels, including customers aged 75 or over, on finding, opening and withdrawing from a pot.
  • monitorMonitor uptake, balances, transfers, opt-outs and complaints over time, including by age and prior awareness where available.

Open-text themes it coded

Unaware of pots or wants Pennywell to explain them 125Wants a higher or more competitive interest rate 45Difficulty finding pots or completing setup 39Wants reassurance about speed or availability of withdrawals 10

The survey it planned

2 screening questions and 9 questions, as the model wrote them.

  1. S1

    Do you currently have a Pennywell current account?

    One answer
    • Yes
    • No
    • Not sure

    Continues if Yes

  2. S2

    Have you ever opened a Pennywell savings pot?

    One answer
    • Yes
    • No
    • Not sure

    Continues if No

  3. Q1

    Before today, were you aware that Pennywell offers savings pots?

    One answer
    • Yes
    • No
    • Not sure
  4. Q2

    Why have you not opened a Pennywell savings pot? Select all that apply.

    Any that apply
    • I do not have money available to save
    • I prefer to keep my savings elsewhere
    • The interest rate is too low
    • I did not know the interest rate
    • I did not know enough about how pots work
    • I thought setting one up would take too much time or effort
    • I intended to open one but have not got round to it
    • I have concerns about keeping my savings with Pennywell
    • I do not need another savings account
    • Another reason
    • Not sure
    • Prefer not to say

    Routing Shown only if Q1 is: Yes

  5. Q3

    Approximately how much do you currently hold in savings outside Pennywell? Include savings accounts and cash ISAs, including fixed-term accounts. Do not include investments or money kept for everyday spending.

    One answer
    • £0
    • £1–£999
    • £1,000–£4,999
    • £5,000–£9,999
    • £10,000–£24,999
    • £25,000–£49,999
    • £50,000 or more
    • Not sure
    • Prefer not to say
  6. Q4

    Of your savings outside Pennywell, approximately how much could you realistically move into an instant-access savings account within the next three months without disrupting your financial plans?

    One answer
    • £0
    • £1–£999
    • £1,000–£4,999
    • £5,000–£9,999
    • £10,000–£24,999
    • £25,000–£49,999
    • £50,000 or more
    • Not sure
    • Prefer not to say
  7. Q5

    Pennywell savings pots are instant-access savings accounts inside the Pennywell app. They currently pay 3.1% AER. If the rate stayed at 3.1% AER and you received no prompt or reminder, how likely would you be to open a pot and put money into it in the next three months?

    One answer
    • Definitely would not
    • Probably would not
    • Might or might not
    • Probably would
    • Definitely would
    • Not sure
  8. Q6

    Now suppose the rate stayed at 3.1% AER, but Pennywell showed you an in-app prompt about pots and a reminder on payday with a link to start setting one up. How likely would you be to open a pot and put money into it in the next three months?

    One answer
    • Definitely would not
    • Probably would not
    • Might or might not
    • Probably would
    • Definitely would
    • Not sure
  9. Q7

    Now suppose the rate rose to 3.85% AER, but you received no prompt or reminder. How likely would you be to open a pot and put money into it in the next three months?

    One answer
    • Definitely would not
    • Probably would not
    • Might or might not
    • Probably would
    • Definitely would
    • Not sure
  10. Q8

    Finally, suppose the rate rose to 3.85% AER and Pennywell showed you an in-app prompt about pots and a reminder on payday with a link to start setting one up. How likely would you be to open a pot and put money into it in the next three months?

    One answer
    • Definitely would not
    • Probably would not
    • Might or might not
    • Probably would
    • Definitely would
    • Not sure
  11. Q9

    What is your age?

    One answer
    • 18–34
    • 35–54
    • 55–74
    • 75 or over
    • Prefer not to say
Sample plan

Invite a random sample of Pennywell current-account customers whose account records show they have never opened a pot. Stratify invitations by age band, oversampling smaller bands, and use the screeners to confirm eligibility. With about 700 completes, aim for 175 per age band where feasible; at that size the worst-case 95% margin of error is approximately ±7.4 percentage points per band, before weighting or nonresponse effects. The board's ±5-point requirement needs approximately 385 independent completes per band, or 1,540 in total, before allowing for weighting; it cannot be guaranteed with 700 completes. If that precision is mandatory, increase the target and invite approximately 385 divided by the expected eligible completion rate in each band, with reserve random samples and reminders. Check whether enough eligible customers exist in each band, particularly those aged 75 or over. If not, report achievable precision rather than promise the target. Use account-data age for sampling and, where permission and data quality allow, for analysis of respondents who do not answer Q9.