Benchmarks / SurveyBench / What Ledgerline's accountancy practices want built next / Gemini 4 Neon

Measured by Spring Prompt

Gemini 4 Neon: What Ledgerline's accountancy practices want built next

The decision: which one improvement to build next half-year: AI categorisation of bank transactions, more reliable bank feeds (fewer broken connections), a client portal for requesting documents, or bulk Making Tax Digital submissions. The brief and the data →

Verdict
✗ Not sound
Research score
24 of 100
Analysis rating
629
Head to head, this task
won 0 of 6

Why it is not sound

The survey

  • Answer bands that overlap or leave gaps

The analysis

  • Produced no usable analysis

Scorecard

The analysis

This model produced no usable analysis for this task.

The survey it planned

2 screening questions and 8 questions, as the model wrote them.

  1. S1

    Is your practice based in the UK and currently using Ledgerline for accountancy or bookkeeping work?

    One answer
    • Yes, we are a UK accountancy or bookkeeping practice currently using Ledgerline
    • Yes, we are a UK practice, but we no longer use Ledgerline
    • No, we are based outside the UK
    • No, I use Ledgerline for a non-accountancy or personal business

    Continues if Yes, we are a UK accountancy or bookkeeping practice currently using Ledgerline

  2. S2

    What is your role in choosing or buying software for your practice?

    One answer
    • I make the final decision
    • I share or influence the decision
    • I use Ledgerline, but I am not involved in software decisions
    • Not sure

    Continues if I make the final decision; I share or influence the decision

  3. Q1

    Which of the following tasks currently slow your practice down the most when using Ledgerline? Please select up to three.

    Any that apply
    • Reconnecting broken or disconnected bank feeds
    • Manually categorising bank transactions
    • Chasing clients for documents and records
    • Submitting Making Tax Digital (MTD) returns one client at a time
    • Keeping client records and details up to date
    • Tracking client deadlines and task status
    • Something else
    • None of these slow our practice down
  4. Q2

    Thinking about the task that slows your practice down the most, what happens and roughly how much time does it cost your team in a typical week? (Optional)

    Open text

    Routing Shown only if Q1 is: Reconnecting broken or disconnected bank feeds; Manually categorising bank transactions; Chasing clients for documents and records; Submitting Making Tax Digital (MTD) returns one client at a time; Keeping client records and details up to date; Tracking client deadlines and task status; Something else

  5. Q3

    Ledgerline can build only one major improvement in the next six months. Please rank these four options from 1 (most valuable to your practice) to 4 (least valuable to your practice).

    Ranking
    • AI categorisation of bank transactions (automatic suggestions for how to code incoming bank transactions)
    • More reliable bank feeds (fewer broken connections and less need to reconnect accounts)
    • Client portal for requesting documents (a secure space to request and receive files from clients)
    • Bulk Making Tax Digital (MTD) submissions (preparing and submitting MTD returns for multiple clients at once)
  6. Q4

    If Ledgerline offered AI categorisation of bank transactions as an optional paid add-on in addition to your current plan, which statement best describes your practice?

    One answer
    • We would not use this feature, even if it were included at no extra cost
    • We would use it if included in our current plan, but we would not pay extra for it
    • We would consider paying extra for it if the price were right
    • We would definitely pay extra for it
    • Not sure
  7. Q5

    What is the most your practice would be willing to pay per user, per month for an AI bank transaction categorisation add-on?

    One answer
    • Up to £5 per user, per month
    • £6 to £10 per user, per month
    • £11 to £15 per user, per month
    • £16 to £25 per user, per month
    • More than £25 per user, per month
    • Not sure

    Routing Shown only if Q4 is: We would consider paying extra for it if the price were right; We would definitely pay extra for it

  8. Q6

    Which Ledgerline plan is your practice currently on?

    One answer
    • Free plan (1 user)
    • Paid plan (charged per user, per month)
    • Not sure
  9. Q7

    Including yourself, how many people work in your practice (partners, accountants, bookkeepers and support staff)?

    One answer
    • 1 (sole practitioner)
    • 2 to 9 staff
    • 10 or more staff
  10. Q8

    How often does someone in your practice use Ledgerline?

    One answer
    • Every working day
    • 2 to 4 times a week
    • About once a week
    • 1 to 3 times a month
    • Less than once a month
Sample plan

INVITATION AND CHANNELS: Invite all 8,200 Ledgerline practice accounts (6,970 free; 1,230 paid) via two tracked channels: an email invitation to every account admin and an in-app banner on login. Append an encrypted practice_account_id and channel tag (email vs in_app) to each survey link so responses can be deduplicated to one per practice, reconciled against CRM plan/seat data, and checked for active-user channel bias (in-app banners over-represent frequent users, while admin emails reach infrequent users). ONE-PER-PRACTICE DEDUPLICATION: Lock the practice_account_id once an eligible respondent completes the survey. If multiple eligible users in a paid multi-user practice start simultaneously, keep the final decision-maker (S2) over a decision influencer, and otherwise retain the earliest completed response. TARGET COMPLETES AND STRATIFIED QUOTAS: Target ~550 total completes, disproportionately oversampling paid practices because they represent 15% of accounts (1,230) but 100% of revenue and nearly all multi-staff practices (since the free plan is limited to 1 user). Set soft quotas of ~275 paid completes (roughly 22% response rate among paid accounts, supported by 2 polite email reminders to non-responding paid admins) and ~275 free completes (roughly 4% response rate; turn off the free-plan in-app banner once the free quota is filled). Within the 550 completes, monitor practice-size quotas to maximise coverage of 2-9 staff (~175-200) and 10+ staff (census-effort outreach targeting ~75-100, depending on actual population size in Ledgerline), with sole practitioners capped at ~250-300. PRECISION TRADE-OFF (IMPORTANT STAKEHOLDER FLAG): Achieving +/-5 percentage points at 95% confidence (worst-case p=0.5) requires n=385 per subgroup without finite population correction (FPC). Across three practice-size bands, even with FPC on Ledgerline's 8,200-practice base, meeting +/-5pp in every size band would require roughly 850-1,000+ completes, which is not feasible within a ~550-complete cap. With ~550 completes (~183 per band if evenly split), expect roughly +/-7.2pp margin of error per size band (slightly narrower with FPC if the 10+ staff customer base is small, e.g., <300 accounts). Flag this trade-off to Product before launch: either proceed with ~550 completes at ~+/-7-8pp precision per size band (sufficient to detect large 15+ point priority gaps) or expand sample size and fieldwork incentives for 2-9 and 10+ staff practices if +/-5pp per band is mandatory. SURVEY PROGRAMMING CONTROLS: Randomise option order in Q1 (anchoring 'Something else' and 'None of these slow our practice down' at the bottom, with 'None of these' set as mutually exclusive and max 3 selections enforced) and randomise all four options in Q3 to prevent order bias. Estimated completion time is 3-4 minutes.