Benchmarks / SurveyBench / What Ledgerline's accountancy practices want built next / Gemini 4 Neon
Gemini 4 Neon: What Ledgerline's accountancy practices want built next
The decision: which one improvement to build next half-year: AI categorisation of bank transactions, more reliable bank feeds (fewer broken connections), a client portal for requesting documents, or bulk Making Tax Digital submissions. The brief and the data →
- Verdict
- ✗ Not sound
- Research score
- 24 of 100
- Analysis rating
- 629
- Head to head, this task
- won 0 of 6
Why it is not sound
The survey
- Answer bands that overlap or leave gaps
The analysis
- Produced no usable analysis
Scorecard
The analysis
This model produced no usable analysis for this task.
The survey it planned
2 screening questions and 8 questions, as the model wrote them.
-
S1
Is your practice based in the UK and currently using Ledgerline for accountancy or bookkeeping work?
One answer- Yes, we are a UK accountancy or bookkeeping practice currently using Ledgerline
- Yes, we are a UK practice, but we no longer use Ledgerline
- No, we are based outside the UK
- No, I use Ledgerline for a non-accountancy or personal business
Continues if Yes, we are a UK accountancy or bookkeeping practice currently using Ledgerline
-
S2
What is your role in choosing or buying software for your practice?
One answer- I make the final decision
- I share or influence the decision
- I use Ledgerline, but I am not involved in software decisions
- Not sure
Continues if I make the final decision; I share or influence the decision
-
Q1
Which of the following tasks currently slow your practice down the most when using Ledgerline? Please select up to three.
Any that apply- Reconnecting broken or disconnected bank feeds
- Manually categorising bank transactions
- Chasing clients for documents and records
- Submitting Making Tax Digital (MTD) returns one client at a time
- Keeping client records and details up to date
- Tracking client deadlines and task status
- Something else
- None of these slow our practice down
-
Q2
Thinking about the task that slows your practice down the most, what happens and roughly how much time does it cost your team in a typical week? (Optional)
Open textRouting Shown only if Q1 is: Reconnecting broken or disconnected bank feeds; Manually categorising bank transactions; Chasing clients for documents and records; Submitting Making Tax Digital (MTD) returns one client at a time; Keeping client records and details up to date; Tracking client deadlines and task status; Something else
-
Q3
Ledgerline can build only one major improvement in the next six months. Please rank these four options from 1 (most valuable to your practice) to 4 (least valuable to your practice).
Ranking- AI categorisation of bank transactions (automatic suggestions for how to code incoming bank transactions)
- More reliable bank feeds (fewer broken connections and less need to reconnect accounts)
- Client portal for requesting documents (a secure space to request and receive files from clients)
- Bulk Making Tax Digital (MTD) submissions (preparing and submitting MTD returns for multiple clients at once)
-
Q4
If Ledgerline offered AI categorisation of bank transactions as an optional paid add-on in addition to your current plan, which statement best describes your practice?
One answer- We would not use this feature, even if it were included at no extra cost
- We would use it if included in our current plan, but we would not pay extra for it
- We would consider paying extra for it if the price were right
- We would definitely pay extra for it
- Not sure
-
Q5
What is the most your practice would be willing to pay per user, per month for an AI bank transaction categorisation add-on?
One answer- Up to £5 per user, per month
- £6 to £10 per user, per month
- £11 to £15 per user, per month
- £16 to £25 per user, per month
- More than £25 per user, per month
- Not sure
Routing Shown only if Q4 is: We would consider paying extra for it if the price were right; We would definitely pay extra for it
-
Q6
Which Ledgerline plan is your practice currently on?
One answer- Free plan (1 user)
- Paid plan (charged per user, per month)
- Not sure
-
Q7
Including yourself, how many people work in your practice (partners, accountants, bookkeepers and support staff)?
One answer- 1 (sole practitioner)
- 2 to 9 staff
- 10 or more staff
-
Q8
How often does someone in your practice use Ledgerline?
One answer- Every working day
- 2 to 4 times a week
- About once a week
- 1 to 3 times a month
- Less than once a month
Sample plan
INVITATION AND CHANNELS: Invite all 8,200 Ledgerline practice accounts (6,970 free; 1,230 paid) via two tracked channels: an email invitation to every account admin and an in-app banner on login. Append an encrypted practice_account_id and channel tag (email vs in_app) to each survey link so responses can be deduplicated to one per practice, reconciled against CRM plan/seat data, and checked for active-user channel bias (in-app banners over-represent frequent users, while admin emails reach infrequent users). ONE-PER-PRACTICE DEDUPLICATION: Lock the practice_account_id once an eligible respondent completes the survey. If multiple eligible users in a paid multi-user practice start simultaneously, keep the final decision-maker (S2) over a decision influencer, and otherwise retain the earliest completed response. TARGET COMPLETES AND STRATIFIED QUOTAS: Target ~550 total completes, disproportionately oversampling paid practices because they represent 15% of accounts (1,230) but 100% of revenue and nearly all multi-staff practices (since the free plan is limited to 1 user). Set soft quotas of ~275 paid completes (roughly 22% response rate among paid accounts, supported by 2 polite email reminders to non-responding paid admins) and ~275 free completes (roughly 4% response rate; turn off the free-plan in-app banner once the free quota is filled). Within the 550 completes, monitor practice-size quotas to maximise coverage of 2-9 staff (~175-200) and 10+ staff (census-effort outreach targeting ~75-100, depending on actual population size in Ledgerline), with sole practitioners capped at ~250-300. PRECISION TRADE-OFF (IMPORTANT STAKEHOLDER FLAG): Achieving +/-5 percentage points at 95% confidence (worst-case p=0.5) requires n=385 per subgroup without finite population correction (FPC). Across three practice-size bands, even with FPC on Ledgerline's 8,200-practice base, meeting +/-5pp in every size band would require roughly 850-1,000+ completes, which is not feasible within a ~550-complete cap. With ~550 completes (~183 per band if evenly split), expect roughly +/-7.2pp margin of error per size band (slightly narrower with FPC if the 10+ staff customer base is small, e.g., <300 accounts). Flag this trade-off to Product before launch: either proceed with ~550 completes at ~+/-7-8pp precision per size band (sufficient to detect large 15+ point priority gaps) or expand sample size and fieldwork incentives for 2-9 and 10+ staff practices if +/-5pp per band is mandatory. SURVEY PROGRAMMING CONTROLS: Randomise option order in Q1 (anchoring 'Something else' and 'None of these slow our practice down' at the bottom, with 'None of these' set as mutually exclusive and max 3 selections enforced) and randomise all four options in Q3 to prevent order bias. Estimated completion time is 3-4 minutes.