Benchmarks / SurveyBench / What Ledgerline's accountancy practices want built next / MiniMax M3.1 Flash
MiniMax M3.1 Flash: What Ledgerline's accountancy practices want built next
The decision: which one improvement to build next half-year: AI categorisation of bank transactions, more reliable bank feeds (fewer broken connections), a client portal for requesting documents, or bulk Making Tax Digital submissions. The brief and the data →
- Verdict
- ✗ Not sound
- Research score
- 30 of 100
- Analysis rating
- 484
- Head to head, this task
- won 0 of 5
Why it is not sound
The analysis
- Produced no usable analysis
Scorecard
The analysis
This model produced no usable analysis for this task.
The survey it planned
4 screening questions and 8 questions, as the model wrote them.
-
S1
Do you make or influence decisions about which software your practice uses?
One answer- Yes
- No
Continues if Yes
-
S2
Is your practice currently using Ledgerline?
One answer- Yes, we currently use Ledgerline
- No, we have stopped using Ledgerline
- No, we have never used Ledgerline
Continues if Yes, we currently use Ledgerline
-
S3
Which of the following best describes your practice?
One answer- A UK accountancy practice
- A UK bookkeeping practice
- A UK practice providing both accountancy and bookkeeping services
- An accountancy or bookkeeping practice outside the UK
- Another type of organisation
Continues if A UK accountancy practice; A UK bookkeeping practice; A UK practice providing both accountancy and bookkeeping services
-
S4
How many people work at your practice, including you?
One answer- 1 person (sole practitioner)
- 2–9 people
- 10 or more people
Continues if 1 person (sole practitioner); 2–9 people; 10 or more people
-
Q1
In a typical week, what most slows down your practice's work when using Ledgerline? Please describe the task and the problem it creates.
Open text -
Q2
Thinking about the past three months, which of the following have caused delays, extra checking or rework in your practice while using Ledgerline? Select up to three.
Any that apply- Bank transactions had to be categorised manually
- Bank transactions were categorised incorrectly
- A bank-feed connection was broken, disconnected, or missing transactions
- Client records had to be updated manually
- Client documents had to be requested or chased outside Ledgerline
- Making Tax Digital (MTD) submissions had to be prepared, checked, or submitted one client at a time
- Deadlines were difficult to track or meet
- None of these
-
Q3
Thinking about the next six months, please rank the following four Ledgerline improvements from most to least useful for your practice. If two improvements are equally useful, place them in either order.
Ranking- More reliable bank feeds (fewer broken connections)
- AI categorisation of bank transactions
- A client portal for requesting documents
- Bulk Making Tax Digital (MTD) submissions
-
Q4
In the past 30 days, how often has your practice experienced a bank feed that was broken, disconnected, or missing transactions?
One answer- Never
- Once
- 2–3 times
- 4–10 times
- More than 10 times
- We do not use Ledgerline bank feeds
-
Q5
How useful would AI categorisation of bank transactions be for your practice if it accurately categorised most transactions and flagged only uncertain items for review?
ScaleScale 1-5: Not at all useful to Extremely useful
-
Q6
Suppose Ledgerline offered AI categorisation as a paid add-on on top of your current plan, accurately categorising most transactions and flagging only uncertain items for review. What is the maximum additional amount per user per month that your practice would be willing to pay?
One answer- £0 — we would not pay for it
- £1–£4 per user per month
- £5–£9 per user per month
- £10–£19 per user per month
- £20–£29 per user per month
- £30 or more per user per month
- Not sure
- Not applicable — our practice does not categorise bank transactions
-
Q7
Which Ledgerline plan is your practice currently on?
One answer- Free plan (one user)
- Paid plan
- Not sure
-
Q8
In a typical week, how often does your practice use Ledgerline?
One answer- Several times a day
- About once a day
- Several times a week
- About once a week
- Less than once a week
Sample plan
Invite the admin of every one of the 8,200 practice accounts by email, covering all 6,970 free-plan and 1,230 paid-plan accounts, and show the in-app banner to Ledgerline users when they log in. Use the account or practice ID to retain only one completed response per practice, while screening out users who do not make or influence software decisions. Before fielding, tabulate the account frame by practice size and plan and use targeted email and banner reminders to fill quotas. The requested 550 completes and the requested precision are mathematically incompatible: with 550 responses split evenly across three size bands, each band has about 183 responses and the worst-case 95% margin is approximately ±7.2 percentage points before weighting. If ±5 points is binding, do not close the field at 550: obtain at least 385 usable completes in each size band, or 1,155 in total, and preferably 400 per band, or about 1,200 overall, continuing until the weighted effective sample size is at least 385 in every size band. Aim for at least 300 paid-plan completes overall and at least 100 paid-plan completes in each size band where available; oversample paid and hard-to-reach size-plan cells, then calibrate survey weights to the full account distribution by size and plan. If a size band contains fewer than 385 practices, census that band and disclose any nonresponse. If 550 is a hard cap, report the wider actual confidence intervals rather than claiming ±5-point precision. Randomise the feature-related options in Q2 and Q3, make None of these exclusive in Q2, and keep the price bands in Q6 ordered.