Benchmarks / SurveyBench / What Ledgerline's accountancy practices want built next / Qwen4 Max
Qwen4 Max: What Ledgerline's accountancy practices want built next
The decision: which one improvement to build next half-year: AI categorisation of bank transactions, more reliable bank feeds (fewer broken connections), a client portal for requesting documents, or bulk Making Tax Digital submissions. The brief and the data →
- Verdict
- ✓ Sound research
- Research score
- 100 of 100
- Analysis rating
- 1,373
- Head to head, this task
- won 13 of 15
Scorecard
Numbers 11 of 11 right
| Question | Its answer | True | |
|---|---|---|---|
| What percentage of all respondents ranked AI categorisation of bank transactions first? | ✓ | 37.8% | 37.8% |
| Among respondents on a paid plan, what percentage ranked more reliable bank feeds first? | ✓ | 48.3% | 48.3% |
| What is the mean rank of more reliable bank feeds across all respondents (1 = highest priority, 4 = lowest), to two decimal places? | ✓ | 1.99 | 1.99 |
| What percentage of all respondents ranked AI categorisation of bank transactions last (4th)? | ✓ | 42.2% | 42.2% |
| What percentage of all respondents said yes, their practice would pay £8 per user per month for AI categorisation? | ✓ | 27.7% | 27.7% |
| What is the 95% margin of error, in percentage points, on the unweighted share of all respondents who ranked AI categorisation first? | ✓ | 4.06 points | 4.06 points |
| Is the difference in the share ranking AI categorisation first between practices of 2-9 people and practices of 10-19 people statistically significant at the 95% level? (1 for yes, 0 for no) | ✓ | No | No |
| How many of the open-text answers mention bank feeds breaking, disconnecting or arriving late? | ✓ | 74 | 74 |
| How many of the open-text answers from respondents on a paid plan mention chasing clients for documents? | ✓ | 10 | 10 |
| Ledgerline has 1,230 paid practices. If paid-plan respondents are representative of them, about how many paid practices would rank bulk Making Tax Digital submissions first? | ✓ | 325 | 324 |
| What percentage of the 8,200 practice accounts invited completed the survey? | ✓ | 6.7% | 6.7% |
Traps in the data 7 of 7 handled
- ✓AI categorisation has the most first places (38%, against 35% for bank feeds) but also the most last places (42%): it is polarising. More reliable bank feeds have the best mean rank (1.99, against 2.57 for AI) and the fewest last places (7%).
- ✓Free-plan users are 46% of respondents but bring in no revenue; paid practices (15% of accounts, all of the revenue) put bank feeds first (48%) and bulk MTD (26%) ahead of AI (10%). The AI first-choice lead comes from free users (70% of whom rank it first). Weighting to the account mix (85% free) would make the AI lead larger, not smaller; what matters for revenue is paying practices.
- ✓28% said their practice would pay £8 per user per month for AI categorisation (free plan 40%, paid 17%): a stated intention about a hypothetical add-on.
- ✓Every one of the 12 practices with 20 or more people ranked the client portal first.
- ✓69% of completes came from an in-app banner seen only by users who logged in; 52% of banner respondents use Ledgerline on 5 or more days a week against 19% of email respondents and 18% of all accounts. The sample over-represents heavy users and the response rate is 6.7%.
- ✓Practices of 2-9 people rank AI first more often than practices of 10-19 (20% against 14%, bases 172 and 106), but the difference is not significant at 95%.
- ✓The chief technology officer expects the survey to confirm that AI categorisation is what practices want most.
The analysis
Build more reliable bank feeds next half-year, and tell the board that the survey backs feeds among paying customers — not AI. Three independent lines of evidence converge: feeds is the consensus choice overall (mean rank 1.99, 73.8% top-two, only 7.5% last of 548); it wins decisively among the 1,230 practices that produce all our revenue (48.3% first, mean 1.76, 5.1% last of 296, ≈594 practices); and it is the largest open-text theme (74 of 186 answers), with 25 of those describing the manual-coding backlog as a downstream consequence of feed failure. Frame the investment as revenue protection and retention for the paid base, not as new add-on revenue: we have no evidence quantifying churn from broken feeds, and that is the number to go and get. Answer the CTO directly: AI's 37.8% first-place share is within ±4.1pp of feeds' 34.9%, it is the last choice for 42.2% overall and 65.5% of paid respondents, and its support is concentrated in free-plan sole traders (70.2% of free, 60.9% of 'just me') who pay nothing — so no, the survey does not back building it next. Answer the finance director: £8 AI add-on revenue is not forecastable from stated intent; the paid 'yes' rate of 16.9% implies ~208 practices (155–261), i.e. roughly £20k ARR at one seat each, £60k at three, £100k at five, before discounting intent inflation — and we do not know seats. Answer the head of sales: the portal signal is 12 respondents in the 20+ band and is contradicted at 10-19 (6.6% first, 27.4% last); it is a hypothesis for a targeted sales study, not a priority. Alongside the feeds work, do two cheap things: (a) ship rules/recurring-transaction auto-coding and fix the existing poor suggestions (L511, L193, L339) to attack coding pain without an AI bet, and (b) check performance telemetry, since slowness drew 26 of 186 mentions — more than document chasing — and is worst in our larger paid accounts. Name bulk MTD submissions as the fast-follow (26.4% of paid first, 30.2% of 10-19, ~325 paid practices) and run a scoping spike now so it can start the moment feeds work is de-risked.
Four things would flip or reshape this. (1) Telemetry showing feed breakage is already rare or concentrated in a couple of banks, or that most failures are bank-side/Open Banking re-authorisation rules we cannot remove — then the work shifts from 'reliability' to re-auth UX and detection, and bulk MTD becomes the better use of the half-year. (2) A non-response and weighting exercise (plan × usage × channel, plus ARR and seat weights from account records) showing the paid-column result is an artefact of who answered — particularly if the email arm or low-usage paid practices rank bulk MTD or the portal first. (3) A live price test, not a survey question, showing the AI add-on converts a material share of paid seats at £8 (order of £250k+ incremental ARR) with low cannibalisation, or evidence that AI is a strong free-to-paid conversion lever for the 6,970 free accounts — that would justify AI as a parallel commercial bet. (4) Win/loss and ARR-by-size data showing the client portal is decisive in large-firm deals worth more than the retention value of feeds; 12 survey responses cannot establish this, but 20 lost deals could. Evidence that would strengthen rather than change the decision: a demonstrated association between feed breakage and churn/downgrade in account data, and measured hours lost per broken connection.
Findings
-
More reliable bank feeds is the only option with broad consent rather than a faction: it is the top-two choice for nearly three-quarters of respondents and almost nobody's bottom choice. It is the safest build for the whole user base.
All 548 completes: bank feeds mean rank 1.99 (best of four), 34.9% first + 38.9% second = 73.8% top-two, only 7.5% last. AI categorisation: 37.8% first but 42.2% last, mean 2.57. Portal mean 2.77, bulk MTD mean 2.67. Q2 order was randomised, so this is not a position effect.
-
Among the customers who actually generate revenue, bank feeds wins outright and AI categorisation is actively rejected. The roadmap decision should be made on the paid column, not the total column.
Paid plans (n=296): bank feeds 48.3% first, mean rank 1.76, only 5.1% last. AI 10.1% first and 65.5% last (mean 3.35). Bulk MTD 26.4% first (mean 2.23), portal 15.2% first (mean 2.66). Applied to the 1,230 paid practices: feeds ~594, bulk MTD ~325, portal ~187, AI ~124 would rank each first. Paid respondents are 24% of all paid practices (296/1,230) — far better coverage than the free side (252 respondents for ~6,970 free accounts, 3.6%).
-
Answer to the CTO: no. The survey does not back building AI categorisation next. Its first-place 'lead' is inside sampling noise, and it is produced almost entirely by free-plan sole traders who pay us nothing.
AI 37.8% first vs bank feeds 34.9% (n=548): a 2.9pp gap against a 95% margin of error of ±4.1pp on the AI share alone — not a lead. AI first: 70.2% of free plan (n=252) vs 10.1% of paid (n=296); 60.9% of 'just me' (n=258) vs 20.3% of 2-9 (n=172) and 14.2% of 10-19 (n=106). Free plans are 85% of the 8,200 accounts and 0% of revenue. AI is the last choice for 65.5% of paid respondents.
-
The 'Total' column must not be quoted to the board as a picture of our customers: the sample is heavily skewed to payers and heavy users, and the skew differs by recruitment channel. Any population-level claim needs weighting first.
548 completes from 8,200 invited accounts = 6.7% response, self-selected. Sample 54.0% paid vs 15% in account records; 41.2% use Ledgerline 5+ days/week vs 18% of accounts. Banner respondents (376) 52% heavy-use vs email respondents (172) 19% — the email arm roughly matches the population's 18%. Reweighting the Q2 first-choice shares to the 85/15 account plan mix gives AI ~61% and feeds ~23%; reweighting to revenue-bearing practices reverses it. Q6 matched account records for every respondent, so the plan split itself is reliable.
-
Answer to the finance director: we cannot forecast AI add-on revenue from this survey, and the plausible ceiling is small. The paying base's stated demand is ~17%, we have no seat counts, and stated intent at a single price is an upper bound, not a forecast.
Q3 paid 'yes' 16.9% of 296 (±4.3pp at 95% → 12.6–21.2%); 'no' 57.4%, 'not sure' 25.7%. 16.9% × 1,230 paid practices ≈ 208 practices (range ~155–261). At £8/user/month that is ~£20k ARR at 1 seat/practice, ~£60k at 3 seats, ~£100k at 5 seats — and Q5 counts people at the practice, not Ledgerline seats, so the seat multiplier is unknown. Even converting every paid 'not sure' only reaches 42.6% (~524 practices). Free-plan 'yes' is 40.5% but free-plan 'no' is also 40.5% (n=252), and free is a single-user plan generating no revenue today. Overall 27.7% 'yes' is not a population figure given the 54%-paid sample.
-
Answer to the head of sales: the 'biggest practices all want the portal' claim rests on 12 respondents and is contradicted by the next size band down. Treat it as a hypothesis to test with sales evidence, not a build priority.
20+ people: base 12 (2.2% of 548), 100% ranked the portal first — 12 people, 95% interval roughly 76–100%, and one changed answer moves it 8pp. 10-19 people (n=106): portal only 6.6% first and 27.4% last, while feeds is 49.1% first and bulk MTD 30.2% first. Paid mean rank puts the portal third of four (2.66). Open text: 15 of 186 answers mention chasing documents, 10 from paid respondents, and 4 of those 10 are the 20+ band (L215, L509, L540, L544). We have no account-record count or ARR for 20+ practices, so we cannot size the prize.
-
Bulk MTD submissions is the credible runner-up and should be the named fast-follow, especially for 10-19-person paid practices. It is the second-strongest paid signal and concentrates in deadline weeks, which is when churn risk peaks.
Paid (n=296): 26.4% first, mean rank 2.23, only 7.8% last. 10-19 people (n=106): 30.2% first. 2-9 people (n=172): 22.7% first. ~325 of the 1,230 paid practices would rank it first. Open text: 25 of 186 answers describe one-by-one MTD filing, 22 of them from paid respondents (e.g. L289 'multiply by 90 and you lose a day', L368 140 clients, L111 60-odd returns).
-
Feed failure and manual coding are one causal chain, not two independent problems: a third of feed complaints describe the coding backlog as the consequence. Fixing feeds removes part of the case for AI, while AI on late or missing data would not fix the reconciliation problem.
Open text (186 answers, 34% of completes): 74 mention feed failure (breaks, disconnects, re-authorisation, late or missing transactions) — the largest theme; 56 mention manual categorisation; 25 mention both, several explicitly causally (L004, L075, L159, L235, L378, L410, L468, L472, L495: feed dies → weeks of hand-coding). Coding complaints are 40 free vs 16 paid respondents, matching the free/solo skew in Q2.
-
Some of the coding pain may be removable cheaply, without an AI bet — test rules/recurring-transaction auto-coding and fix the existing suggestions before committing to an £8 AI add-on.
L339: 'repeating the same choices for recurring payments'; L193: 'no rule seems to code them automatically'; L511 (paid): 'the suggestions are rarely right' — i.e. a categorisation aid already exists and is failing. 56 of 186 open answers cite manual coding, but only 16.9% of 296 paid respondents would pay £8 for AI and 65.5% rank it last.
-
Application slowness is an unlisted problem mentioned more often than document chasing, and it hits our larger paid accounts. Check telemetry before assuming the four surveyed options exhaust the roadmap.
26 of 186 open answers (14%) describe slow pages, search, reports or timeouts, versus 15 for document chasing and 25 for MTD. Larger paid practices are explicit: L279 ('fine at 50 clients, noticeably slower at 300'), L309 (8-10s per client record, 200 clients), L355, L164, L093. 25 of 186 (13%) report no problems at all, so Q2's forced ranking made everyone name a priority regardless of whether they feel pain.
-
Do not segment the roadmap on the 2-9 vs 10-19 difference in AI demand — it is not statistically distinguishable.
AI ranked first: 20.3% of 2-9 people (n=172) vs 14.2% of 10-19 (n=106). Difference 6.1pp, standard error ~4.6pp, 95% interval roughly -2.9pp to +15.1pp — includes zero.
Caveats
- Response rate is 6.7% (548 of 8,200 invited accounts) and self-selected. Tables are unweighted. The sample is 54% paid vs 15% of accounts, and 41.2% heavy-use (5+ days/week) vs 18% of accounts. The 'Total' column describes engaged respondents, not our customer base and not our revenue base.
- Channel skew is measurable: 52% of banner respondents (376) are heavy users vs 19% of email respondents (172); the population is 18%. The email arm is closer to the population on usage. We cannot cross-tab Q2 by channel from the tables supplied — that cut is needed before any population-level claim.
- The paid column (296) covers 24% of the 1,230 revenue-generating practices and is the best evidence we have, but it is still self-selected: respondents may be the ones with a grievance or the ones most engaged. Direction of any residual bias is unknown.
- The 20+ people cell is 12 respondents. '100%' means 12 people; the 95% interval runs roughly 76–100%. It cannot support a roadmap commitment, and we have no account-record count, seat count or ARR for that band.
- The AI-vs-feeds first-place gap (37.8% vs 34.9%) is smaller than the ±4.1pp margin of error on either share. The 2-9 vs 10-19 AI gap (6.1pp, SE ~4.6pp) is also not significant. Do not present either as a real difference.
- Q2 is a forced, ipsative ranking of exactly four options with no 'none of these' and no intensity or cost trade-off: it tells us relative order, not how much anyone cares or would pay. 25 of 186 open-text answers (13%) say nothing slows them down, yet all 548 had to name a first choice.
- Q3 is stated purchase intent at a single price, with no comparison price for the other three improvements and no seat data (Q5 counts people at the practice, not Ledgerline seats; the free plan is one user). Stated 'yes' typically overstates real purchase substantially. It cannot be converted into a revenue forecast — only into a bounded illustration.
- Q3 was put to free-plan respondents too, 40.5% of whom said yes; whether an £8 add-on is even buyable on the free plan, and whether it would convert free accounts to paid, is undefined in the question and untested.
- Open text was answered by only 186 of 548 (34%), a further self-selection. Use it for mechanism and language, never for prevalence or for sizing.
- bankfeed_mentions = 74 counts any feed-failure mention (breaks, disconnects, re-authorisation, late or missing transactions). Of these, 67 explicitly describe breaking, disconnecting or late arrival; the other 7 (L173, L252, L259, L318, L322, L492, L506) describe missing transactions only. documents_paid = 10 excludes chasing clients to re-authorise bank access, which is counted under feeds.
- S1 admitted influencers as well as decision-makers and one response per practice was kept; we cannot split budget-holders from influencers in the tables. Q5 size is self-reported headcount, not seats or client volume. Only Q6 is verified against account records.
- Fielding was a single two-week window in September. January self-assessment and VAT quarter-end pains (document chasing, MTD batching, coding backlogs) are seasonal and may be under- or over-salient relative to a full year.
- The survey contains no churn, expansion, win/loss, seat or ARR data, and no product telemetry. It can rank stated priorities; it cannot size the revenue at stake for any of the four options.
Next steps
- decisionCommit the next half-year's primary build to bank-feed reliability, scoped as: detect and surface silent disconnections, reduce dependence on clients re-authorising, and detect late/missing/duplicate transactions before reconciliation. Record bulk MTD submissions as the named fast-follow.
- monitorInstrument and baseline feed health from product telemetry: monthly disconnection rate per connection, share of failures that are silent, days from failure to detection, transaction arrival lag distribution, missing/duplicate rate, broken down by bank (Barclays, Monzo, Santander, Tide, Starling) and by practice size/plan.
- researchRe-cut and weight the survey before anything else goes to the board: Q2/Q3 by recruitment channel, by plan × usage, and by decision role (S1); apply account-record weights for plan and usage; report paid-only and email-only sensitivity analyses alongside the unweighted totals.
- researchQuantify the commercial stake of feed failure in account data: compare churn, downgrade, seat contraction and support-ticket volume for paid practices with high versus low feed-breakage exposure over the last 12 months.
- experimentShip a low-cost coding assist to a pilot cohort: rules/memory for recurring transactions plus improved default suggestions, and measure actual coding time and correction rates. Do not build the full AI add-on yet.
- experimentIf coding assist shows real time savings, run a live price test of the AI add-on (real checkout or a paid pilot at £8 and one or two alternative prices) on paid practices, capturing seat counts; optionally test it as a free-to-paid conversion offer separately.
- researchRun 8-12 depth interviews with 10-19 and 20+ person paid practices, plus a win/loss review of large-firm deals, focused on document chasing and the portal; pull practice counts, seats and ARR by size band from account records.
- researchScope bulk MTD submissions technically (effort, HMRC API constraints, deadline-load risks) and check performance telemetry (p95 page load, report/search timeouts) by practice size and client count.
- monitorTrack the paid-plan priority ranking and feed-related support contacts in a short quarterly pulse to paying admins only, with an intensity/'none of these' option and a January fielding wave.
Open-text themes it coded
Bank feeds unreliable: connections break, silently disconnect, need re-authorising, or transactions arrive late or missing 74Manual categorisation/coding of bank transactions is the main time sink 56Making Tax Digital returns must be filed one client at a time (no bulk filing) 25Chasing clients for documents/records by email and phone 15Application performance: slow page loads, search, reports and timeouts (not one of the four surveyed options) 26No problems reported with Ledgerline 25
The survey it planned
3 screening questions and 8 questions, as the model wrote them.
-
S1
Which of these best describes your practice?
One answer- Accountancy practice
- Bookkeeping practice
- Both accountancy and bookkeeping
- None of these
Continues if Accountancy practice; Bookkeeping practice; Both accountancy and bookkeeping
-
S2
Is your practice based in the UK?
One answer- Yes
- No
Continues if Yes
-
S3
What part do you play in decisions about the software your practice uses?
One answer- I make the final decision
- I decide jointly with others
- I influence the decision but do not make it
- I have no say in software decisions
Continues if I make the final decision; I decide jointly with others; I influence the decision but do not make it
-
Q1
In the last 3 months, which of the following have slowed your practice down when using Ledgerline? Select all that apply.
Any that apply- Bank-feed connections breaking or needing to be reconnected
- Categorising bank transactions one by one
- Chasing clients for missing documents or records
- Sending Making Tax Digital submissions one client at a time
- Keeping track of client deadlines
- Setting up or importing client records
- Finding things in Ledgerline or working out how to use it
- Something else (you can describe it in the final question)
- Nothing has slowed us down
- Don't know
-
Q2
Which ONE of those costs your practice the most time in a typical week?
One answer- Bank-feed connections breaking or needing to be reconnected
- Categorising bank transactions one by one
- Chasing clients for missing documents or records
- Sending Making Tax Digital submissions one client at a time
- Keeping track of client deadlines
- Setting up or importing client records
- Finding things in Ledgerline or working out how to use it
- Something else
- Don't know
Routing Shown only if Q1 is: Bank-feed connections breaking or needing to be reconnected; Categorising bank transactions one by one; Chasing clients for missing documents or records; Sending Making Tax Digital submissions one client at a time; Keeping track of client deadlines; Setting up or importing client records; Finding things in Ledgerline or working out how to use it; Something else (you can describe it in the final question)
-
Q3
Ledgerline can build only ONE of the following in the next six months. Rank them by how much difference each would make to your practice: 1 = biggest difference, 4 = smallest difference. Please rank all four. If your practice would not use one of them, put it last.
Ranking- More reliable bank feeds: connections break less often and stay connected
- Automatic category suggestions for bank transactions: the software suggests a category for each transaction, which you check and can change
- A client portal: request documents from clients and receive them in one place
- Bulk Making Tax Digital submissions: submit for many clients in one go
-
Q4
Thinking about the improvement you ranked 1: how urgent is it for your practice?
ScaleScale 1-5: Not urgent - nice to have to Urgent - we struggle without it
-
Q5
The automatic category suggestions described above (sometimes called AI categorisation) would be an optional paid add-on, charged per user per month on top of your current plan. Suggestions would sometimes be wrong, so someone in your practice would still need to check them. At £PRICE per user per month, would your practice buy this add-on?
One answer- Definitely would buy
- Probably would buy
- Probably would not buy
- Definitely would not buy
- We would not pay extra for this at any price
- We don't use bank feeds in Ledgerline
- Don't know
-
Q6
Including you, how many people work in your practice?
One answer- Just me (sole practitioner)
- 2-9 people
- 10-24 people
- 25 or more people
-
Q7
Roughly how many of your practice's clients are set up in Ledgerline?
One answer- None
- 1-10
- 11-50
- 51-150
- 151-500
- More than 500
- Don't know
-
Q8
Anything else you would like to tell us about what slows your practice down, or about the four possible improvements? Please write briefly. (Optional)
Open text
Sample plan
UNIT AND FRAME: one response per practice account, from all 8,200 UK accountancy/bookkeeping practice accounts on the Ledgerline base (6,970 free, 1,230 paid). Each account gets one tokenised link; the first submitted response locks the token, the in-app banner is then suppressed for every user of that account, and the admin email link resolves to the same token. The admin may forward the link to the colleague who actually decides on software; the screener (S1-S3) removes non-decision-makers and non-UK/non-practice respondents. FIELDWORK: in-app banner on login for all users plus email to the account admin on day 1, reminders day 6 and day 12, close day 21. Invitation and questionnaire are neutral: they say 'help us choose what to build next', name all four options nowhere in the invite, and never mention that anyone internally favours AI. Small neutral thank-you (entry into a prize draw for charity donations) to lift response. Estimated response 12-18% of accounts = roughly 980-1,470 completes. PRECISION - THE STATED 550 IS NOT ENOUGH, FLAG BEFORE FIELDWORK: plus/minus 5pp at 95% for a worst-case proportion (p=0.5) needs n=385 per practice-size cell, i.e. about 1,155 completes across three cells before any finite-population correction. At 550 completes (about 183 per cell) the margin is about plus/minus 7.2pp. Recommended target: about 1,000 completes (minimum 900), using finite-population correction against the actual size distribution pulled from account data before launch. Illustrative quotas if the base is roughly 5,300 sole / 2,500 with 2-9 staff / 400 with 10+: sole >=360, 2-9 >=340, 10+ = census attempt, >=200 (required n with FPC about 359 / 334 / 197); add about 10% headroom for weighting design effect. Verify the real size split from account records (seats, payroll/seat declarations, CSM records) in week 1 and re-set quotas from it; monitor Q6 daily. The 10+ stratum is likely small, so census it: invite every 10+ account, add a personal CSM/account-manager nudge and a phone follow-up offer. If the 10+ population is under about 200, plus/minus 5pp may be unreachable even at full census response - report the achieved margin rather than the requested one. PAID SUBGROUP: invite all 1,230 paid accounts and target >=300 paid completes (FPC-corrected requirement about 293) so paid-plan results also carry about plus/minus 5pp; that needs about a 24% paid response rate, so prioritise paid reminders. Free-plan sole practitioners will flood in first: cap that cell once its quota is met and redirect reminder effort to 10+ and paid accounts. FALLBACK, agreed with product in writing before launch: either extend fieldwork/add reminders to reach about 1,000, or accept about plus/minus 7pp per size cell at 550, or collapse to two size bands (sole vs 2+). Plan x size and intensity x plan intersections are NOT powered to plus/minus 5pp at any feasible n and will be labelled directional. DATA QUALITY (no separate attention question, to protect the 8-question/5-minute budget): drop screener fails, duplicate tokens, completes under 40% of median duration, straight-lined rankings with identical text patterns plus gibberish open text; report exclusions. APPENDED DATA (declared in the invite privacy notice): plan, paid seats, tenure, login active days, number of connected bank feeds, MTD submission volume, broken-feed support tickets in last 12 months - used for intensity, validation and weighting instead of extra questions. WEIGHTING: inverse selection probability plus non-response adjustment to known plan x size population totals; report weighted estimates with unweighted bases and 95% CIs. Estimated completion time from the pilot (n=15): about 3.5-4.5 minutes; pilot also checks that the ranking works on mobile and that no option wording is read as 'the one they want us to pick'.