Benchmarks / SurveyBench / Why Tallyroom trials don't convert / Mistral Large 4
Mistral Large 4: Why Tallyroom trials don't convert
The decision: where to put next quarter's product and sales effort to lift trial-to-paid conversion: a new accounting integration, assisted setup, a cheaper plan, or a shorter trial. The brief and the data →
- Verdict
- ✗ Not sound
- Research score
- 68 of 100
- Analysis rating
- 815
- Head to head, this task
- won 6 of 15
Why it is not sound
The survey
- Presumed the stakeholder's view instead of testing it
- A leading question: Q6
The analysis
- Got at least one set number wrong
- Reported a material finding the data does not support
- Made a recommendation that does not follow from the data
- Missed a trap: Teams of 50-249 people are 22% of respondents but 10% of lapsed trials. They cite missing integration and use Sage far more, so unweighted figures overstate integration and Sage demand and understate setup time: unweighted, missing integration is the top reason; weighted, setup time is.
- Missed a trap: 460 of 1,640 invited admins completed (about 28%), with a £10 voucher; those who respond may differ from those who do not.
Findings the judges found unsupported
Sage integration addresses the single largest identifiable blocker: 23.7% of lapsed trials need it before paying, and 61.3% of those blocked by missing integrations say they would likely pay if Tallyroom connected to their software.
The 23.7% Sage requirement is an unweighted respondent figure, not an estimate for all lapsed trials. Size weighting reduces it to approximately 18.9%. Calling Sage the single largest blocker also conflates a prompted requirement with the main reason for not paying; weighted main reasons put setup time first.Setup friction is the second-largest barrier, concentrated in small teams: 27.4% of 1-9 employee firms cite lack of setup time as the main reason, versus 10.0% of 50-249 firms.
Setup time is not the second-largest barrier in the target population. The analysis's own weighting gives approximately 21.6% for setup time, ahead of approximately 16.3% for missing integration.Custom approval chains are a significant unmet need affecting 22.0% of lapsed trials, particularly larger teams where multi-step or conditional approvals are required.
The 22.0% figure describes unweighted respondents, not all lapsed trials. Weighted custom-approval demand is approximately 20.4%. The larger-team pattern is supported, but the population percentage is overstated.
Scorecard
Numbers 10 of 11 right
| Question | Its answer | True | |
|---|---|---|---|
| What percentage of respondents said they would need a Sage integration before paying? | ✓ | 23.7% | 23.7% |
| Weighted to the company-size mix of all lapsed trials, what percentage would need a Sage integration? | ✓ | 18.1% | 18.2% |
| Among businesses with 1-9 employees, what percentage gave 'didn't have time to set it up' as the main reason? | ✓ | 27.4% | 27.4% |
| How many of the open-text answers mention not having time to set Tallyroom up, or setup taking too long? | ✓ | 24 | 24 |
| What is the 95% margin of error, in percentage points, on the unweighted share of respondents needing a Sage integration? | ✓ | 3.9 points | 3.89 points |
| Is the difference in the share giving 'too expensive' as their main reason between businesses with 1-9 employees and those with 50-249 statistically significant at the 95% level? (1 for yes, 0 for no) | ✓ | No | No |
| Weighted to the company-size mix of all 1,640 lapsed trials, about how many teams' main reason was not having time to set it up? | ✓ | 355 | 355 |
| How many of the open-text answers from businesses with 1-9 employees mention not having time to set Tallyroom up, or setup taking too long? | ✗ | 5 | 14 |
| Weighted to the company-size mix of all lapsed trials, what percentage would give 'no integration with our accounting software' as their main reason? | ✓ | 16.3% | 16.3% |
| What percentage of the trial admins invited completed the survey? | ✓ | 28% | 28% |
| Among respondents whose main reason was no integration with their accounting software, what percentage said they would be likely (fairly or very) to pay if Tallyroom connected to it? | ✓ | 61.3% | 61.3% |
Traps in the data 5 of 7 handled
- ✓Teams on 'Other' accounting software (base 17) mostly cite missing integration.
- ✗Teams of 50-249 people are 22% of respondents but 10% of lapsed trials. They cite missing integration and use Sage far more, so unweighted figures overstate integration and Sage demand and understate setup time: unweighted, missing integration is the top reason; weighted, setup time is.
- ✓Q2 is multi-select: percentages are of respondents and add up to more than 100%.
- ✓Likelihood to pay is a hypothetical stated intention.
- ✓The head of sales believes the trial is too long and teams forget about it.
- ✗460 of 1,640 invited admins completed (about 28%), with a £10 voucher; those who respond may differ from those who do not.
- ✓Only teams that did not pay were surveyed, so the survey cannot show what distinguishes them from teams that did; paying teams may also have struggled with setup.
The analysis
Prioritise building the Sage integration and implementing assisted setup/onboarding for next quarter. Do not shorten the trial. Run a pricing experiment (flat-rate vs per-user) for the price-sensitive segment in parallel. Sage integration targets the largest identifiable blocker (23.7% need it; 61.3% of integration-blocked users would pay if connected), directly answering the finance director's question and the product head's demand assessment. Assisted setup addresses the second barrier (18.7% overall, 27.4% of small teams) and is the only lever that addresses the sales head's concern about teams forgetting the trial without cutting the trial period (which only 0.2% cite as too long). Custom approval chains (22.0% demand) should be queued as the next feature after these two. A cheaper headline price is less urgent than fixing the per-user model for teams with low submission rates.
If a pilot with Sage integration shows trial-to-paid conversion no better than the current Xero/QuickBooks baseline, or if assisted setup reduces time-to-first-value but does not improve final conversion rates, the priority should shift to pricing model changes or approval workflow flexibility. Additionally, if win/loss analysis shows Sage users are locked into existing Sage-native solutions (e.g., Sage's own expenses tools) and are unwilling to switch despite integration availability, the addressable market for Sage integration may be smaller than survey intent suggests.
Findings
-
Sage integration addresses the single largest identifiable blocker: 23.7% of lapsed trials need it before paying, and 61.3% of those blocked by missing integrations say they would likely pay if Tallyroom connected to their software.
Q3: 23.7% select 'Sage integration' as a requirement (base 460). Q5: Among the 93 respondents whose main reason was 'no integration with our accounting software' (20.2% of total, Q2), 61.3% rate themselves 4-5 on likelihood to pay £6/user/month if connected. Q4: Sage is used by 25.4% of respondents, second only to Xero (39.6%).
-
Setup friction is the second-largest barrier, concentrated in small teams: 27.4% of 1-9 employee firms cite lack of setup time as the main reason, versus 10.0% of 50-249 firms.
Q2: 27.4% of 1-9 employees (base 168) vs 10.0% of 50-249 employees (base 110) cite 'didn't have time to set it up'. Weighted to the population mix (55% 1-9, 35% 10-49, 10% 50-249), approximately 355 of 1,640 lapsed trials fall into this category. 24 of 109 open-text answers independently mention setup time or complexity.
-
Cutting the trial from 30 to 14 days is not supported: virtually no one cites the trial as too long, and the 'forgot the trial' issue is minor compared to integration and setup barriers.
Q2: Only 0.2% say 'the trial was too long', 1.5% say 'too short', and 3.7% say 'forgot the trial was running'. By contrast, 20.2% cite missing integration and 18.7% cite setup time. The sales head's hypothesis that teams forget long trials is not borne out; those who forgot represent a small minority.
-
Price sensitivity (15.9%) is closely tied to the per-user pricing model rather than the absolute £6 price point: 20.4% want per-company pricing, and open-text answers frequently object to paying for seats where few employees submit expenses.
Q2: 15.9% cite 'too expensive' as the main reason. Q3: 20.4% select 'per-company pricing (not per user)' as a requirement. Open text: Multiple respondents (e.g., T023, T059, T208, T343, T402) explicitly calculate annual per-user costs and reject them because not all staff claim expenses, suggesting a pricing model mismatch rather than simple price resistance.
-
Custom approval chains are a significant unmet need affecting 22.0% of lapsed trials, particularly larger teams where multi-step or conditional approvals are required.
Q3: 22.0% select 'custom approval chains' (base 460), rising to 27.3% among 50-249 employee firms. Q2: 12.6% cite 'approval workflow didn't fit' as the main reason. Open text: 15 answers describe specific workflow mismatches (e.g., T010, T180, T247, T392) involving multiple approvers, external accountants, or batch approvals.
Caveats
- The survey captures stated reasons from lapsed trials only; we cannot observe why current paying customers succeeded, creating a reverse survivorship bias that may overstate barriers relative to enablers.
- Q5 measures stated likelihood to pay (61.3% net likely), not actual willingness to pay; hypothetical bias typically overstates real conversion rates by 10-30%.
- The open-text base is small (109 of 460 respondents, 24%) and self-selected; theme frequencies may not represent the full sample, particularly for niche software (FreeAgent n=33, Other n=17).
- Weighting assumes the company-size mix of all lapsed trials (55% 1-9, 35% 10-49, 10% 50-249) is the correct target population; if conversion behaviour differs by size in ways not captured by the sample, weighted estimates may be imprecise.
- The survey was completed by trial admins only; end-user (employee) experience and adoption barriers may differ from the admin perspective captured here.
Next steps
- decisionBuild Sage integration (Sage 50 / Sage Business Cloud) and launch to lapsed Sage-user trials
- decisionImplement assisted setup: guided onboarding flow, bulk import tools, and dedicated support for the first 14 days
- experimentA/B test flat-rate (per-company) pricing against per-user pricing for teams with low expense-submission density
- researchInterview 20-30 respondents who selected 'None of these' in Q3 and cited 'decided we didn't need it'
- monitorMonitor conversion rates by accounting software post-Sage launch, and test automated trial reminder emails (vs shortening trial)
- researchFollow up with the 38.7% of integration-blocked users who say they are unlikely to pay even if connected
Open-text themes it coded
Missing accounting software integration (Sage, FreeAgent, or other) 22Setup time, complexity, or incomplete onboarding 24Price, cost, or per-user pricing model objections 23Approval workflow does not fit team's process 15Did not need an expenses tool / no problem to solve 12Mobile app or receipt scanning failures 10Chose a competitor or alternative tool 8
The survey it planned
4 screening questions and 10 questions, as the model wrote them.
-
S1
Are you the person who set up the Tallyroom trial for your team?
One answer- Yes
- No
Continues if Yes
-
S2
How many employees does your company have?
One answer- 1-9 employees
- 10-49 employees
- 50-249 employees
- 250 or more employees
Continues if 1-9 employees; 10-49 employees; 50-249 employees
-
S3
Is your team based in the UK?
One answer- Yes
- No
Continues if Yes
-
S4
Did your Tallyroom trial end in the last 90 days without your team converting to a paid plan?
One answer- Yes
- No
Continues if Yes
-
Q1
What was the main reason your team did not convert to a paid Tallyroom plan?
One answer- The price (£6 per user/month) was too high
- We didn't use it enough during the trial to see the value
- It didn't connect to our accounting software (we don't use Xero or QuickBooks)
- We couldn't get set up / it was too complicated to start using
- We were still deciding when the trial ended
- We chose a different expense management tool
- Other (please specify):
- Don't know
-
Q2
Which accounting software does your team currently use?
One answer- Xero
- QuickBooks
- Another accounting software (please specify):
- We don't use accounting software yet
- Not sure
-
Q3
During your 30-day trial, how many days did your team actively use Tallyroom?
One answer- 0 days (we never logged in)
- 1-7 days
- 8-14 days
- 15-21 days
- 22-30 days
-
Q4
Which statement best describes your view on the 30-day trial length?
One answer- 30 days was too short – we needed more time to evaluate properly
- 30 days was about right
- 30 days was too long – we lost interest or forgot about it before the end
- We stopped using it early, regardless of how long the trial was
-
Q5
Did you complete the setup and have staff submit at least one expense during the trial?
One answer- Yes, we fully set it up and staff submitted expenses
- We set it up but no staff submitted any expenses
- We started setup but didn't finish
- We never logged in / never started setup
-
Q6
Which ONE of the following would most likely have made your team convert to a paid plan?
One answer- A lower price per user per month
- Help getting set up (e.g., a guided onboarding session or dedicated support)
- Integration with our specific accounting software (if not Xero or QuickBooks)
- A shorter trial (e.g., 14 days instead of 30) so we would decide faster
- None of these – we wouldn't have converted regardless
- Don't know
Judges: leadingThe shorter-trial option supplies the favourable consequence 'so we would decide faster', steering respondents toward the stakeholder's proposed mechanism.
The option for a shorter trial adds a persuasive justification ('so we would decide faster') which leads the respondent.
-
Q7
How important is it that Tallyroom connects to your accounting software?
One answer- Essential – we wouldn't use Tallyroom without it
- Important, but not essential
- Nice to have, but not a deciding factor
- Not important at all
Routing Shown only if Q2 is: Another accounting software (please specify):; We don't use accounting software yet; Not sure
Judges: assumptiveIt refers to 'your accounting software' but is explicitly shown to teams that said they do not use accounting software.
Q7 is shown to people who answered 'We don't use accounting software yet' or 'Not sure' at Q2. It still asks about connecting to 'your accounting software', which assumes they have some.
-
Q8
What was the main reason you stopped using Tallyroom before the trial ended? (Select all that apply)
Any that apply- It was too complicated / hard to use
- We didn't see the value in it
- The price was too high
- We were missing features we needed
- We forgot about it / lost interest
- We decided to use a different tool
- We ran out of time to evaluate it properly
- Other (please specify):
Routing Shown only if Q3 is: 0 days (we never logged in); 1-7 days; 8-14 days
Judges: assumptive, ambiguousUsing the product on 0–14 days does not establish that a team stopped before the trial ended; some never started, while others may have used it intermittently throughout. 'Main reason' also conflicts with 'Select all that apply', leaving respondents unsure whether to give one principal reason or every applicable reason.
Q8 is shown to people who never logged in (0 days), yet it asks why they 'stopped using' Tallyroom, which assumes they started. It also assumes that low-usage respondents stopped before the trial ended. The stem asks for 'the main reason' but the instruction says 'select all that apply', so respondents cannot tell whether to give one answer or several.
-
Q9
What is your role in the company?
One answer- Owner / Director
- Finance Manager / Accountant / Bookkeeper
- Office Manager / Operations
- Other (please specify):
-
Q10
Is there anything else that would have helped your team decide to pay for Tallyroom?
Open text
Sample plan
Email invitations sent to the admin of every UK-based team in the product database whose Tallyroom trial ended in the last 90 days without converting to paid, excluding companies with 250+ employees. Target: approximately 450 completed responses (incentive: £10 voucher per completer). Quotas: Monitor and enforce minimum targets per company-size band to support analysis: aim for at least 150 completes each for 1-9, 10-49, and 50-249 employees. Note on precision: To achieve ±6 percentage points at 95% confidence for each company-size band requires approximately 267 completes per band (total n≈800). With 450 completes allocated equally (150 per band), the margin of error is ±8 percentage points. Recommendation: If budget allows, increase to 800 completes to meet the ±6% precision requirement; otherwise, report confidence intervals and treat band-level differences of less than 16 percentage points as directional only.