Benchmarks / SurveyBench / Why Tallyroom trials don't convert / GPT-6 Luna
GPT-6 Luna: Why Tallyroom trials don't convert
The decision: where to put next quarter's product and sales effort to lift trial-to-paid conversion: a new accounting integration, assisted setup, a cheaper plan, or a shorter trial. The brief and the data →
- Verdict
- ✓ Sound research
- Research score
- 97 of 100
- Analysis rating
- 1,102
- Head to head, this task
- won 5 of 10
Scorecard
Numbers 11 of 11 right
| Question | Its answer | True | |
|---|---|---|---|
| What percentage of respondents said they would need a Sage integration before paying? | ✓ | 23.7% | 23.7% |
| Weighted to the company-size mix of all lapsed trials, what percentage would need a Sage integration? | ✓ | 18.1% | 18.2% |
| Among businesses with 1-9 employees, what percentage gave 'didn't have time to set it up' as the main reason? | ✓ | 27.4% | 27.4% |
| How many of the open-text answers mention not having time to set Tallyroom up, or setup taking too long? | ✓ | 24 | 24 |
| What is the 95% margin of error, in percentage points, on the unweighted share of respondents needing a Sage integration? | ✓ | 3.9 points | 3.89 points |
| Is the difference in the share giving 'too expensive' as their main reason between businesses with 1-9 employees and those with 50-249 statistically significant at the 95% level? (1 for yes, 0 for no) | ✓ | No | No |
| Weighted to the company-size mix of all 1,640 lapsed trials, about how many teams' main reason was not having time to set it up? | ✓ | 355 | 355 |
| How many of the open-text answers from businesses with 1-9 employees mention not having time to set Tallyroom up, or setup taking too long? | ✓ | 14 | 14 |
| Weighted to the company-size mix of all lapsed trials, what percentage would give 'no integration with our accounting software' as their main reason? | ✓ | 16.3% | 16.3% |
| What percentage of the trial admins invited completed the survey? | ✓ | 28% | 28% |
| Among respondents whose main reason was no integration with their accounting software, what percentage said they would be likely (fairly or very) to pay if Tallyroom connected to it? | ✓ | 61.3% | 61.3% |
Traps in the data 7 of 7 handled
- ✓Teams on 'Other' accounting software (base 17) mostly cite missing integration.
- ✓Teams of 50-249 people are 22% of respondents but 10% of lapsed trials. They cite missing integration and use Sage far more, so unweighted figures overstate integration and Sage demand and understate setup time: unweighted, missing integration is the top reason; weighted, setup time is.
- ✓Q2 is multi-select: percentages are of respondents and add up to more than 100%.
- ✓Likelihood to pay is a hypothetical stated intention.
- ✓The head of sales believes the trial is too long and teams forget about it.
- ✓460 of 1,640 invited admins completed (about 28%), with a £10 voucher; those who respond may differ from those who do not.
- ✓Only teams that did not pay were surveyed, so the survey cannot show what distinguishes them from teams that did; paying teams may also have struggled with setup.
The analysis
Keep the 30-day trial for now; do not cut it to 14 days based on these responses. Put next-quarter effort into an assisted-setup experiment, with particular attention to 1–9 employee teams, and run targeted Sage customer discovery plus a bounded integration business case. If feasible, test Sage integration with a pilot or waitlist before committing to a full build. In parallel, test per-company or capped pricing with actual purchasing behaviour rather than launching a cheaper plan solely on stated objections. Track trial engagement and reminders, but treat them as supporting measures, not the primary investment.
I would reconsider a shorter trial if a controlled 14-day versus 30-day test showed higher paid conversion without reducing activation or team adoption. I would prioritise a full Sage build if a validated count of reachable Sage prospects, technical cost estimate, and pilot or pre-commitment evidence showed incremental paid conversion sufficient to meet the business case. I would elevate pricing over setup if a pricing experiment produced materially higher paid conversion or revenue per trial without harming retention.
Findings
-
Prioritise assisted setup as the clearest near-term conversion experiment, especially for the smallest businesses. Setup friction is a more common stated blocker than trial length.
Not having time to set up was the main reason for 18.7% overall and 27.4% of 1–9 employee businesses (n=168). Applying the observed size-specific rates to the lapsed-trial population's size mix estimates about 355 teams. Setup time or complexity also appeared in 24 of the 109 open-text answers.
-
Do not shorten the 30-day trial to 14 days on this evidence. The responses do not support the claim that the trial is too long; instead, some teams say setup was unfinished when the trial ended.
Only 0.2% selected “the trial was too long” and 3.7% “forgot the trial was running”; 1.5% said the trial was too short. Eight open-text answers explicitly connect the trial ending with unfinished setup.
-
Sage is a substantial integration opportunity among lapsed trials, but treat the stated need as a prioritisation signal—not a forecast of paid customers. Validate the reachable Sage segment and conversion economics before committing to a full build.
23.7% said they would need a Sage integration before paying; weighted to the lapsed-trial population's company-size mix, this is 18.1%. Among Sage users, 85.5% selected Sage integration as a need (n=117), and 58.1% gave missing accounting integration as their main reason. The requested integration is concentrated in a relevant segment, but these are self-reported needs.
-
There is a promising stated willingness-to-pay signal for resolving integration blockers, but it cannot answer how many teams will actually pay. Use it to justify a targeted Sage conversion test, not as a revenue forecast.
Among respondents whose main reason was missing integration, 61.3% said they would be fairly or very likely to pay £6 per user per month if connected (n=93). Across all respondents, the comparable likely share was 37.6% (n=460).
-
Do not make a segment-specific pricing decision from the apparent size difference. Pricing deserves a test, but the difference in stated main-reason shares between the smallest and largest reported size groups is not statistically significant at the 95% level.
“Too expensive” was the main reason for 19.0% of 1–9 employee businesses (n=168) and 15.5% of 50–249 employee businesses (n=110); the approximate two-proportion test does not find a significant difference.
Caveats
- Only 460 of 1,640 invited trial admins completed the survey, a 28.0% response rate. Respondents may differ from nonrespondents; weighting corrects only for the stated company-size mix, not other sources of nonresponse bias.
- The 3.9 percentage-point margin of error is the conventional simple-random-sample approximation for the unweighted Sage-need share. Because this was a voluntary survey with nonresponse, it is not a formal guarantee of precision.
- The reported likelihood to pay is hypothetical stated intent at a specified price, not observed conversion. The survey cannot establish how many teams would actually pay after an integration is built.
- The Sage integration question asked what respondents would need, not whether Sage support alone would cause them to convert. Other blockers, implementation costs, and the number of reachable prospective customers remain unknown.
- The survey did not experimentally compare 14- and 30-day trials, assisted setup, pricing plans, or integrations. Reported reasons are self-attributed and do not establish causal effects on conversion.
- Several subgroup bases are modest, particularly FreeAgent (n=33) and Other accounting software (n=17). Open-text themes are based on 109 answers and are qualitative rather than population estimates.
Next steps
- experimentRun an assisted-onboarding experiment against the current self-serve experience, prioritising small teams and measuring setup completion and paid conversion.
- researchValidate Sage demand with interviews and a technical/commercial discovery phase; recruit Sage users for a pilot or explicit purchase-intent commitment before deciding on a full integration build.
- decisionKeep the 30-day trial unchanged and test onboarding prompts or timely reminders rather than shortening the trial immediately.
- experimentTest per-company or capped pricing with qualified trial users before changing the price plan.
- monitorMonitor actual trial-to-paid conversion and reasons for non-conversion by accounting software, company size, setup completion, and trial engagement.
Open-text themes it coded
Missing accounting software integration 22Setup took too long or respondents lacked time to complete it 24Price or per-user pricing was too high or unsuitable 24Approval workflow did not fit the team's sign-off process 15Mobile app, receipt capture, or upload reliability problems 10Team decided it did not need an expenses app 12Trial ended while setup was still incomplete 8Chose another expense product 8
The survey it planned
5 screening questions and 9 questions, as the model wrote them.
-
S1
Were you the main admin for your team’s Tallyroom free trial?
One answer- Yes
- No
Continues if Yes
-
S2
Is your team based in the UK?
One answer- Yes
- No
Continues if Yes
-
S3
Did your team’s Tallyroom free trial end within the last 90 days?
One answer- Yes
- No
- I’m not sure
Continues if Yes
-
S4
Did your team start a paid Tallyroom subscription after that trial?
One answer- Yes
- No
- I’m not sure
Continues if No
-
S5
How many people does your organisation employ in total, including you?
One answer- 1–9
- 10–49
- 50–249
- 250 or more
- I’m not sure
- Prefer not to say
Continues if 1–9; 10–49; 50–249
-
Q1
Which best describes how far your team got with Tallyroom during the trial?
One answer- We did not get set up
- We set up an account but did not invite staff
- We invited staff, but no expenses were submitted
- Expenses were submitted, but none were approved
- Expenses were approved, but we did not complete the accounting software handoff
- We used the full process: staff submitted expenses, a manager approved them, and they went to our accounting software
- I’m not sure
-
Q2
What were the main reasons your team did not start paying for Tallyroom? Select up to three.
Any that apply- We did not have time to set it up or try it properly
- We had difficulty getting started
- It did not fit our expense process
- It was missing a feature we needed
- It did not connect to the accounting software we use
- The £6 per user per month price was too high
- We did not have enough time to evaluate it before the trial ended
- We forgot about it or lost momentum during the trial
- We no longer needed an expenses app
- We chose another product
- The trial was longer than we needed
- Other reason (please specify)
- I’m not sure
-
Q3
Which accounting software does your business currently use? Select all that apply.
Any that apply- Xero
- QuickBooks
- Sage Accounting
- FreeAgent
- Another accounting software (please specify)
- We do not use accounting software
- I’m not sure
-
Q4
Which missing integrations or features, if any, would have made Tallyroom more useful to your team? Select up to three.
Any that apply- An integration with Sage Accounting
- An integration with FreeAgent
- An integration with another accounting software (please specify)
- Integrations with payroll, HR, or other business software
- Mileage expense tracking
- More options for setting up approval steps
- Automated reminders for staff or approvers
- More ways to manage or report on expenses
- None; the available integrations and features were sufficient
- Other (please specify)
- I’m not sure
-
Q5
Which statement best describes the 30-day trial length for your team?
One answer- It was too short for us to evaluate Tallyroom
- It was about the right length
- It was longer than we needed
- It was long enough, but we forgot about Tallyroom or lost momentum before it ended
- I don’t remember
-
Q6
If the trial had lasted 14 days instead of 30, what do you think would most likely have happened?
One answer- We would have been more likely to start paying because we would have made a decision sooner
- We would have been less likely to start paying because we would have needed more time to evaluate it
- We would have been about as likely to start paying
- We would have been more likely to forget about it or lose momentum
- I’m not sure
-
Q7
If a 30-minute one-to-one setup session with a Tallyroom specialist had been available during your trial, how likely is it that your team would have started a paid subscription?
ScaleScale 1-5: Very unlikely to Very likely
-
Q8
How did the £6 per user per month price affect your team’s decision?
One answer- It was acceptable; price was not a reason we did not subscribe
- It felt somewhat expensive, but we might have subscribed at that price
- It was too expensive for our team
- Price was not relevant to our decision
- I’m not sure
-
Q9
Which one change would have been most likely to make your team start a paid Tallyroom subscription?
One answer- An integration with another accounting software we use
- A feature we needed
- Help setting up Tallyroom
- A lower price or cheaper plan
- A shorter, 14-day trial
- More time to evaluate Tallyroom
- None of these would have changed our decision
- Other change (please specify)
- I’m not sure
Sample plan
Email an invitation to the admin of every UK team in the product database whose Tallyroom trial ended in the last 90 days without the team starting a paid subscription. Invite one admin per team, use the screener to confirm eligibility, and offer a £10 voucher for completing the online survey. Aim for about 450 completes, with equal quotas of 150 in each employee-size band (1–9, 10–49, 50–249) so the bands can be compared. This target does not meet the requested ±6 percentage-point precision at 95% confidence: with 150 completes per band, the worst-case margin is about ±8 points. Approximately 267 completes per band (801 total) are needed for ±6 points under a conservative large-population assumption. Invite the full eligible cohort and, if ±6-point precision is essential, extend fieldwork or increase the completion target; recalculate using the finite population in each band once those counts are known. Weight overall results to the eligible database distribution by size if quotas make the achieved sample disproportionate.