Benchmarks / SurveyBench / Why Tallyroom trials don't convert / Particle-1

Measured by Spring Prompt

Particle-1: Why Tallyroom trials don't convert

The decision: where to put next quarter's product and sales effort to lift trial-to-paid conversion: a new accounting integration, assisted setup, a cheaper plan, or a shorter trial. The brief and the data →

Verdict
✗ Not sound
Research score
94 of 100
Analysis rating
1,193
Head to head, this task
won 8 of 13

Why it is not sound

The survey

  • A double-barrelled question: Q7

Scorecard

Numbers 11 of 11 right

QuestionIts answerTrue
What percentage of respondents said they would need a Sage integration before paying?✓23.7%23.7%
Weighted to the company-size mix of all lapsed trials, what percentage would need a Sage integration?✓18.1%18.2%
Among businesses with 1-9 employees, what percentage gave 'didn't have time to set it up' as the main reason?✓27.4%27.4%
How many of the open-text answers mention not having time to set Tallyroom up, or setup taking too long?✓2424
What is the 95% margin of error, in percentage points, on the unweighted share of respondents needing a Sage integration?✓3.9 points3.89 points
Is the difference in the share giving 'too expensive' as their main reason between businesses with 1-9 employees and those with 50-249 statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
Weighted to the company-size mix of all 1,640 lapsed trials, about how many teams' main reason was not having time to set it up?✓355355
How many of the open-text answers from businesses with 1-9 employees mention not having time to set Tallyroom up, or setup taking too long?✓1414
Weighted to the company-size mix of all lapsed trials, what percentage would give 'no integration with our accounting software' as their main reason?✓16.3%16.3%
What percentage of the trial admins invited completed the survey?✓28%28%
Among respondents whose main reason was no integration with their accounting software, what percentage said they would be likely (fairly or very) to pay if Tallyroom connected to it?✓61.3%61.3%

Traps in the data 7 of 7 handled

  • ✓Teams on 'Other' accounting software (base 17) mostly cite missing integration.
  • ✓Teams of 50-249 people are 22% of respondents but 10% of lapsed trials. They cite missing integration and use Sage far more, so unweighted figures overstate integration and Sage demand and understate setup time: unweighted, missing integration is the top reason; weighted, setup time is.
  • ✓Q2 is multi-select: percentages are of respondents and add up to more than 100%.
  • ✓Likelihood to pay is a hypothetical stated intention.
  • ✓The head of sales believes the trial is too long and teams forget about it.
  • ✓460 of 1,640 invited admins completed (about 28%), with a £10 voucher; those who respond may differ from those who do not.
  • ✓Only teams that did not pay were surveyed, so the survey cannot show what distinguishes them from teams that did; paying teams may also have struggled with setup.

The analysis

Recommendation

Prioritise assisted setup/onboarding as next quarter's main sales and product-led conversion effort, paired with focused Sage integration discovery and a build business case. Do not cut the trial to 14 days: trial length is not the blocker, and setup time is. Answer the head of sales directly: no, shortening the trial is not supported. Answer the head of product: Sage demand is clear but concentrated—23.7% of all respondents need Sage, 85.5% of Sage users need it, and weighted demand is 18.1%. Answer finance: among those whose main reason was no integration, 61.3% say they would be likely to pay if connected, but treat this as intent, not a conversion forecast. A cheaper plan is worth testing, but not leading with, because price is material and per-company pricing has demand, yet the small-vs-large price difference is not statistically significant.

What would change it

Change the recommendation if a controlled assisted-setup pilot fails to improve setup completion or trial-to-paid conversion. Also revisit if Sage interviews show low willingness to pay or unacceptable variant scope, if a pricing test shows materially better conversion and revenue per account, or if a controlled trial-length test shows 14 days does no harm.

Findings

  1. Assisted setup is the broadest lever: setup time was the largest weighted main reason for not paying among lapsed trials, and open text confirms unfinished setup rather than lack of interest.

    High confidenceBase 460

    Weighted to the 1,640 lapsed-trial size mix, 21.6% (about 355 teams) gave 'didn't have time to set it up' as main reason; unweighted 18.7% (86/460). 24 of 109 open-text answers mention setup time/too long; 14 of those are 1-9 employee firms.

  2. Sage integration has clear, concentrated demand and should be evaluated as a product bet, not assumed from overall averages.

    High confidenceBase 460

    23.7% of all respondents (109/460) say they would need Sage integration before paying; this is 85.5% of Sage users (100/117). Among Sage users, 58.1% (68/117) gave no integration as main reason. Weighted to population size mix, Sage need is 18.1%.

  3. Connecting to accounting software could convert a meaningful slice of integration-blocked lapsed trials, but stated intent is not a revenue forecast.

    Medium confidenceBase 93

    Among respondents whose main reason was no integration, 61.3% said fairly/very likely to pay £6/user/month if connected (base 93); overall, 37.6% said likely if connected (base 460).

  4. Cutting the trial to 14 days is not supported by the data and risks worsening the setup problem.

    High confidenceBase 460

    Only 0.2% said trial was too long, 3.7% forgot it was running, and 1.5% said too short (base 460). Setup time was 18.7% unweighted/21.6% weighted main reason.

  5. Price is a material secondary blocker, but the small-vs-large difference is not statistically significant; pricing changes need testing rather than assumption.

    High confidenceBase 460

    Too expensive was 15.9% overall (73/460), 19.0% in 1-9 (32/168) vs 15.5% in 50-249 (17/110); difference 3.5pp, SE ~4.6pp, not significant at 95%. 20.4% would need per-company pricing before paying.

  6. Approval workflow is a real but more concentrated blocker, especially in larger teams; custom approval chains are a credible product requirement.

    Medium confidenceBase 460

    Approval workflow didn't fit was 12.6% overall (58/460), 21.8% in 50-249 (24/110), and 22.0% would need custom approval chains before paying.

  7. The raw sample over-represents larger businesses versus the lapsed-trial population, so unweighted totals should not be used alone for prioritisation.

    High confidenceBase 460

    Sample is 36.5% 1-9, 39.6% 10-49, 23.9% 50-249; population is 55%, 35%, 10%. Weighting changes setup from 18.7% to 21.6% and Sage need from 23.7% to 18.1%.

Caveats

  • The survey covers only lapsed non-payers; it cannot directly estimate conversion among all trials, payers, or active prospects.
  • Tables are unweighted and the sample skews to larger firms than the 1,640 lapsed-trial population; weighted figures should drive prioritisation.
  • Q5 is a hypothetical willingness-to-pay question, not observed purchase; treat the 61.3% likely-if-integrated as upper-bound intent.
  • Small bases limit precision for FreeAgent (33), Other (17), segment cells, and the 109 open-text answers; open text is a self-selected subset.
  • Q2 forces a single main reason and Q3 allows multiple selections, so the survey does not measure the relative weight of overlapping blockers.
  • Sage demand may vary by Sage variant (Sage 50, Business Cloud, Sage 200); the survey does not separate integration scope or effort.
  • Completion was 28.0% (460/1,640), so non-response bias may affect all estimates.

Next steps

  • decisionKeep the 30-day trial for now and do not cut to 14 days.
  • experimentRun an assisted-setup/concierge onboarding pilot for new trials, prioritising 10-249 employee teams and trials with incomplete setup.
  • researchInterview 15-20 Sage users and no-integration respondents to scope the highest-demand Sage variant and validate willingness to pay.
  • experimentTest per-company or flat pricing for small teams alongside current per-user pricing.
  • monitorInstrument the trial setup funnel and reasons for non-conversion by software and company size.

Open-text themes it coded

Missing accounting-software integration 22Setup time / implementation burden 24Price / per-user cost 23Approval workflow mismatch 15Mobile app / receipt scanning reliability 10No need / low expense volume / overkill 12Chose competitor / other tool 8

The survey it planned

5 screening questions and 10 questions, as the model wrote them.

  1. S1

    Did you personally administer your team's Tallyroom trial — for example, setting up the account, inviting staff, or managing expenses?

    One answer
    • Yes
    • No

    Continues if Yes

  2. S2

    When did your team's Tallyroom trial end?

    One answer
    • Within the last 30 days
    • 31–60 days ago
    • 61–90 days ago
    • More than 90 days ago
    • I don't know / the trial has not ended

    Continues if Within the last 30 days; 31–60 days ago; 61–90 days ago

  3. S3

    How many employees does your team have in the UK?

    One answer
    • 1–9
    • 10–49
    • 50–249
    • 250 or more
    • I don't know

    Continues if 1–9; 10–49; 50–249

  4. S4

    Is your team currently paying for Tallyroom?

    One answer
    • Yes
    • No

    Continues if No

  5. S5

    Where is your team based?

    One answer
    • United Kingdom
    • Outside the United Kingdom

    Continues if United Kingdom

  6. Q1

    Which of the following, if any, contributed to your team not continuing with Tallyroom after the trial? Select all that apply.

    Any that apply
    • We did not have enough time to evaluate it during the trial
    • The trial ended before we were ready to decide
    • The trial was too long and we lost momentum
    • It was too expensive for our budget
    • It was missing an integration we needed
    • It did not have a feature we needed
    • Setup or data import took too much time or effort
    • We chose a different product
    • We no longer needed an expenses app
    • Other
    • None of these
  7. Q2

    How much do you agree or disagree with the following statement: 'Tallyroom was easy to set up for our team.'

    Scale

    Scale 1-5: Strongly disagree to Strongly agree

  8. Q3

    Which of the following would have made the biggest difference to your team choosing to pay for Tallyroom? Select up to 3.

    Any that apply
    • Integration with Sage
    • Integration with FreeAgent
    • Integration with Microsoft Dynamics 365 Business Central
    • Integration with Zoho Books
    • Integration with FreshBooks
    • Integration with another accounting system
    • Excel or CSV export
    • Better mobile receipt capture
    • Faster manager approvals
    • Custom approval workflows
    • Matching card spending to receipts
    • Better reporting
    • Assisted setup or data import
    • None of these
  9. Q4

    How important was the price of £6 per user per month in your team's decision not to continue with Tallyroom?

    Scale

    Scale 1-5: Not at all important to Extremely important

  10. Q5

    Thinking about the length of your Tallyroom trial, which statement best describes your experience?

    One answer
    • It was too short — we needed more time
    • It was about right
    • It was too long — we lost momentum
    • I don't remember / not sure
  11. Q6

    How many people in your team used Tallyroom during the trial?

    One answer
    • 1
    • 2–5
    • 6–10
    • 11–25
    • 26 or more
    • Not sure
  12. Q7

    How much do you agree or disagree with the following statement: 'Tallyroom gave us enough help to set up and import our data.'

    Scale

    Scale 1-5: Strongly disagree to Strongly agree

    Judges: double-barrelled

    One rating combines the adequacy of setup help and data-import help, which could differ. It also requires a rating of import help even from teams that did not need or attempt data import, with no not-applicable option.

    Asks respondents to agree/disagree with two things at once: 'set up' and 'import our data'.

  13. Q8

    If Tallyroom had cost £3 per user per month during your trial instead of £6, how likely would your team have been to continue?

    Scale

    Scale 1-5: Very unlikely to Very likely

  14. Q9

    What is the one thing that would have most likely made your team continue with Tallyroom?

    Open text
  15. Q10

    Which accounting software does your team currently use? Select all that apply.

    Any that apply
    • Xero
    • QuickBooks
    • Sage
    • FreeAgent
    • Microsoft Dynamics 365 Business Central
    • Zoho Books
    • FreshBooks
    • KashFlow
    • ClearBooks
    • None
    • Other
Sample plan

Invite the admin of every UK team whose Tallyroom trial ended in the last 90 days without paying, drawn from the product database. With about 1,900 trials per quarter and 18% paying, there are roughly 1,550 lapsed trials per quarter; a £10 voucher, an email invitation and two reminders should yield about 450 completes. Set quotas of 150 completes for each company-size band (1–9, 10–49, 50–249 employees); screen out 250+ employees, non-UK teams, current payers and non-admins. Important precision note: 450 completes gives roughly ±8 percentage points per band at 95% confidence. The requested ±6 percentage points per band requires about 267 completes per band, or about 801 completes in total. Recommend either increasing fielding to about 810 completes or agreeing with stakeholders to report ±8 percentage points per band.