Benchmarks / SurveyBench / Why Tallyroom trials don't convert / Gemini 4 Argon
Gemini 4 Argon: Why Tallyroom trials don't convert
The decision: where to put next quarter's product and sales effort to lift trial-to-paid conversion: a new accounting integration, assisted setup, a cheaper plan, or a shorter trial. The brief and the data →
- Verdict
- ✗ Not sound
- Research score
- 83 of 100
- Analysis rating
- 985
- Head to head, this task
- won 22 of 27
Why it is not sound
The survey
- A double-barrelled question: Q2
- A double-barrelled question: Q3
- A double-barrelled question: Q6
- A double-barrelled question: Q7
- A double-barrelled question: Q9
The analysis
- Missed a trap: Only teams that did not pay were surveyed, so the survey cannot show what distinguishes them from teams that did; paying teams may also have struggled with setup.
Scorecard
Numbers 11 of 11 right
| Question | Its answer | True | |
|---|---|---|---|
| What percentage of respondents said they would need a Sage integration before paying? | ✓ | 23.7% | 23.7% |
| Weighted to the company-size mix of all lapsed trials, what percentage would need a Sage integration? | ✓ | 18.1% | 18.2% |
| Among businesses with 1-9 employees, what percentage gave 'didn't have time to set it up' as the main reason? | ✓ | 27.4% | 27.4% |
| How many of the open-text answers mention not having time to set Tallyroom up, or setup taking too long? | ✓ | 21 | 24 |
| What is the 95% margin of error, in percentage points, on the unweighted share of respondents needing a Sage integration? | ✓ | 3.9 points | 3.89 points |
| Is the difference in the share giving 'too expensive' as their main reason between businesses with 1-9 employees and those with 50-249 statistically significant at the 95% level? (1 for yes, 0 for no) | ✓ | No | No |
| Weighted to the company-size mix of all 1,640 lapsed trials, about how many teams' main reason was not having time to set it up? | ✓ | 355 | 355 |
| How many of the open-text answers from businesses with 1-9 employees mention not having time to set Tallyroom up, or setup taking too long? | ✓ | 12 | 14 |
| Weighted to the company-size mix of all lapsed trials, what percentage would give 'no integration with our accounting software' as their main reason? | ✓ | 16.3% | 16.3% |
| What percentage of the trial admins invited completed the survey? | ✓ | 28% | 28% |
| Among respondents whose main reason was no integration with their accounting software, what percentage said they would be likely (fairly or very) to pay if Tallyroom connected to it? | ✓ | 61.3% | 61.3% |
Traps in the data 6 of 7 handled
- ✓Teams on 'Other' accounting software (base 17) mostly cite missing integration.
- ✓Teams of 50-249 people are 22% of respondents but 10% of lapsed trials. They cite missing integration and use Sage far more, so unweighted figures overstate integration and Sage demand and understate setup time: unweighted, missing integration is the top reason; weighted, setup time is.
- ✓Q2 is multi-select: percentages are of respondents and add up to more than 100%.
- ✓Likelihood to pay is a hypothetical stated intention.
- ✓The head of sales believes the trial is too long and teams forget about it.
- ✓460 of 1,640 invited admins completed (about 28%), with a £10 voucher; those who respond may differ from those who do not.
- ✗Only teams that did not pay were surveyed, so the survey cannot show what distinguishes them from teams that did; paying teams may also have struggled with setup.
The analysis
Prioritise Sales effort on Assisted Setup (coupled with onboarding simplification) for next quarter to capture immediate conversion gains among Xero and QuickBooks trials, while Product scopes and builds a Sage Business Cloud integration alongside targeted approval-workflow fixes. Do NOT shorten the 30-day trial, and test switching from all-employee pricing to 'active-user' or tiered company pricing rather than cutting the headline £6 rate.
- If database audit shows that >65% of Sage trialists are on legacy desktop Sage 50/200 rather than Sage Business Cloud, pause the Sage integration (as engineering effort per converted team will be too high) and redirect Product effort entirely to self-serve onboarding and flexible approval workflows. 2) If trial telemetry shows that 'Didn't have time to set it up' trialists actually logged in for less than 2 minutes total (indicating polite brush-off/low intent rather than genuine setup friction), scale back high-touch assisted setup and focus on self-serve templates.
Findings
-
To the Head of Sales: Do not cut the trial from 30 to 14 days. Teams are not forgetting the trial or finding it too long; rather, small teams are running out of time to complete setup amidst their daily workload. A 14-day trial would actively hurt conversion.
Only 0.2% of respondents said the trial was too long and 3.7% forgot it was running (base 460). Conversely, 'Didn't have time to set it up' is the #1 reason for lapsing once weighted to the actual population mix (21.6%, or ~355 of the 1,640 lapsed teams, and 27.4% among 1-9 employee businesses, base 168). Open-text responses (21 mentions) show single-person admins struggle to find an uninterrupted block of time during the month to configure categories, approval rules, and users.
-
To the Head of Product: Demand for a Sage integration is substantial, though raw survey figures overstate it because larger firms were more likely to respond. Even after correcting for this skew, Sage is the single largest integration gap, and 'Sage' actually means at least three distinct products (Sage Business Cloud, Sage 50, Sage 200).
Unweighted, 23.7% (±3.9 pp, base 460) say they need a Sage integration before paying, and 85.5% of the 117 Sage users want it. However, 50-249 employee firms are over-represented in the sample (23.9% vs 10.0% in the database). Weighted to the true population by company size, Sage demand is 18.1% (~297 lapsed teams), and 'no integration' as the main blocker drops from 20.2% to 16.3% (~267 teams). Open text reveals fragmented requirements across Sage Business Cloud/Accounting (T334, T339, T433), Sage 50 (T413), and Sage 200 (T241).
-
To the Finance Director: Connecting to their accounting software will not automatically convert the 61.3% who said they would be likely to pay. Stated willingness-to-pay is heavily inflated, and secondary blockers—specifically rigid approval workflows and per-user pricing for infrequent claimants—will stop many integrated teams from converting.
Among the 93 respondents whose main reason for lapsing was lack of integration, 61.3% scored 4 or 5 on likelihood to pay £6/user/month (25.8% 'very likely', 35.5% 'fairly likely'). However, among respondents who ALREADY have an integration (Xero and QuickBooks, base 293), teams still lapsed due to setup time (24.7% Xero, 22.5% QB), price (18.1% Xero, 21.6% QB), and approval workflows (14.3% Xero, 15.3% QB). Furthermore, among Sage users (base 117), 28.2% also say they need custom approval chains and 20.5% need per-company pricing before paying.
-
Pricing resistance is driven more by the 'per-seat' model than the £6 headline figure, and affects all company sizes equally. Teams resent paying for every employee when only a few submit expenses or headcount fluctuates.
15.9% overall gave 'too expensive' as their main reason (base 460), with no statistically significant difference between 1-9 employee firms (19.0%, base 168) and 50-249 employee firms (15.5%, base 110; p = 0.44). In Q3, 20.4% requested 'per-company pricing (not per user)', rising to 26.1% among QuickBooks users (base 111). Open text confirms the friction is paying for non-claiming seats (T059, T208, T322, T402) or seasonal staff (T355).
-
Rigid approval workflows are a major hidden blocker across all segments, particularly for larger firms, because Tallyroom forces a single, claim-by-claim, in-app line-manager sign-off.
12.6% cited approval workflows as the main blocker (rising to 21.8% in 50-249 employee firms, base 110), and 22.0% overall (27.3% in 50-249 firms) need custom approval chains before paying (base 460). Open text (14 mentions) details five specific missing capabilities: skipping middle management in flat teams (T010), batch/monthly sign-off (T161, T300), routing by expense type or value threshold with dual sign-off (T180, T392), sequential office-manager-then-owner or external accountant review (T247, T455), and email/paper-friendly sign-off (T077, T117, T216).
Caveats
- Severe sample skew by company size: 50-249 employee businesses make up 23.9% of respondents vs 10.0% of the lapsed-trial population, while 1-9 employee businesses are under-represented (36.5% vs 55.0%). Unweighted totals overstate enterprise blockers (Sage, approval workflows) and understate small-business blockers (setup time, FreeAgent).
- Data integrity discrepancies in Q6 vs Q1: Several respondents classified as '50-249 employees' in the survey tables explicitly state in open text that they have far fewer staff (e.g., T010 says 'nine people', T059 and T322 say '25', T087 says 'team of five', T164 says 'four-person', T343 says '30 staff', while T023 and T449 are in '1-9 employees' but mention 15 and 18 staff). Self-reported Q6 company size is noisy and should be cross-checked against database records.
- Stated-intent inflation on Q5: 61.3% of non-integrated users say they are 'fairly' or 'very' likely to pay £6/user/month if integrated. In B2B research, top-two-box stated intent routinely overstates actual conversion by 2x to 3x, and even respondents who blamed 'too expensive' (23.3%) or 'didn't have time to set up' (27.9%) claimed they would likely pay in Q5.
- Non-response bias: While a 28.0% completion rate (460/1,640) is strong, the £10 voucher and email outreach likely attracted more engaged admins or those with a specific grievance (like a missing integration) compared to teams who simply ghosted after signing up.
- Sage product fragmentation: The survey asked about 'Sage' as a single option, but open-text responses reveal users are split between Sage Business Cloud/Accounting (cloud API), Sage 50 (often desktop/hybrid), and Sage 200 (mid-market ERP). A single integration will not capture the entire Sage segment.
Next steps
- experimentKeep the trial at 30 days and run an A/B test offering a 15-minute 'Done-With-You' assisted setup call (plus pre-built category/approval templates) to new Xero and QuickBooks trials.
- researchSurvey or audit the 117 Sage respondents (and tag Sage variants at signup) to quantify the exact split between Sage Business Cloud, Sage 50, and Sage 200 before committing engineering resources.
- experimentTest an 'active claimant' or tiered flat-band pricing package against the £6/seat/month all-employee model on the pricing page and in sales recovery outreach.
- decisionScope and ship quick-win approval workflow flexibility (optional single-stage/direct-to-MD sign-off, batch approval view, and sequential 2-step approval).
- monitorInvestigate and fix mobile receipt upload failures on 4G/Android and OCR cropping/decimal errors.
Open-text themes it coded
Missing accounting integration (Sage, FreeAgent, or other packages) 22Lack of time to complete setup or overly complex configuration 24Price too high or per-user pricing model misfits teams with few/seasonal claimants 23Approval workflow too rigid (single approver, claim-by-claim, in-app only) 15Low expense volume or company cards make dedicated software unnecessary 12Mobile app reliability and OCR/receipt scanning failures 10Chose a competitor or card-based expense solution 8
The survey it planned
2 screening questions and 10 questions, as the model wrote them.
-
S1
How were you involved in your team's recent trial of Tallyroom?
One answer- I set up the trial or helped decide whether to buy Tallyroom
- I tried Tallyroom, but someone else decided whether to buy it
- I never used Tallyroom or don't remember the trial
Continues if I set up the trial or helped decide whether to buy Tallyroom
-
S2
How many employees work at your organisation in total (including yourself)?
One answer- 1 to 9 employees
- 10 to 49 employees
- 50 to 249 employees
- 250 or more employees
Continues if 1 to 9 employees; 10 to 49 employees; 50 to 249 employees
-
Q1
How far did your team get during your 30-day Tallyroom trial? Select all that apply.
Any that apply- Created an account, but didn't configure settings or test it
- Connected our accounting software (Xero or QuickBooks)
- Uploaded or photographed test receipts
- Invited staff members or managers to try it
- Approved real expenses and sent them to our accounting software
-
Q2
Why did your team decide not to buy a Tallyroom subscription at the end of the trial? Select up to 3 reasons.
Any that apply- £6 per user per month was too expensive for us
- It did not integrate with our accounting software
- It was missing a specific feature we needed
- We didn't have time or needed help to set it up properly
- We ran out of time to properly test it with the team
- We set it up, got busy with other things, and forgot about the trial
- Staff or managers found the app too difficult to use
- We chose a different expenses app
- We decided to stick with spreadsheets or our current process
- Other reason
Judges: double-barrelledIt assumes the team made a decision not to buy. The invitation also includes trials that simply lapsed without a purchase decision.
The option 'We didn't have time or needed help to set it up properly' joins two different barriers in one answer: lack of time and needing assistance. This muddies the evidence for the assisted-setup option.
-
Q3
Of the reasons you selected, which single factor was the biggest barrier to subscribing?
One answer- £6 per user per month was too expensive for us
- It did not integrate with our accounting software
- It was missing a specific feature we needed
- We didn't have time or needed help to set it up properly
- We ran out of time to properly test it with the team
- We set it up, got busy with other things, and forgot about the trial
- Staff or managers found the app too difficult to use
- We chose a different expenses app
- We decided to stick with spreadsheets or our current process
- Other reason
Judges: double-barrelledIt repeats the same combined option, 'didn't have time or needed help to set it up', so the biggest barrier cannot separate time from needing help.
Inherits the double-barrelled options from Q2.
-
Q4
Which accounting software does your business primarily use?
One answer- Xero
- QuickBooks
- Sage (e.g., Sage Business Cloud, Sage 50)
- FreeAgent
- Zoho Books
- FreshBooks
- Clear Books
- KashFlow
- NetSuite
- Spreadsheets / None
- Other
-
Q5
What specific feature or integration was Tallyroom missing that you needed?
Open textRouting Shown only if Q2 is: It did not integrate with our accounting software; It was missing a specific feature we needed
-
Q6
Thinking about the 30-day length of the free trial, which statement best matches your experience?
One answer- 30 days was too short — we needed more time to test it across an expense cycle
- 30 days was about the right amount of time
- 30 days was too long — having a whole month reduced our urgency and we lost momentum
- The length of the trial made no difference to our decision
Judges: double-barrelledThe too-short and too-long answers combine a judgement about duration with a particular explanation. A respondent could think the trial was too long without having lost urgency or momentum, or too short for reasons unrelated to an expense cycle.
The options attach specific reasons to sentiments (e.g., 'too long — having a whole month reduced our urgency...'), forcing respondents to agree with both the feeling and the specific justification.
-
Q7
If Tallyroom had offered a 14-day trial instead of 30 days, how would that have affected your team?
One answer- It would have focused our attention and made us MORE likely to finish testing and subscribe
- It would not havegiven us enough time, making us LESS likely to subscribe
- It would have made no difference to whether we subscribed
Judges: double-barrelledThe positive answer combines finishing testing and subscribing. A shorter trial could increase the likelihood of completing testing without increasing the likelihood of subscribing.
Like Q6, the options attach specific rationales ('focused our attention', 'not have given us enough time') to the answers, meaning respondents cannot express a preference without also agreeing to the provided reason.
-
Q8
How do you feel about Tallyroom's price of £6 per user per month?
One answer- It is good value for money
- It is acceptable, if the product meets all our needs
- It is too expensive per user for our budget
- We prefer a flat monthly fee rather than paying per user
- I did not look at the pricing
-
Q9
Which ONE of the following would have been most likely to convince your team to become a paying Tallyroom customer?
One answer- Direct integration with our accounting software
- A free 1-to-1 setup call with a specialist to configure our account and train staff
- A lower-cost plan (e.g., £3 per user per month, or a flat fee for infrequent claimers)
- A longer trial period to finish testing with our team
- A shorter trial period with guided milestones to keep us on track
- A specific product feature that was missing
- None of these — we do not need an expenses app right now
Judges: double-barrelledThe shorter-trial option bundles a shorter duration with guided milestones. Selecting it cannot distinguish demand for a shorter trial from demand for guidance.
Two options each combine separate offers. 'A shorter trial period with guided milestones' bundles trial length with guidance, so it cannot isolate support for a shorter trial. 'A lower-cost plan (£3 per user or a flat fee)' bundles a lower per-user price with a different pricing model.
-
Q10
How is your team managing staff expenses now?
One answer- Paper receipts and spreadsheets
- Built-in expense tools inside our accounting software (e.g., Xero Expenses, QuickBooks)
- Another dedicated expenses or receipt app (e.g., Dext, Pleo, Expensify, Soldo)
- Company cards without a receipt-capture app
- We don't have a formal process
Sample plan
POPULATION & INVITATION LIST: Invite all ~1,558 UK trial admins whose 30-day trial ended without converting in the last 90 days (82% of ~1,900 quarterly trials), extracted from the product database with £10 voucher incentive. PRECISION & SAMPLE REALITY: Achieving ±6 percentage points at 95% confidence (p=0.5) within each of the 3 employee-size bands requires ~267 completes per band (~801 total)—or ~171–215 per band with finite population correction—which exceeds both the ~450 complete budget and the likely population of lapsed 50–249 employee trials in a single 90-day window. QUOTA & SAMPLING STRATEGY: Set a target of 450 completes divided into equal quotas of n=150 per size band (1–9, 10–49, 50–249 employees). At n=150 per band, the maximum margin of error is ±8.0% at 95% confidence for a 50% proportion (and passes the ±6.0% threshold for proportions ≤18% or ≥82%, or after applying finite population correction in smaller bands). To maximise completes in the harder-to-fill 10–49 and 50–249 bands, invite 100% of lapsed trials in those bands first with 2 reminder emails; if the 90-day pool in 50–249 is exhausted below 150 completes, extend the lookback window to 180 days for 50–249 employee teams (flagging cohort age in analysis) and link survey responses to database records (signup date, login count, receipts uploaded) to check for non-response bias.