Gemini 3.1 Pro Preview: Why Tallyroom trials don't convert

The decision: where to put next quarter's product and sales effort to lift trial-to-paid conversion: a new accounting integration, assisted setup, a cheaper plan, or a shorter trial. The brief and the data →

Verdict: ✗ Not sound
Research score: 63 of 100
Analysis rating: 810
Head to head, this task: won 7 of 10

Why it is not sound

The survey

Presumed the stakeholder's view instead of testing it
A research objective no question serves
Ignored the precision the brief asked for, which the sample cannot deliver
A leading question: Q1
A leading question: Q5
A leading question: Q8
A double-barrelled question: Q8

The analysis

Reported a material finding the data does not support
Missed a trap: Teams of 50-249 people are 22% of respondents but 10% of lapsed trials. They cite missing integration and use Sage far more, so unweighted figures overstate integration and Sage demand and understate setup time: unweighted, missing integration is the top reason; weighted, setup time is.
Missed a trap: 460 of 1,640 invited admins completed (about 28%), with a £10 voucher; those who respond may differ from those who do not.

Findings the judges found unsupported

Shortening the trial to 14 days will likely worsen conversion rates rather than help them.The evidence supports not cutting the trial outright, but does not establish that a 14-day trial would worsen conversion. Its effect would require a controlled experiment.
There is a strong, lucrative demand for a Sage integration, especially among larger SMEs.Demand for Sage connectivity is supported, but calling it lucrative is not established by hypothetical payment intentions. The 61.3% also covers all respondents citing missing integration, not specifically Sage users, and unweighted demand overrepresents larger businesses.
Setup time and complexity is the single biggest roadblock for the micro-businesses that make up our largest funnel segment.The extrapolated count is incorrect: 1,640 × 55% × 27.4% is approximately 247 teams, not 355. This remains an extrapolation from responding teams, rather than an observed count of lost teams.

Scorecard

Numbers 11 of 11 right

Question		Its answer	True
What percentage of respondents said they would need a Sage integration before paying?	✓	23.7%	23.7%
Weighted to the company-size mix of all lapsed trials, what percentage would need a Sage integration?	✓	18.1%	18.2%
Among businesses with 1-9 employees, what percentage gave 'didn't have time to set it up' as the main reason?	✓	27.4%	27.4%
How many of the open-text answers mention not having time to set Tallyroom up, or setup taking too long?	✓	24	24
What is the 95% margin of error, in percentage points, on the unweighted share of respondents needing a Sage integration?	✓	3.9 points	3.89 points
Is the difference in the share giving 'too expensive' as their main reason between businesses with 1-9 employees and those with 50-249 statistically significant at the 95% level? (1 for yes, 0 for no)	✓	No	No
Weighted to the company-size mix of all 1,640 lapsed trials, about how many teams' main reason was not having time to set it up?	✓	355	355
How many of the open-text answers from businesses with 1-9 employees mention not having time to set Tallyroom up, or setup taking too long?	✓	14	14
Weighted to the company-size mix of all lapsed trials, what percentage would give 'no integration with our accounting software' as their main reason?	✓	16.3%	16.3%
What percentage of the trial admins invited completed the survey?	✓	28%	28%
Among respondents whose main reason was no integration with their accounting software, what percentage said they would be likely (fairly or very) to pay if Tallyroom connected to it?	✓	61.3%	61.3%

Traps in the data 5 of 7 handled

✓Teams on 'Other' accounting software (base 17) mostly cite missing integration.
✗Teams of 50-249 people are 22% of respondents but 10% of lapsed trials. They cite missing integration and use Sage far more, so unweighted figures overstate integration and Sage demand and understate setup time: unweighted, missing integration is the top reason; weighted, setup time is.
✓Q2 is multi-select: percentages are of respondents and add up to more than 100%.
✓Likelihood to pay is a hypothetical stated intention.
✓The head of sales believes the trial is too long and teams forget about it.
✗460 of 1,640 invited admins completed (about 28%), with a £10 voucher; those who respond may differ from those who do not.
✓Only teams that did not pay were surveyed, so the survey cannot show what distinguishes them from teams that did; paying teams may also have struggled with setup.

The analysis

Recommendation

Direct product effort towards building a Sage integration to convert larger, higher-value accounts. Direct sales effort towards assisted setup, as time-to-setup is the leading barrier for the small businesses that make up 55% of the pipeline. Do not shorten the 30-day trial.

What would change it

Evidence that the unit economics of assisted onboarding for 1-9 employee businesses are negative due to their low LTV, or technical discovery showing that a Sage integration is prohibitively expensive to build.

Findings

Shortening the trial to 14 days will likely worsen conversion rates rather than help them.

High confidenceBase 460

Only 3.7% forgot the trial was running and 0.2% found it too long, whereas 18.7% overall (and 27.4% of 1-9 employee teams) cited not having enough time to set it up as their main barrier to paying.
There is a strong, lucrative demand for a Sage integration, especially among larger SMEs.

High confidenceBase 460

23.7% of all respondents requested it (rising to 40.9% for 50-249 employee companies). Among teams leaving due to lack of accounting integration, 61.3% report they would be likely to pay £6/user/month if connected.
Setup time and complexity is the single biggest roadblock for the micro-businesses that make up our largest funnel segment.

High confidenceBase 168

1-9 employee businesses make up 55% of all lapsed trials, and 27.4% of them failed to convert simply because they didn't have time to complete setup. This represents roughly 355 lost teams over the quarter.
Our strict per-user pricing model is pushing away teams that have many employees but few regular claimants.

Medium confidenceBase 460

20.4% need per-company (flat-rate) pricing rather than per-user, and 15.9% overall left because it was too expensive. Open text confirms the multiplier effect blocks sign-off for mid-sized firms.

Caveats

The survey data is unweighted; large companies (50-249 employees) are overrepresented in the responses (23.9%) compared to the actual population of lapsed trials (10%).
Self-reported willingness to pay (£6/user/month for Sage connectivity) often overestimates actual buying behavior in reality.
This research captures lapsed users only; we do not know the needs and barriers of successful paying customers, though they may share similar pain points.

Next steps

decisionMaintain the trial period at 30 days.
researchStart technical discovery and scoping for a Sage integration.
experimentRun an assisted setup pilot for a cohort of 1-9 employee businesses.
researchInvestigate alternative pricing tiers, such as active-user billing or flat per-company rates.

Open-text themes it coded

Setup takes too long or lacked time to configure 24Missing Sage integration 15Per-user pricing is unsuitable or too expensive 23Approval workflow didn't fit internal processes 15Mobile app or receipt scanning failures 10Decided expenses software isn't needed 12

The survey it planned

2 screening questions and 8 questions, as the model wrote them.

SQ1
Were you involved in evaluating the Tallyroom free trial for your business?
One answer
- Yes, I was the primary decision maker
- Yes, I was part of the evaluation team
- No, I was not involved in the evaluation
Continues if Yes, I was the primary decision maker; Yes, I was part of the evaluation team
SQ2
How many employees does your company currently have?
One answer
- 1-9 employees
- 10-49 employees
- 50-249 employees
- 250 or more employees
Continues if 1-9 employees; 10-49 employees; 50-249 employees
Q1
Which of the following were reasons your company decided not to subscribe to Tallyroom after the free trial? (Select all that apply)
Any that apply
- The £6 per user monthly price was too high
- It didn't integrate with our accounting software
- It lacked specific features we needed
- It was too difficult or confusing to set up
- We forgot about the trial or lost momentum before it ended
- We chose a competing software instead
- We decided we don't need an expenses app right now
Judges: leading
The closed list has no way to report an unlisted reason, steering responses towards the supplied explanations.
The trial-length options are lopsided. 'We forgot about the trial or lost momentum' fits the head of sales' view, but there is no matching option such as 'the trial was too short to evaluate properly' and no 'Other'. Answers are pushed towards the too-long story.
Q2
You mentioned the app was difficult to set up. If we had offered a free 1-on-1 assisted setup call with an expert, would you have been more likely to subscribe?
One answer
- Yes, definitely
- Yes, probably
- No, it wouldn't have made a difference
- Not sure
Routing Shown only if Q1 is: It was too difficult or confusing to set up
Q3
Which accounting software does your business currently use to manage its finances?
One answer
- Xero
- QuickBooks
- Sage
- FreeAgent
- KashFlow
- Other
- We do not use accounting software
Q4
You mentioned Tallyroom lacked specific features. What exact features or capabilities were missing that your team needed?
Open text

Routing Shown only if Q1 is: It lacked specific features we needed
Q5
How would you rate the length of the 30-day free trial for evaluating Tallyroom?
Scale

Scale 1-5: Far too short (didn't have time to properly test) to Far too long (we lost momentum and forgot about it)

Judges: leading
The long-trial endpoint embeds the stakeholder's proposed explanation—losing momentum and forgetting—rather than measuring trial length alone.
Adding specific reasons into the scale endpoints (like 'we lost momentum and forgot about it') leads the respondent towards the stakeholder's hypothesis and assumes their reasoning.
Q6
If the free trial had been 14 days instead of 30 days, how would this have impacted your team's evaluation?
One answer
- It would have been better (created more urgency and focus to test)
- It would not have changed our evaluation process
- It would have been worse (not enough time to test it properly)
Q7
If Tallyroom offered a cheaper basic plan with only core receipt snapping and approval capabilities (removing advanced features), how interested would your business be?
One answer
- Very interested - we would likely subscribe
- Somewhat interested
- Not interested - price wasn't our main issue
- Not interested - we need the advanced features
Judges: assumptive
The negative options assume disinterest must reflect either price not being the main issue or needing advanced features, excluding other reasons. 'Cheaper' gives no price, so respondents can evaluate materially different offers.
The 'Not interested' options force respondents to adopt a specific reason for their lack of interest, assuming one of these two reasons applies to everyone.
Q8
Please rank the following potential improvements based on how likely they would be to convince you to subscribe to Tallyroom today (1 = Most likely to convince you, 4 = Least likely).
Ranking
- A cheaper basic pricing plan
- Direct integration with your specific accounting software
- A guided 1-on-1 setup and onboarding call
- A shorter, more structured 14-day trial
Judges: leading, double-barrelled
The trial option bundles a shorter duration with greater structure and presents both as an improvement, so its ranking cannot isolate support for shortening. The accounting option assumes respondents use accounting software despite Q3 allowing otherwise. The cheaper plan's price remains unspecified.
'A shorter, more structured 14-day trial' combines two changes in one option (shorter and more structured). Adding 'more structured' makes the shorter trial sound more appealing than the other options, which are described plainly. This steers rankings towards the stakeholder's preferred option.

Sample plan

Send email invitations to the product database of admins whose Tallyroom trials expired without payment in the last 90 days. Offer a £10 voucher incentive. We need 450 total completes. To achieve the requested precision (within +/- 6 percentage points at 95% confidence per company size band), implement strict quotas of exactly 150 completes for each of the three company-size bands (1-9, 10-49, 50-249). Given the finite population of ~1,550 lapsed trials per quarter, 150 completes per segment yields a margin of error of approximately 6.5%.