Benchmarks / SurveyBench / Why Tallyroom trials don't convert / Claude Fable 5.2
Claude Fable 5.2: Why Tallyroom trials don't convert
The decision: where to put next quarter's product and sales effort to lift trial-to-paid conversion: a new accounting integration, assisted setup, a cheaper plan, or a shorter trial. The brief and the data →
- Verdict
- ✗ Not sound
- Research score
- 99 of 100
- Analysis rating
- 1,683
- Head to head, this task
- won 7 of 10
Why it is not sound
The survey
- A leading question: Q10
Scorecard
Numbers 11 of 11 right
| Question | Its answer | True | |
|---|---|---|---|
| What percentage of respondents said they would need a Sage integration before paying? | ✓ | 23.7% | 23.7% |
| Weighted to the company-size mix of all lapsed trials, what percentage would need a Sage integration? | ✓ | 18.1% | 18.2% |
| Among businesses with 1-9 employees, what percentage gave 'didn't have time to set it up' as the main reason? | ✓ | 27.4% | 27.4% |
| How many of the open-text answers mention not having time to set Tallyroom up, or setup taking too long? | ✓ | 24 | 24 |
| What is the 95% margin of error, in percentage points, on the unweighted share of respondents needing a Sage integration? | ✓ | 3.9 points | 3.89 points |
| Is the difference in the share giving 'too expensive' as their main reason between businesses with 1-9 employees and those with 50-249 statistically significant at the 95% level? (1 for yes, 0 for no) | ✓ | No | No |
| Weighted to the company-size mix of all 1,640 lapsed trials, about how many teams' main reason was not having time to set it up? | ✓ | 355 | 355 |
| How many of the open-text answers from businesses with 1-9 employees mention not having time to set Tallyroom up, or setup taking too long? | ✓ | 14 | 14 |
| Weighted to the company-size mix of all lapsed trials, what percentage would give 'no integration with our accounting software' as their main reason? | ✓ | 16.3% | 16.3% |
| What percentage of the trial admins invited completed the survey? | ✓ | 28% | 28% |
| Among respondents whose main reason was no integration with their accounting software, what percentage said they would be likely (fairly or very) to pay if Tallyroom connected to it? | ✓ | 61.3% | 61.3% |
Traps in the data 7 of 7 handled
- ✓Teams on 'Other' accounting software (base 17) mostly cite missing integration.
- ✓Teams of 50-249 people are 22% of respondents but 10% of lapsed trials. They cite missing integration and use Sage far more, so unweighted figures overstate integration and Sage demand and understate setup time: unweighted, missing integration is the top reason; weighted, setup time is.
- ✓Q2 is multi-select: percentages are of respondents and add up to more than 100%.
- ✓Likelihood to pay is a hypothetical stated intention.
- ✓The head of sales believes the trial is too long and teams forget about it.
- ✓460 of 1,640 invited admins completed (about 28%), with a £10 voucher; those who respond may differ from those who do not.
- ✓Only teams that did not pay were surveyed, so the survey cannot show what distinguishes them from teams that did; paying teams may also have struggled with setup.
The analysis
Keep the 30-day trial, and put next quarter's sales and customer-success effort into assisted setup. Run it as a controlled pilot for new trials on Xero and QuickBooks, with setup nudges early in the trial. It targets the largest weighted loss (about 22% of lapsed trials, roughly 355 teams per 90 days), needs no build, and can show a result within the quarter. Put product effort into Sage, but in two stages. First, spend 2-3 weeks finding out which Sage product(s) the demand sits in and whether lapsed Sage teams will make a real commitment. Then build the dominant variant if that holds. The demand is credible, but the revenue case rests on stated intent from 93 people and should be planned at 30-60 recovered teams per cohort, not 115. Do not launch a cheaper per-user plan; instead test whether a flat or active-user pricing structure removes the objection. Before any of these numbers go into a business case, re-run the size weighting on database company size.
Assisted setup would lose priority if the pilot shows no conversion lift over control, or if product analytics show most 'no time' teams never returned after day one, which would mean disengagement rather than setup effort. A shorter trial would come back into play if those analytics show activity collapsing after week two, or if a trial-length A/B shows 14 days converting at least as well. Sage should move from 'validate then build' to 'build now' if a majority of contacted Sage teams pre-commit and one Sage product covers most of them. It should be dropped or deferred if pre-commitment is under about 20% or demand splits evenly across three or four Sage products. If re-matching to database size materially changes the weights, the ranking of setup against integration should be revisited. A pricing change would move up if a flat or active-user offer converts price-objectors without reducing revenue per account.
Findings
-
Head of sales: do not cut the trial to 14 days. The survey gives no support for 'too long, teams forget', and the largest group of lost teams ran out of time rather than interest.
As main reason, 1 of 460 (0.2%) chose 'trial too long' and 17 (3.7%) 'forgot the trial was running', against 86 (18.7%) 'didn't have time to set it up' and 7 (1.5%) 'trial too short'. In the open text nobody mentions forgetting or excess length; 24 of 109 describe setup time, and 9 of those say the trial ended before setup was finished (T022, T073, T131, T146, T263, T340, T371, T431, T460). Forgetting is probably under-reported, since people who forgot are less likely to answer a survey, but even tripled it would sit below setup time.
-
Setup time is the largest addressable loss, and it sits mainly with small teams on software we already connect to. Assisted setup is the only one of the four options that needs no build and no price cut.
18.7% unweighted (86 of 460). Weighted to the lapsed-trial size mix it is 21.6%, about 355 of 1,640 teams, ahead of price (16.6%) and missing integration (16.3%). It is 27.4% among 1-9 employees (base 168) against 10.0% among 50-249 (base 110), a significant gap. About 70 of the 86 are Xero or QuickBooks users (24.7% of 182; 22.5% of 111). This is a stated reason; whether help with setup actually converts these teams is untested.
-
Head of product: Sage demand is real but concentrated, and smaller than the headline figure. About 1 in 4 respondents, or about 1 in 5.5 of all lapsed trials once weighted, say they need it. Almost all of that demand is existing Sage users, and for a majority of them it was the deal-breaker.
23.7% need Sage (109 of 460, ±3.9 points); weighted to the population size mix, 18.1%, about 300 of 1,640 teams. Among Sage users (base 117), 85.5% need it and 58.1% (68) gave missing integration as the main reason. It is the most specific unprompted theme in the open text, with 15 of 109 answers. Two limits apply. First, 42% of Sage users gave a different main reason and 28.2% also want custom approval chains, so integration is necessary but not always sufficient. Second, respondents name Sage 50, Sage 200, Sage Business Cloud and Sage Accounting, so 'a Sage integration' may be several builds.
-
Finance director: the survey cannot say how many will pay. It gives a ceiling. Six in ten of those blocked by integration say they would be likely to pay £6 per user, and only a quarter say 'very likely'.
Among the 93 whose main reason was no integration, 61.3% are fairly or very likely (57 people, roughly ±10 points), 25.8% very likely (24) and 25.8% unlikely even with the integration. This group mixes Sage (68), FreeAgent (11) and other software (14). My rough sizing: about 267 integration-blocked teams per 90-day cohort (16.3% of 1,640), of which roughly 185-195 are Sage. Stated intent gives a ceiling of about 115 Sage teams. Counting only 'very likely' gives about 50, and I would plan on 30-60 until real commitments are tested.
-
Do not quote the 37.6% 'likely to pay' total from Q5. The question was put to everyone, including Xero and QuickBooks users, for whom Tallyroom already connects and who still did not pay.
The Q5 base is all 460, of whom 63.7% (293) use Xero or QuickBooks, so the premise 'if Tallyroom connected to your software' is already true for them. Only the 'no integration' column (base 93) is interpretable. The table also gives no split by software, so Sage-specific intent cannot be isolated.
-
A cheaper plan is weakly supported. Price is the third reason and does not vary by size, and the complaint is more about paying per user for people who rarely claim than about the £6 level.
'Too expensive' is 15.9% (73 of 460), 16.6% weighted. It is 19.0% among 1-9 against 15.5% among 50-249, which is not significant (z about 0.8). 20.4% (94) say they need per-company pricing. Of 23 open-text price comments, about 13 object to the per-user or per-seat model and several say a flat fee would have been fine (T343, T402, T059, T208, T322, T355). A lower per-user price would also cut revenue from teams that pay today.
-
Approval-workflow fit is a meaningful loss that none of the four options addresses, and it appears strongest in larger teams. The needs are varied, so 'custom approval chains' would fix only some.
12.6% main reason (58 of 460); 21.8% among 50-249 (24 of 110) against 7.1% among 1-9. 22.0% (101) say they need custom approval chains. The 15 open-text answers split between multi-step or conditional approvers (T180, T392, T455, T247, T010), batch sign-off (T161, T300) and sign-off outside the app by email, paper or verbally (T013, T077, T117, T216). The size pattern depends on a company-size variable that looks unreliable.
-
About 1 in 5 lost trials are not winnable by any of the four options: they decided they did not need an expenses tool or chose a competitor.
'Decided we didn't need it' is 10.0% (46) and 'chose another tool' 8.7% (40), together 18.7% of 460. The open text has 12 no-need answers, typically very small claim volumes or a company card, and 8 naming competitors (Pleo twice, Dext, Rydoo, Soldo, Expensify, Webexpenses, Emburse). This is a lead-qualification issue rather than a product one.
-
FreeAgent and other packages are a secondary long tail. FreeAgent demand skews to the smallest firms, and the bases are too small to size it.
7.2% need FreeAgent (33 of 460), about 9.1% weighted. Among FreeAgent users (base 33), 72.7% need it and 33.3% (11) cite integration as the main reason. Users of other packages (base 17): 82.4% (14) cite integration, spread across KashFlow, Clear Books and others.
Caveats
- The company-size answers (Q6) look unreliable. Of 12 open-text answers that state a headcount, 11 contradict their size band (e.g. T010 '50-249' says 'only nine people'; T087 '50-249' says 'team of five'; T449 '1-9' says 'all 18 of us'). Every size cut and every weighted figure here, including 18.1% Sage, 16.3% integration and about 355 setup teams, rests on this variable and should be re-run on database size.
- The sample is not the population. The response rate was 28% (460 of 1,640) with a £10 incentive, and by self-report it skews large (1-9: 36.5% against 55%; 50-249: 23.9% against 10%). Tables are unweighted, so Sage and integration figures, which rise with size, overstate the population. Disengaged teams, including those who forgot, are probably under-represented.
- Q5 is hypothetical stated intent at a named price, and for 64% of respondents the premise is already true. Treat 61.3% as a ceiling on conversion, not a forecast, and ignore the 37.6% total.
- Q3 offered only product features. It had no option for setup help or anything about the trial, so 'need' figures favour integrations and cannot be compared with assisted setup. The aided list also inflates agreement.
- Q2 forces a single reason and is a recollection after the event. Several open-text answers give two reasons, such as Sage plus price or setup plus app failures.
- The survey covers only lapsed trials. With no paying comparison group and no count of total trials, it cannot show what distinguishes teams that pay or convert any finding into percentage points of trial-to-paid. Sage users who never started a trial are also invisible, so total Sage demand may be understated.
- Small bases: FreeAgent 33, other software 17, mobile app problems 29, and 93 for the key Q5 column (about ±10 points). The 1-9 against 50-249 price difference is not significant.
- The open text is 109 of 460 answers and it is unclear how these were selected. Use it for the nature of the problems, not their proportions.
- 'Sage' is several products (Sage 50, Sage 200, Sage Business Cloud/Accounting all appear). The survey did not ask which, so one build may cover only part of the demand.
Next steps
- decisionKeep the 30-day trial this quarter; add setup prompts and trial-ending reminders in the first two weeks instead of shortening it.
- experimentPilot assisted setup: randomly offer half of new Xero and QuickBooks trials a booked 30-minute onboarding call or done-for-you configuration, prioritising smaller teams.
- researchRe-match the 460 respondents to database company size and connected software, then re-weight the key figures.
- researchValidate Sage before committing the build: contact the roughly 100 Sage respondents and other lapsed Sage trials, record which Sage product they use, and offer a waitlist with a concrete commitment such as an agreed start date or card on file.
- monitorCapture accounting software at trial sign-up and report trial-to-paid by software, size and setup milestone each month.
- experimentTest pricing structure with price-objecting lapsed teams: a flat small-team tier or charging only for active claimants, against £6 per user.
- researchInterview 10-12 teams lost on approvals, focused on 10+ employee firms, to separate multi-step, batch and offline sign-off needs.
Open-text themes it coded
No integration with our accounting software (15 Sage; 3 FreeAgent; 4 other packages) 22No time to set up / setup took too long 24Too expensive / per-user pricing does not suit us 23Approval workflow did not fit how we sign off 15Mobile app and receipt scanning problems 10Chose a competitor 8Decided we did not need an expenses tool 12
The survey it planned
2 screening questions and 10 questions, as the model wrote them.
-
S1
Which of these best describes your part in your company's free trial of Tallyroom, the expenses app?
One answer- I set up or managed the trial for my company
- I used Tallyroom during the trial, but someone else set it up and managed it
- I was not involved in a Tallyroom trial
- I don't remember a Tallyroom trial
Continues if I set up or managed the trial for my company
-
S2
What happened when your company's Tallyroom free trial came to an end?
One answer- We took out a paid Tallyroom subscription
- We did not take out a paid Tallyroom subscription
- Our trial is still running
- Don't know
Routing Shown only if S1 is: I set up or managed the trial for my company
Continues if We did not take out a paid Tallyroom subscription
-
Q1
Including yourself, how many people work at your company in total?
One answer- 1 to 9
- 10 to 49
- 50 to 249
- 250 or more
-
Q2
In your own words, what was the main reason your company did not take out a paid Tallyroom subscription after the trial? A sentence or two is fine.
Open text -
Q3
Which one of these comes closest to the main reason your company did not subscribe to Tallyroom?
One answer- It did not connect to our accounting software
- It was missing another feature we needed
- Setting it up took more time or effort than we could give it
- Our staff or managers did not take to using it
- It was difficult to use
- It did not work reliably (for example, errors reading receipts)
- The price was too high for us
- The trial was too short for us to finish evaluating it
- We got busy with other things and the trial ran out before we had tried it properly
- We chose a different expenses app
- We decided our existing way of handling expenses was good enough
- We do not have enough expenses to need an app
- Someone else (for example an owner, director or our accountant) decided against it
- We still intend to subscribe but have not done so yet
- Other (please specify)
- Don't know / can't remember
-
Q4
Apart from your main reason, which of these, if any, also played a part in your company not subscribing? Please select all that apply.
Any that apply- It did not connect to our accounting software
- It was missing another feature we needed
- Setting it up took more time or effort than we could give it
- Our staff or managers did not take to using it
- It was difficult to use
- It did not work reliably (for example, errors reading receipts)
- The price was too high for us
- The trial was too short for us to finish evaluating it
- We got busy with other things and the trial ran out before we had tried it properly
- We chose a different expenses app
- We decided our existing way of handling expenses was good enough
- We do not have enough expenses to need an app
- Someone else (for example an owner, director or our accountant) decided against it
- We still intend to subscribe but have not done so yet
- Other (please specify)
- There were no other reasons
- Don't know / can't remember
-
Q5
How easy or difficult was it to set Tallyroom up for your team (for example, adding staff, choosing who approves expenses, and connecting your accounting software)?
One answer- Very easy
- Fairly easy
- Neither easy nor difficult
- Fairly difficult
- Very difficult
- We did not try to set it up
-
Q6
Which of these, if any, did your team need but could not get from Tallyroom during your trial? Please select all that apply.
Any that apply- A connection to our accounting software
- A connection to our payroll software
- Mileage claims
- Company card transactions brought in automatically
- Receipts in foreign currencies
- More than one level of approval
- VAT recorded the way we need it
- Paying staff back directly through the app
- Reports on spending
- Other (please specify)
- Nothing we needed was missing
- Don't know - we did not use it enough to say
-
Q7
Which accounting software does your company mainly use?
One answer- Xero
- QuickBooks
- Sage Business Cloud Accounting (online)
- Sage 50 (desktop)
- FreeAgent
- KashFlow
- Zoho Books
- Our accountant keeps our books in their own software
- We use spreadsheets or no accounting software
- Other (please specify)
- Don't know
-
Q8
Thinking back over the 30-day trial, when did your company reach its decision not to subscribe?
One answer- In the first week
- In the second week
- In the third or fourth week
- After the trial had ended
- We never made an active decision - the trial simply came to an end
- Don't know / can't remember
-
Q9
For trying out Tallyroom properly in your company, was the 30-day trial...
One answer- Much too short
- A little too short
- About right
- A little longer than needed
- Much longer than needed
- Don't know / no opinion
-
Q10
Tallyroom costs £6 per user per month. For your company, is that...
One answer- Much more than we would be willing to pay
- A little more than we would be willing to pay
- About what we would be willing to pay
- Less than we would be willing to pay
- We did not get as far as considering the price
Judges: leadingThe price scale is unbalanced: it offers two categories above willingness to pay but only one below, giving greater emphasis and differentiation to the view that the price is too high.
The price scale is unbalanced. It has two 'more than we would pay' points ('Much more', 'A little more') but only one 'Less than' point and no 'much less' option. This tilts answers towards 'too expensive' and could overstate the case for a cheaper plan.
Sample plan
WHO IS INVITED: Every admin of a UK team whose trial ended in the last 90 days without paying. That is about 1,560 people (1,900 trials x 82% non-payers), so this is a census with no sampling. Each admin gets a unique link, one initial email and two reminders over about 10 days, and a £10 voucher on completion. The email subject and body stay neutral ('tell us about your trial') and do not mention trial length or any other hypothesis.
COMPLETES: The target is 450, but that needs about a 29% response rate from lapsed trialists. That is optimistic; 15-25% (about 230-390 completes) is more realistic. Screen-outs from S1 and S2 are expected to be few.
QUOTAS: No quota will close, because we cannot afford to turn anyone away. Instead, each band (1-9, 10-49, 50-249) has a monitoring target. Bands are tracked daily from Q1, and extra reminders go to any band that falls behind. Respondents answering '250 or more' complete the survey but are reported separately and excluded from the band results.
PRECISION - WHAT IS AND IS NOT ACHIEVABLE: The requested +/-6 points at 95% in each band cannot be guaranteed with 450 completes.
- Requirement without population correction: about 267 completes per band, or about 800 in total.
- With correction for the small population: about 200 completes if a band has 800 lapsed teams, about 175 if it has 500, about 130 if it has 250, and about 95 if it has 150.
- Best case with 450: split evenly at 150 per band, the margin is about +/-8 points, or roughly +/-6 to 7 after population correction.
- Likely case: lapsed teams probably skew to 1-9 employees. If the 50-249 band has only around 150-200 lapsed teams, it may yield about 40-60 completes, giving roughly +/-11 to 13 points.
- Differences between bands are less precise still. With 150 per band, two bands must differ by about 11 points or more (roughly 16 before population correction) to be told apart.
- Before launch we will count lapsed teams per band, using seat count from the database as a proxy, and tell Product the achievable margin per band. Results will carry the margins actually achieved.
- If +/-6 per band is essential, the options are to widen the window to 180 days (at the cost of poorer recall) or to keep the survey running for next quarter's lapsed trials and pool the results.
FIELDWORK AND QUALITY:
- The answer list in Q3 and Q4 is shown in random order, the same order for both questions per respondent. 'Other', 'There were no other reasons' and 'Don't know' stay fixed at the bottom. 'There were no other reasons', 'Nothing we needed was missing' and the 'Don't know' options are exclusive.
- The Q6 feature list must be checked with Product before launch so that it does not list things Tallyroom already does well.
- There is no attention-check question, because the 10-question limit is better spent on the objectives and links are unique and invitation-only. Quality is controlled instead by one complete per link, removing speeders (under about 60 seconds) and reviewing Q2 for nonsense answers.
- With the respondent's knowledge (stated in the invitation), each response is linked to trial account data: seats invited, receipts submitted, whether accounting software was connected, and the day of last activity. This is why the survey does not ask about usage.
- Responders are compared with non-responders on these database fields. Total-level results are weighted to the size-band and activity profile of all lapsed trials.