Benchmarks / SurveyBench / Why Quaymark's customer score fell / Claude Fable 5.2

Measured by Spring Prompt

Claude Fable 5.2: Why Quaymark's customer score fell

The decision: whether to reverse the per-shipment fee and bring back the subscription, invest in shipment tracking and proactive delay alerts, or add account managers for mid-sized customers. The brief and the data →

Verdict: ✓ Sound research
Research score: 100 of 100
Analysis rating: 1,683
Head to head, this task: won 9 of 9

Scorecard

Numbers 12 of 12 right

Question		Its answer	True
What is the Q3 2026 wave's NPS across all respondents (percentage scoring 9-10 minus percentage scoring 0-6), unweighted?	✓	12.9	12.9
Weighted to the shipments-a-month mix of all 4,720 active accounts, what is the Q3 2026 wave's NPS?	✓	17.8	17.8
Re-weighted to the shipments-a-month mix of the Q1 2026 wave's respondents, what is the Q3 2026 wave's NPS?	✓	20.3	20.3
Among respondents who scored 0-6 and answered the open question, what percentage of answers mention tracking (not being able to see where shipments are) or late or missed collections or deliveries?	✓	68.9%	68.9%
How many of the open-text answers mention the per-shipment fee or its cost?	✓	21	21
What is the 95% margin of error, in percentage points, on the Q3 2026 wave's unweighted share of respondents scoring 9-10?	✓	4.1 points	4.12 points
Among accounts booking 100 or more shipments a month, is the change in the share scoring 9-10 between the Q1 2026 and Q3 2026 waves statistically significant at the 95% level? (1 for yes, 0 for no)	✓	No	No
What percentage of the accounts invited to the Q3 2026 wave completed it?	✓	11%	11%
Among Q3 2026 respondents booking fewer than 20 shipments a month, what percentage said the per-shipment fee is worse for their business than the subscription?	✓	49.2%	49.2%
Among accounts booking fewer than 20 shipments a month, by how many points did NPS fall between the Q1 2026 and Q3 2026 waves?	✓	3.9 points	3.9 points
How many Q3 2026 respondents who send shipments to the Republic of Ireland scored 0-6?	✓	13	13
Weighted to the shipments-a-month mix of all 4,720 active accounts, about how many accounts would score 0-6?	✓	1,001	1,001

Traps in the data 8 of 8 handled

✓The Q1 wave (+24) was phone interviews by account managers; the Q3 wave (+13) was online. Interviewer-administered scores, especially when the interviewer is the customer's own account manager, run higher, so part of the fall may be the change of method rather than a change in loyalty.
✓Accounts booking fewer than 20 shipments a month are 61% of Q3 respondents against 30% in Q1 and 40% of active accounts, and they score lowest (Q3 NPS +5). Re-weighted to Q1's mix the Q3 NPS is +20.3, and to the active-account mix +17.8, so much of the 11-point fall is a change in who answered.
✓The commercial director believes the fee caused the fall. Among detractors who explained their score, 69% mention tracking or delays and 18% the fee; only 39% of all respondents say the fee is worse than the subscription (49% of small accounts, 15% of the largest).
✓Respondents who ship to Ireland (base 14) are nearly all detractors (NPS -93).
✓45% of small accounts say they would be likely to choose a monthly subscription if offered: a stated intention about a hypothetical, with no price given.
✓521 of 4,720 invited accounts completed (about 11%); unhappy customers and small accounts may be more likely to respond online.
✓Among accounts booking 100 or more a month (bases 98 and 48), the promoter share moved from 49% to 52%, NPS +38 to +38: not significant.
✓The 69% figure is a share of the 45 detractors who answered the open question, not of all respondents or all customers.

The analysis

Recommendation

Invest in tracking and proactive delay alerts. This has the broadest and most consistent support: two-thirds of respondents pick tracking or alerts, in every size band, and it accounts for 69% of detractor comments. Start by auditing which partner hauliers actually supply scan and ETA data, because comments suggest the data is missing, not just badly displayed. Pair it with haulier on-time performance management, because alerts do not fix late collections.

Do not reverse the per-shipment fee on this evidence. About a third of the weighted base say it is worse and roughly a quarter say it is better. Those who say worse are concentrated among small shippers. Check billing and churn data first, then test a priced optional subscription or monthly cap for accounts that are worse off.

Do not add account managers for mid-sized customers. Only 16% of them ask for one.

For the board: report Q3 NPS as about +18 weighted (+13 unweighted) and as a new baseline. Do not report an 11-point fall in loyalty. Like-for-like the fall is about 3 points and within noise.

Treat Ireland as a lead to check in operational data, not a proven problem.

What would change it

Fee: I would move towards restoring a subscription if billing and churn data show accounts that pay more since March are leaving or shipping less at a materially higher rate, or if a priced subscription test gets strong real take-up, not just stated interest.
Tracking: I would redirect the spend from product features to haulier data integration and haulier performance if the audit shows tracking data is not available from hauliers, or if on-time data shows the underlying problem is lateness.
Account managers: I would revisit this for the largest accounts only if interviews with 100+ shippers and revenue-at-risk data show a named contact would protect significant revenue.
Ireland: I would escalate it to a priority if lane data shows Ireland on-time rates well below Great Britain and the number of Ireland accounts is material.
Trend: I would accept that NPS is falling if a like-for-like online wave, weighted to the base, shows a significant drop from this wave.

Findings

Chief executive: no, do not tell the board that loyalty fell 11 points. Most of the gap between +24 and +13 comes from who was asked and how. On a like-for-like customer mix the fall is about 3 points, which is within sampling noise. The honest board line is: NPS is about +18 once weighted to the customer base, and this wave is a new baseline.

High confidenceBase 933

Q1 was 412 phone interviews by account managers with their own customers: 30% small, 46% mid, 24% large. Q3 was an online survey of all accounts with 521 completes: 61% small, 30% mid, 9% large. The customer base is 40/42/18. Q3 NPS is +12.9 unweighted, +17.8 weighted to the base, and +20.3 re-weighted to the Q1 mix, against +23.5 in Q1. Within bands: small +8.9 to +5.0, mid +25.8 to +21.4, large +37.8 to +37.5. Each band change has a margin of roughly ±15 points, so none is significant. The remaining 3 to 4 points could also be a mode effect, since people score higher when their own account manager asks by phone. NPS measures willingness to recommend, not loyalty or retention.
Tracking and delay information is the biggest and most consistent problem, across every size band. It is the best-supported of the three investment options.

High confidenceBase 521

Q7: live tracking 41.3% (±4 points) and delay alerts 34.2%. 65.6% chose at least one of the two, against 22.6% for lower booking fees and 14.2% for an account manager. The net is 63% to 73% in every size band. Q4: 28.6% are dissatisfied with seeing where shipments are and only 41.2% are satisfied. Dissatisfaction is 48.4% among detractors, 35.6% among passives and 8.0% among promoters. Open text: 42 of 186 answers complain about tracking and 29 about late or missed collections or deliveries, 59 in total. Among detractors who commented, 31 of 45 (69%) cite tracking or delays and 8 of 45 (18%) cite the fee.
Commercial director: no, the survey does not show customers want the subscription back across the board. The fee is a real grievance concentrated in small shippers, but it is not the main reason for low scores. There is moderate, unpriced interest in a subscription as an option, not a mandate to reverse.

Medium confidenceBase 521

Q5: 39.0% say the fee is worse, 20.0% better and 29.9% about the same. Weighted to the customer base that is roughly 33% worse, 23% better and 33% the same. Worse is 49.2% among small shippers (n=319, ±5.5), 25.3% among mid (n=154) and 14.6% among large (n=48, 7 people). Q6: 39.5% say they would be likely to choose a subscription if offered and 36.7% unlikely. No price was shown, and the question asked about an alternative, not a reversal. Among those who say the fee is worse, 61.6% are likely to choose it (n=203). The fee is mentioned in 21 of 186 comments (11%), 13 of them from passives scoring 7-8. 29.4% of promoters also say the fee is worse. The 11-point drop cannot be attributed to the fee (see the first finding).
The survey may understate the damage from the fee, because customers who left after March are not in it.

Medium confidenceBase 521

The sample is active accounts with a shipment in the last three months, and the screener (S1) repeats that condition. Anyone who stopped booking after the fee change is excluded. Billing and churn data are needed to see whether activity fell among accounts that now pay more.
Late and missed collections are a problem in their own right. Tracking and alerts will make delays visible, not prevent them.

Medium confidenceBase 521

Q3: 22.4% are dissatisfied with punctuality, 44.1% among detractors against 8.0% among promoters. 29 of 186 comments describe late or missed collections or deliveries, 12 of them together with missing tracking information. Several say tracking data is simply absent for some hauliers (Q457, Q521, Q117).
The case for account managers for mid-sized customers is not supported. The interest that exists is among the largest accounts, and even there tracking ranks higher.

Medium confidenceBase 154

Q7 'a named account manager': 15.6% among 20-99 shippers (n=154, about 24 people), 11.0% among small shippers, and 31.2% among 100+ shippers (n=48, 15 people, ±13 points). Among 100+ shippers live tracking is 54.2%. No open-text answer asks for an account manager.
Operations director: there is a signal worth checking, but the survey cannot show the size of an Ireland problem. It rests on 14 respondents, and they are lukewarm about lateness more than furious.

Low confidenceBase 14

13 of 14 Ireland shippers scored 0-6 (NPS -93), against 21.2% of the 496 who do not ship to Ireland. A gap that large is unlikely to be chance, but 14 self-selected respondents (2.7% of the sample) cannot be scaled to the base. The seven who commented all scored 5 or 6: five describe late or missed collections or deliveries, one tracking and one the fee. That is the same complaint as other customers, only more concentrated.
The largest accounts are stable: no detectable change in recommendation among 100+ shippers.

Low confidenceBase 48

The share scoring 9-10 was 49.0% in Q1 (n=98) and 52.1% in Q3 (n=48). The 3-point difference is not significant (z about 0.35) and NPS is about +38 in both waves. The Q3 base of 48 is small, so only a large change would show.
What customers value is fast booking. Protect it while changing anything else.

Medium confidenceBase 186

79 of 186 comments (42%) praise quick, easy quoting and booking ('a couple of minutes'). It is the dominant reason given by promoters. Six of those 79 praise booking and criticise tracking in the same answer.

Caveats

The Q1 and Q3 waves are not comparable. Q1 was by phone, with account managers interviewing a selection of their own customers. Q3 was an online survey emailed to every active account. Mode, interviewer and sample all changed, alongside the fee change in March. The 11-point gap cannot be read as a change in customer sentiment, nor attributed to the fee.
The Q3 response rate is 11% (521 of 4,720) and the tables are unweighted. Small shippers are over-represented (61% against 40% of the base) and large shippers under-represented (9% against 18%). Weighting to the base moves NPS from +13 to about +18. Weighting corrects for size mix only, not for who chose to respond within each band.
Margins of error are about ±4 points on total-sample percentages and about ±6.5 points on the Q3 NPS. They are about ±14 points on a percentage among 100+ shippers (n=48). The Ireland group (n=14) is too small for percentages to mean much.
The survey covers active accounts only. Customers who stopped booking after the fee change are absent, so the fee's effect on retention is invisible here.
Q6 is a hypothetical with no price attached and asks about a subscription as an alternative, not a return to subscription-only. Stated likelihood usually overstates real take-up. Q5 measures perception of cost, not actual bills.
Q7 allowed only two picks and offered tracking and alerts as two separate options, which helps the combined figure. 'Lower booking fees' is not the same as wanting the subscription back.
Open-text counts come from the 186 of 521 who chose to comment and were coded by one analyst. Treat them as indicating reasons, not as prevalence.
NPS is a recommendation-intention measure. It is not evidence of loyalty, retention or revenue.
Relationships between satisfaction, fee views and NPS group are correlations. They do not prove that fixing tracking will raise NPS by any given amount.

Next steps

decisionApprove discovery and a first release of tracking and proactive delay alerts. Begin with an audit of scan and ETA data coverage by partner haulier.
researchAnalyse billing and activity data before and after March 2026 by size band: who actually pays more, and whether those accounts have churned or reduced shipments. Include accounts that have lapsed.
experimentRun a priced test of an optional monthly subscription or monthly fee cap with a random sample of accounts, focused on small and frequent shippers.
researchPull on-time collection and delivery data for Republic of Ireland lanes against Great Britain and count how many accounts ship to Ireland. Interview 8-10 Ireland shippers, including respondents who agree to follow-up.
monitorFix the tracker method: same online method, same all-active-accounts frame, results weighted to the size mix of the base, reported with margins of error. Use Q3 2026 as the baseline, retire the Q1 comparison, and do not have account managers conduct interviews.
decisionDo not hire account managers for mid-sized accounts now. Interview a small number of 100+ shippers on what they would want from a named contact.
monitorSet up a haulier scorecard for on-time collection and delivery and missed slots, and review it monthly with operations.
researchRun a short survey or call programme with accounts that stopped booking after March 2026.

Open-text themes it coded

Booking and quoting is quick and easy (positive) 79Poor tracking or visibility: cannot see where shipments are, no updates or alerts, have to chase 42Late or missed collections and deliveries 29Per-shipment fee costs more or is less predictable than the subscription 21Customer support slow or hard to reach 6Damaged goods and slow claims 5Works fine, no strong feelings 17Too new or too infrequent a user to judge 5

The survey it planned

2 screening questions and 9 questions, as the model wrote them.

S1
Which of these best describes your part in booking shipments through Quaymark for your business?
One answer
- I make most or all of our bookings
- I manage the people who make our bookings and am the main contact for our Quaymark account
- I make some bookings, but someone else is the main contact
- I am not involved in booking shipments through Quaymark
Continues if I make most or all of our bookings; I manage the people who make our bookings and am the main contact for our Quaymark account
S2
When did your business last book a shipment through Quaymark?
One answer
- In the last 3 months
- More than 3 months ago
- Not sure
Continues if In the last 3 months
Q1
How likely is it that you would recommend Quaymark to another business or colleague? Please answer on a scale from 0 to 10.
Scale

Scale 1-11: 0 - Not at all likely to 10 - Extremely likely
Q2
What are the main reasons for the score you just gave? Please mention anything Quaymark does well and anything it does not.
Open text
Q3
Which, if any, of the following has your business experienced with Quaymark in the last three months? Please select all that apply.
Any that apply
- A shipment was collected or delivered later than promised
- A shipment was running late and we were not told until we chased or our own customer told us
- We could not tell from the online tracking where a shipment was
- We contacted Quaymark support about a problem
- We were charged more for a booking than we expected
- None of these
Q4
Quaymark is deciding what to work on next. Please rank the following from most valuable to your business (1) to least valuable (4).
Ranking
- More accurate and more detailed online tracking of shipments
- Alerts sent to you as soon as a shipment is expected to be late
- A named account manager you can contact directly
- A monthly subscription instead of a booking fee for each shipment
Q5
Was your business already booking shipments through Quaymark before March 2026?
One answer
- Yes
- No, we started using Quaymark in March 2026 or later
- Not sure
Q6
In March 2026 Quaymark replaced its monthly subscription with a booking fee for each shipment. Thinking about what your business pays Quaymark in a typical month, how does it compare with what you paid under the monthly subscription?
One answer
- We now pay much more
- We now pay a little more
- We pay about the same
- We now pay a little less
- We now pay much less
- Don't know
Routing Shown only if Q5 is: Yes
Q7
Quaymark now charges a booking fee for each shipment (£3.50, reduced for accounts booking 100 or more shipments a month), with no monthly charge. Until March 2026 it charged a monthly subscription instead (from £25 a month, depending on account size), with no fee per shipment. Which would your business prefer?
One answer
- Strongly prefer a monthly subscription
- Slightly prefer a monthly subscription
- No preference
- Slightly prefer a booking fee for each shipment
- Strongly prefer a booking fee for each shipment
- Don't know
Q8
What is the main reason for your preference?
Open text

Routing Shown only if Q7 is: Strongly prefer a monthly subscription; Slightly prefer a monthly subscription; Slightly prefer a booking fee for each shipment; Strongly prefer a booking fee for each shipment
Q9
On average, about how many shipments does your business book through Quaymark each month?
One answer
- Fewer than 5
- 5 to 19
- 20 to 49
- 50 to 99
- 100 to 249
- 250 or more
- Don't know

Sample plan

WHO IS INVITED: A census, not a sample. One email goes to the main booking contact of each of the 4,720 active UK accounts (at least one shipment booked in the last three months), taken from the customer database. Each account gets a unique link, so there is one response per account and we can attach database fields: actual monthly shipment volume, account start date, spend before and after March 2026, and support complaints. S1 and S2 confirm that the person answering is the main booking contact and that the account is active. Anyone screened out at S1 is asked to forward the link to the main contact.

COMPLETES AND QUOTAS: The target is about 520 completes, an 11% response rate. There are no hard quotas, because we should not turn customers away. Instead we monitor completes daily in the three board bands (fewer than 20, 20 to 99, 100 or more shipments a month, from database volume) and send up to two reminders, aimed first at the bands that are behind. Account managers may encourage customers to respond but must not ask the questions or see individual answers. Answers are confidential, and the invitation says so.

PRECISION, STATED PLAINLY: 520 completes cannot give plus or minus 5 points at 95% confidence in each of the three bands.

With the 9-10 share near 45%, plus or minus 5 points needs about 380 completes per band before the finite-population correction.
After the correction it needs roughly 220 completes from a band of 500 accounts, 305 from a band of 1,500 and 335 from a band of 2,700. That is around 860 completes in total, an 18-20% response rate, and it depends on the real band sizes.
At 520 completes split evenly (about 173 per band), the margin is about plus or minus 7 points per band. A band with only 100 completes is at about plus or minus 9 to 10 points.
The overall 9-10 share will be within about plus or minus 4 points.
If the 100-or-more band has only a few hundred accounts, plus or minus 5 points there needs over half of them to respond.

The board has two options: accept roughly plus or minus 7 to 10 points per band at 520 completes, or extend fieldwork and reminders to reach roughly 860 or more completes spread across the bands. Either way we report the margin actually achieved for each band and do not claim plus or minus 5.

WEIGHTING AND BIAS CHECKS: Overall figures are weighted to the database distribution of accounts across the three bands. Respondents are compared with non-respondents on size, tenure and complaint history. This matters because, with complaints doubled, unhappy customers may be more likely to answer.

FIELDWORK NOTES:

Q1 comes first and Q2 second, before fees or improvements are mentioned, so the scores and reasons are unprompted.
Q1 uses the standard recommend wording. If the Q1 2026 wave used different wording, use that wording exactly.
The order of options in Q3 (except 'None of these') and Q4 is randomised.
The Q7 scale is shown subscription-first to a random half and fee-first to the other half.
Q8 is optional.
There is no trap question, because this is a short survey of business customers. Quality is checked through completion time and the open answers.
Expected length is 4 to 5 minutes.