Benchmarks / SurveyBench / Why Quaymark's customer score fell / GPT-6.1 Sol

Measured by Spring Prompt

GPT-6.1 Sol: Why Quaymark's customer score fell

The decision: whether to reverse the per-shipment fee and bring back the subscription, invest in shipment tracking and proactive delay alerts, or add account managers for mid-sized customers. The brief and the data →

Verdict
✓ Sound research
Research score
100 of 100
Analysis rating
1,650
Head to head, this task
won 8 of 9

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What is the Q3 2026 wave's NPS across all respondents (percentage scoring 9-10 minus percentage scoring 0-6), unweighted?✓12.8612.9
Weighted to the shipments-a-month mix of all 4,720 active accounts, what is the Q3 2026 wave's NPS?✓17.7617.8
Re-weighted to the shipments-a-month mix of the Q1 2026 wave's respondents, what is the Q3 2026 wave's NPS?✓20.3120.3
Among respondents who scored 0-6 and answered the open question, what percentage of answers mention tracking (not being able to see where shipments are) or late or missed collections or deliveries?✓68.89%68.9%
How many of the open-text answers mention the per-shipment fee or its cost?✓2121
What is the 95% margin of error, in percentage points, on the Q3 2026 wave's unweighted share of respondents scoring 9-10?✓4.12 points4.12 points
Among accounts booking 100 or more shipments a month, is the change in the share scoring 9-10 between the Q1 2026 and Q3 2026 waves statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
What percentage of the accounts invited to the Q3 2026 wave completed it?✓11.04%11%
Among Q3 2026 respondents booking fewer than 20 shipments a month, what percentage said the per-shipment fee is worse for their business than the subscription?✓49.2%49.2%
Among accounts booking fewer than 20 shipments a month, by how many points did NPS fall between the Q1 2026 and Q3 2026 waves?✓3.86 points3.9 points
How many Q3 2026 respondents who send shipments to the Republic of Ireland scored 0-6?✓1313
Weighted to the shipments-a-month mix of all 4,720 active accounts, about how many accounts would score 0-6?✓1,0011,001

Traps in the data 8 of 8 handled

  • ✓The Q1 wave (+24) was phone interviews by account managers; the Q3 wave (+13) was online. Interviewer-administered scores, especially when the interviewer is the customer's own account manager, run higher, so part of the fall may be the change of method rather than a change in loyalty.
  • ✓Accounts booking fewer than 20 shipments a month are 61% of Q3 respondents against 30% in Q1 and 40% of active accounts, and they score lowest (Q3 NPS +5). Re-weighted to Q1's mix the Q3 NPS is +20.3, and to the active-account mix +17.8, so much of the 11-point fall is a change in who answered.
  • ✓The commercial director believes the fee caused the fall. Among detractors who explained their score, 69% mention tracking or delays and 18% the fee; only 39% of all respondents say the fee is worse than the subscription (49% of small accounts, 15% of the largest).
  • ✓Respondents who ship to Ireland (base 14) are nearly all detractors (NPS -93).
  • ✓45% of small accounts say they would be likely to choose a monthly subscription if offered: a stated intention about a hypothetical, with no price given.
  • ✓521 of 4,720 invited accounts completed (about 11%); unhappy customers and small accounts may be more likely to respond online.
  • ✓Among accounts booking 100 or more a month (bases 98 and 48), the promoter share moved from 49% to 52%, NPS +38 to +38: not significant.
  • ✓The 69% figure is a share of the 45 detractors who answered the open question, not of all respondents or all customers.

The analysis

Recommendation

Make tracking and proactive delay alerts the primary investment, starting with reliable haulier event coverage, meaningful statuses, updated ETAs and exception notifications rather than merely a new interface. Pair this with operational work on missed collections and late deliveries. Do not reverse per-shipment pricing for everyone: test a clearly priced optional subscription, especially with smaller accounts reporting higher costs, and assess actual uptake, retention and contribution margin. Do not roll out account managers broadly to mid-sized accounts; test a limited service for high-volume or operationally complex accounts only if its economics justify it. Immediately investigate the Ireland-serving complaints. For the board, report Q3 NPS as +13 unweighted and approximately +18 volume-weighted, and describe the Q1 comparison as non-comparable rather than an 11-point fall in loyalty.

What would change it

A priced subscription experiment showing materially better retention and sustainable contribution margin than per-shipment pricing would justify a broader pricing change. Shipment records showing that tracking coverage is already reliable and that complaints arise mainly from actual service failures would shift investment towards haulier reliability rather than tracking presentation. A controlled account-manager pilot producing stronger incremental retention and margin per pound spent than tracking or pricing interventions would justify expanding that service. Verified, route-specific excess lateness or missed collections on Ireland lanes would warrant a targeted operational intervention. A consistent-mode, consistently sampled trend linked to observed retention would support a firmer statement about changes in customer advocacy and loyalty.

Findings

  1. Prioritise reliable shipment tracking and proactive delay alerts. These address the broadest expressed need, across shipment-volume bands, rather than a problem confined to unhappy customers.

    Medium confidenceBase 521

    342 of 521 respondents (65.6%) selected tracking or delay alerts among their top two improvements: 41.3% selected tracking and 34.2% alerts. The combined preference was 65.8% among fewer-than-20 accounts, 63.0% among 20-to-99 accounts and 72.9% among 100-plus accounts. Tracking dissatisfaction was 28.6%, versus 22.4% for punctuality. Tracking dissatisfaction was 48.4% among detractors and 8.0% among promoters; this association does not establish causation.

  2. The core experience is fast booking followed by poor visibility and avoidable chasing. Protect booking simplicity while improving the post-booking experience; alerts alone will not fix unreliable collections or deliveries.

    Medium confidenceBase 186

    Of 186 open answers, 79 praised quick or easy booking, 42 mentioned poor tracking, and 29 mentioned late or missed collections or deliveries. Themes overlap. Among the 45 detractors who answered the open question, 31 (68.9%) mentioned tracking or lateness. Q087 captures the contrast: 'Love how fast the booking process is. Hate that I can't see where my pallets are once they've been collected.'

  3. Commercial director: the survey supports testing an optional subscription, not reversing the per-shipment fee for everyone. Pricing pain is real but concentrated, and stated subscription interest is not a majority.

    Medium confidenceBase 521

    203 of 521 respondents (39.0%) said the fee was worse, while 20.0% said better and 29.9% about the same. Among fewer-than-20 respondents, 49.2% said worse and 44.5% were likely to choose a subscription. Across all respondents, 39.5% were likely and 36.7% unlikely to choose it; only 17.9% were very likely. Among the 203 saying the fee was worse, 61.6% were likely to choose a subscription. There were 21 spontaneous fee or cost mentions among 186 open answers. No subscription price or terms were specified.

  4. Chief executive: do not tell the board that customer loyalty fell by 11 points. Report an approximately 11-point difference between non-comparable, unweighted NPS samples, not a demonstrated fall in loyalty or an effect of the fee.

    High confidenceBase 933

    Q1 NPS was approximately +24 among 412 account-manager phone interviews; Q3 was +13 among 521 online respondents. Smaller accounts comprised 30.1% of Q1 respondents but 61.2% of Q3 respondents and have lower NPS. Reweighting Q3 to the Q1 respondent volume mix raises its NPS to +20.31, removing about 7.5 points of the raw gap descriptively. Within-band NPS falls were approximately 3.9 points for fewer-than-20 accounts and 4.3 for 20-to-99 accounts, with effectively no change for 100-plus accounts. Mode, interviewer, coverage and respondent differences remain unresolved.

  5. Use shipment-volume weighting for the current account-level picture, but do not mistake weighting for proof of representativeness.

    Medium confidenceBase 521

    The Q3 sample volume mix was 61.2% fewer than 20 shipments, 29.6% 20 to 99, and 9.2% 100 or more, compared with population proportions of 40%, 42% and 18%. Applying those population proportions gives NPS +17.76 and a detractor share of approximately 21.2%. Applied to 4,720 accounts, that implies about 1,001 detractor accounts, conditional on respondents representing nonrespondents within each band. Only 521 of 4,720 invited accounts completed the survey (11.04%).

  6. Do not make named account managers the main investment for mid-sized customers on this evidence. Consider a targeted high-volume pilot instead.

    Medium confidenceBase 202

    Among 154 respondents booking 20 to 99 shipments, 15.6% chose a named account manager, versus 40.3% tracking and 35.1% alerts. Demand for account managers was higher among the 48 respondents booking 100-plus shipments (31.2%), but tracking still led at 54.2%. The high-volume promoter share rose from approximately 49.0% of 98 Q1 respondents to 52.1% of 48 Q3 respondents; this difference is not statistically significant at the 95% level.

  7. Operations director: investigate Ireland-serving accounts urgently, but do not yet declare a proven Ireland-route problem.

    Low confidenceBase 14

    13 of the 14 respondents sending shipments to the Republic of Ireland were detractors, producing NPS approximately -93, compared with +16 among 496 non-Ireland-serving respondents. Seven Ireland-serving respondents provided open answers: six mentioned tracking or reliability and one mentioned fees. The answers do not establish that the problematic shipments were Ireland-bound. Q9 identifies customers using that destination, not customers located in Ireland.

Caveats

  • Q3 was an attempted census of active accounts, but the 11.04% completion rate leaves substantial scope for nonresponse bias. It excludes customers who stopped using Quaymark before the active-account window, potentially missing customers lost after the fee change.
  • Q1 was conducted by account managers with their customers by phone; Q3 was self-completed online by eligible bookers across active accounts. Coverage, interviewer effects, mode, seasonality and customer composition can all affect the comparison. These are not matched longitudinal observations.
  • NPS measures stated recommendation, not directly observed loyalty, retention or share of shipping spend. The survey cannot attribute a change to the March pricing change.
  • Weighted answers use integer counts reconstructed from the rounded percentages and column bases: Q3 promoters/detractors are 99/83, 63/30 and 25/7 across the three volume bands. Small discrepancies from calculations using rounded table percentages are expected.
  • Volume weighting adjusts only that observed characteristic. It does not correct differences in willingness to respond within bands, customer tenure, haulier exposure, destination mix or other unmeasured factors. The estimate of 1,001 detractor accounts is an indicative extrapolation, not a count of identified accounts.
  • The 4.12-point promoter margin is a conventional 95% binomial normal-approximation margin, without finite-population correction. It describes sampling uncertainty under random-sampling assumptions, not total uncertainty from this voluntary survey. The high-volume significance calculation also assumes independent random samples.
  • Only 186 of 521 respondents supplied open answers; 45 were detractors. Theme percentages apply to those answering the open question, not all customers. Themes are analyst-coded and overlap.
  • Q5 includes customers without pre-March experience and does not verify their actual bills. Q6 tests an unspecified optional subscription, not a mandatory return to the old model; hypothetical interest cannot establish paid uptake.
  • Q7 is a prompted, up-to-two selection task. Its combined tracking-or-alerts figure counts respondents once, not the sum of the two percentages, and measures stated priority rather than willingness to pay or expected return.
  • Ireland results rest on only 14 respondents, with no population benchmark for Ireland usage and no Ireland-specific satisfaction or shipment-performance data. They are a strong diagnostic signal from these respondents, not a reliable estimate of the whole Ireland-serving customer population.

Next steps

  • decisionFund a staged tracking and delay-alert programme, with initial delivery contingent on an audit of haulier data coverage and timeliness.
  • experimentPilot improved tracking and alerts with a comparable control group, while separately monitoring actual delivery and collection performance.
  • experimentTest explicit optional-subscription prices and terms against the current model, using actual account-level bills and shipment histories.
  • researchContact the 14 Ireland-serving respondents where permission allows, and audit a larger set of Ireland-bound shipments by haulier and lane.
  • researchInvestigate the service needs of high-volume and complex accounts before running a small named-account-manager pilot.
  • monitorEstablish a repeatable survey and behavioural customer-health baseline, including follow-up with nonrespondents and recently lapsed accounts.

Open-text themes it coded

Quick, simple or easy booking 79Missing, stale or uninformative shipment tracking 42Late or missed collections or deliveries 29Per-shipment fee increases costs or reduces budget predictability; preference for subscription 21Slow or difficult access to customer support 6Damaged goods or difficult claims 5Routine satisfaction or no problems, without a specific reason 10Limited experience or neutral assessment 12

The survey it planned

2 screening questions and 10 questions, as the model wrote them.

  1. S1

    Are you the main booking contact for your business's Quaymark account?

    One answer
    • Yes
    • No
    • Not sure

    Continues if Yes

  2. S2

    Has your business booked at least one shipment through Quaymark in the last three months?

    One answer
    • Yes
    • No
    • Not sure

    Continues if Yes

  3. Q1

    How likely are you to recommend Quaymark to another business? Please select a number from 0 to 10.

    Scale

    Scale 1-11: 0 — Not at all likely to 10 — Extremely likely

  4. Q2

    What is the main reason for the score you gave? Please mention anything that has worked well or could be better.

    Open text
  5. Q3

    Over the last three months, about how many shipments has your business booked through Quaymark per month, on average?

    One answer
    • Fewer than 20 shipments a month
    • 20 to 99 shipments a month
    • 100 or more shipments a month
    • Not sure
  6. Q4

    Did you personally use Quaymark for your business before March 2026, when Quaymark charged a monthly subscription?

    One answer
    • Yes
    • No
    • Not sure
  7. Q5

    Thinking about your business's booking needs, how does Quaymark's current per-shipment booking fee compare with its previous monthly subscription in terms of value for money?

    One answer
    • Much worse value
    • Somewhat worse value
    • About the same value
    • Somewhat better value
    • Much better value
    • I do not know enough about the charges to compare
    • I was not involved enough in the previous subscription to compare

    Routing Shown only if Q4 is: Yes

  8. Q6

    In March 2026, Quaymark replaced its monthly subscription, which started at £25 a month for the smallest accounts, with a booking fee of £3.50 per shipment, reduced for accounts booking 100 or more shipments a month. Thinking about your business's expected booking needs, which charging approach would you prefer? Please consider the Quaymark charges, rather than the haulier's shipping price.

    One answer
    • The previous monthly subscription
    • The current per-shipment booking fee
    • No preference between these approaches
    • Neither approach
    • I would need to know the exact charges for my business before choosing
  9. Q7

    Which, if any, of these have worked well for your business when using Quaymark in the last three months? Select all that apply.

    Any that apply
    • The cost of using Quaymark
    • Getting prices and booking shipments
    • Collections and deliveries happening as expected
    • Tracking shipments online
    • Being informed about delays
    • Getting help from Quaymark when needed
    • Something else
    • None of these
    • Not sure
  10. Q8

    Which, if any, of these have caused problems for your business when using Quaymark in the last three months? Select all that apply.

    Any that apply
    • The cost of using Quaymark
    • Getting prices and booking shipments
    • Collections and deliveries happening as expected
    • Tracking shipments online
    • Being informed about delays
    • Getting help from Quaymark when needed
    • Something else
    • None of these
    • Not sure
  11. Q9

    Which of these changes would be most useful to your business? Select up to two. If none would be useful, select 'None of these'.

    Any that apply
    • More accurate, up-to-date shipment tracking online
    • Automatic alerts when a shipment is delayed, with an updated expected arrival time
    • A named Quaymark account manager who knows your business and can help with bookings and problems
    • None of these
    • Not sure
  12. Q10

    If Quaymark could prioritise only one of the following, which would you most want it to do for your business?

    One answer
    • Bring back the previous monthly subscription instead of the current per-shipment booking fee
    • Improve shipment tracking and introduce automatic delay alerts
    • Provide a named account manager who knows my business and can help with bookings and problems
    • Keep the current service and charging approach without these changes
    • None of these
    • Not sure
Sample plan

Invite the main booking contact at each of the 4,720 eligible UK business accounts, using account-specific email links and allowing one response per account. Eligibility is at least one Quaymark shipment in the previous three months; verify the database extract against this rule before sending. Use a neutral invitation describing a short survey about customers' experience, with no reference to an expected preference for subscription pricing. Participation is voluntary and responses should not affect account service. Target about 520 eligible completes, an approximately 11% completion yield. Allow around two weeks and send up to two reminders to nonrespondents, targeting underrepresented account-size groups where necessary. Before launch, calculate each account's average monthly Quaymark shipment volume over the previous three months and count the three size groups. Set minimum completion targets by size rather than relying only on an overall target. The requested ±5 percentage-point precision cannot be promised from 520 completes without these population counts. For planning, the conservative large-population requirement for a proportion is 385 responses per group at 95% confidence. Under random-response assumptions, the finite-population target for a group containing N accounts is the ceiling of N × 1.96² × 0.25 / [0.05² × (N − 1) + 1.96² × 0.25]. For illustration, three roughly equal groups would require about 309 completes each, or 927 overall; 520 split equally would give approximately ±7 percentage points per group under those assumptions. Calculate the actual targets before fieldwork. If their sum exceeds 520, obtain agreement either to increase the completion target or to accept and clearly report wider intervals; do not claim the requested precision has been achieved. This is an attempted census with voluntary response, so sampling calculations do not remove nonresponse bias. Pilot the questionnaire on mobile and desktop with eligible contacts before launch, confirming a typical completion time below six minutes. Randomise the substantive options in Q6 and Q9–Q10, keeping neutral and uncertainty options last. Use the same randomised substantive-option order in Q7 and Q8. Make 'None of these' and 'Not sure' exclusive in multi-select questions; limit Q9 to two substantive selections. Allow respondents to skip substantive questions and record skips separately from explicit uncertainty responses.