Benchmarks / SurveyBench / Why Quaymark's customer score fell / GPT-6.2 Sol

Measured by Spring Prompt

GPT-6.2 Sol: Why Quaymark's customer score fell

The decision: whether to reverse the per-shipment fee and bring back the subscription, invest in shipment tracking and proactive delay alerts, or add account managers for mid-sized customers. The brief and the data →

Verdict
✓ Sound research
Research score
100 of 100
Analysis rating
1,354
Head to head, this task
won 2 of 12

Scorecard

Numbers 12 of 12 right

QuestionIts answerTrue
What is the Q3 2026 wave's NPS across all respondents (percentage scoring 9-10 minus percentage scoring 0-6), unweighted?✓12.912.9
Weighted to the shipments-a-month mix of all 4,720 active accounts, what is the Q3 2026 wave's NPS?✓17.7617.8
Re-weighted to the shipments-a-month mix of the Q1 2026 wave's respondents, what is the Q3 2026 wave's NPS?✓20.3120.3
Among respondents who scored 0-6 and answered the open question, what percentage of answers mention tracking (not being able to see where shipments are) or late or missed collections or deliveries?✓66.7%68.9%
How many of the open-text answers mention the per-shipment fee or its cost?✓2121
What is the 95% margin of error, in percentage points, on the Q3 2026 wave's unweighted share of respondents scoring 9-10?✓4.12 points4.12 points
Among accounts booking 100 or more shipments a month, is the change in the share scoring 9-10 between the Q1 2026 and Q3 2026 waves statistically significant at the 95% level? (1 for yes, 0 for no)✓NoNo
What percentage of the accounts invited to the Q3 2026 wave completed it?✓11.04%11%
Among Q3 2026 respondents booking fewer than 20 shipments a month, what percentage said the per-shipment fee is worse for their business than the subscription?✓49.2%49.2%
Among accounts booking fewer than 20 shipments a month, by how many points did NPS fall between the Q1 2026 and Q3 2026 waves?✓4 points3.9 points
How many Q3 2026 respondents who send shipments to the Republic of Ireland scored 0-6?✓1313
Weighted to the shipments-a-month mix of all 4,720 active accounts, about how many accounts would score 0-6?✓1,0011,001

Traps in the data 8 of 8 handled

  • ✓The Q1 wave (+24) was phone interviews by account managers; the Q3 wave (+13) was online. Interviewer-administered scores, especially when the interviewer is the customer's own account manager, run higher, so part of the fall may be the change of method rather than a change in loyalty.
  • ✓Accounts booking fewer than 20 shipments a month are 61% of Q3 respondents against 30% in Q1 and 40% of active accounts, and they score lowest (Q3 NPS +5). Re-weighted to Q1's mix the Q3 NPS is +20.3, and to the active-account mix +17.8, so much of the 11-point fall is a change in who answered.
  • ✓The commercial director believes the fee caused the fall. Among detractors who explained their score, 69% mention tracking or delays and 18% the fee; only 39% of all respondents say the fee is worse than the subscription (49% of small accounts, 15% of the largest).
  • ✓Respondents who ship to Ireland (base 14) are nearly all detractors (NPS -93).
  • ✓45% of small accounts say they would be likely to choose a monthly subscription if offered: a stated intention about a hypothetical, with no price given.
  • ✓521 of 4,720 invited accounts completed (about 11%); unhappy customers and small accounts may be more likely to respond online.
  • ✓Among accounts booking 100 or more a month (bases 98 and 48), the promoter share moved from 49% to 52%, NPS +38 to +38: not significant.
  • ✓The 69% figure is a share of the 45 detractors who answered the open question, not of all respondents or all customers.

The analysis

Recommendation

Fund a staged tracking and delay-alert pilot first, built on reliable haulier status events and actionable exception messages. Investigate Ireland service performance in parallel. Keep per-shipment pricing while testing a priced optional subscription with accounts whose bills or budget predictability worsened. Defer a broad named-account-manager rollout for mid-sized customers.

What would change it

Expand an optional subscription if a priced live offer produces sustained take-up and better retention or booking economics after accounting for margin and customers switching from per-shipment fees. Consider a full reversal only if it outperforms offering both plans. Shift spending from tracking features to haulier reliability first if the pilot shows that status data are too incomplete for trustworthy alerts or if reliable alerts do not reduce chasing. Move Ireland-specific fixes up the queue if shipment records confirm materially worse performance on those lanes. Reconsider named account management if a targeted pilot shows incremental retention or contribution that covers the cost.

Findings

  1. Prioritise useful shipment tracking and proactive delay alerts. This is the strongest improvement opportunity across booking-volume bands, but better visibility will not fix late shipments on its own.

    High confidenceBase 521

    65.6% selected tracking or delay alerts, counting each respondent once: 41.3% selected tracking and 34.2% selected alerts. The combined share was 63.0% among 20–99 shipment accounts (n=154) and about 65.9% after weighting to the active-account volume mix. Tracking dissatisfaction was 28.6% overall and 48.4% among detractors; punctuality dissatisfaction was 22.4% overall and 44.1% among detractors (n=120). Of the 45 detractors who wrote a comment, 30 mentioned tracking or late or missed shipments. The comments show why visibility matters: stale statuses and missed slots leave customers chasing hauliers and unable to update their own customers.

  2. Commercial director: The survey shows demand for an optional subscription, especially among lower-volume respondents, but does not support reversing the fee for everyone or blaming it for the NPS gap.

    Medium confidenceBase 521

    39.0% rated per-shipment pricing worse, 20.0% better and 29.9% about the same; 39.5% said they would be likely to choose an optional subscription, versus 36.7% unlikely. Among respondents booking fewer than 20 shipments (n=319), 49.2% rated the fee worse and 44.5% were likely to choose a subscription. Among those who rated the fee worse (n=203), 61.6% said they would likely choose one. Twenty-one open answers mentioned the fee or its cost, often citing higher bills or less predictable costs. Interest was measured without a subscription price or terms.

  3. Chief executive: Tell the board that reported NPS was +24 in the Q1 phone survey and +13 in the Q3 online survey. Do not say that customer loyalty fell by 11 points.

    High confidenceBase 933

    Q1 had 412 interviews conducted by account managers with their customers; Q3 had 521 online completes from an invitation to all active accounts. The lowest-volume band made up 30.1% of Q1 respondents and 61.2% of Q3 respondents. Reweighting Q3 to the Q1 respondent mix brings it to +20.31, roughly 3 points below the unrounded Q1 score of +23.5; reweighting to the current active-account mix gives +17.76. Volume mix accounts for much of the headline gap, but the remaining gap cannot be separated from mode, interviewer, coverage and nonresponse differences. In the 100+ band, the promoter share rose from 49.0% (n=98) to 52.1% (n=48), a difference that is not statistically significant (two-sided p≈0.72).

  4. Operations director: Treat the Ireland result as an urgent signal to investigate, not proof of an Ireland-wide service failure.

    Low confidenceBase 14

    13 of 14 respondents who send shipments to the Republic of Ireland were detractors, versus 105 of 496 who do not. Seven Ireland-shipping respondents commented: five described lateness or missed service, one cited tracking, and one cited the fee. The recommendation question covered Quaymark overall; the flagged accounts can also book other routes, and the comments do not establish that the incidents happened on Ireland shipments.

  5. Do not roll out named account managers to the 20–99 shipment band on this evidence.

    Medium confidenceBase 154

    15.6% of mid-volume respondents selected a named account manager, versus 63.0% selecting tracking or alerts. Named account management drew more interest in the 100+ band (31.2%, n=48), which may warrant discovery but is too small a base for a staffing decision.

  6. Preserve the speed and simplicity of booking while improving what happens after booking.

    Medium confidenceBase 186

    Across booking-volume bands, open comments describe getting a quote and confirming a collection in a couple of minutes. Several respondents explicitly praised booking while criticising tracking, suggesting a post-booking problem rather than a need to redesign the booking flow.

Caveats

  • Q3 reached all 4,720 invited accounts, but only 521 completed (11.04%). The screeners capture people who book shipments themselves at recently active accounts, not all buyers or decision-makers. Accounts that stopped using Quaymark after the pricing change are not represented in the active-account frame.
  • Weighting adjusts only for the three shipment-volume bands and assumes respondents represent nonrespondents within each band. Q3 bands are self-reported, whereas Q1 bands came from account records. The estimate of about 1,001 detractor accounts is an extrapolation, not a count. Weighted NPS figures use counts reconstructed from the displayed percentages and bases.
  • The 4.12-point promoter margin of error is a conventional 95% binomial estimate under a simple random-sample assumption. It does not capture nonresponse, coverage or mode bias. The high-volume significance test is likewise a nominal comparison; non-significance is not evidence that the shares were identical.
  • NPS measures stated willingness to recommend, not actual retention. Neither wave comparison nor the associations between Q3 scores, pricing views and service complaints establish what caused a change in recommendation or what an intervention would achieve.
  • Open answers are voluntary and can carry multiple themes. The 66.7% is 30 of 45 detractor comments, not 30 of all 120 detractors. The up-to-two Q7 selections measure relative priority among the options shown, not adoption, willingness to pay or expected return.

Next steps

  • experimentPilot tracking milestones and proactive collection and delivery delay alerts with hauliers that can supply reliable events. Compare against a holdout or staggered rollout.
  • researchAudit Ireland-bound jobs against comparable non-Ireland jobs and follow up with affected accounts. Break down results by haulier, service level, shipment type and lane.
  • experimentCompare actual bills under old and new pricing, then test a clearly priced optional subscription against the current offer. Include lower-volume accounts, customers whose bills increased and, where reachable, recently lapsed customers.
  • monitorEstablish a consistent survey series and report operational and behavioural measures alongside NPS. If the historical trend is important, run a same-period mode comparison using a common sampling frame.
  • decisionDefer mid-volume account-manager hiring. If escalations remain a problem after the tracking pilot, test named escalation ownership with a small set of high-need accounts.

Open-text themes it coded

Fast, easy quoting or booking 79Missing, vague or stale tracking and poor shipment visibility 42Late or missed collections or deliveries 29Per-shipment fee, its cost or preference for the old flat charge 21Support is hard to reach or slow to respond 6Damaged freight or difficult claims 5Works adequately or uneventfully, with no specific reason singled out 17Too new to the platform or too little usage to judge 5

The survey it planned

2 screening questions and 8 questions, as the model wrote them.

  1. S1

    Are you the main contact at your business for booking shipments through Quaymark?

    One answer
    • Yes
    • No

    Continues if Yes

  2. S2

    Has your business booked at least one shipment through Quaymark in the past three months?

    One answer
    • Yes
    • No
    • I'm not sure

    Continues if Yes

  3. Q1

    How likely are you to recommend Quaymark to someone at another business that ships goods?

    Scale

    Scale 1-11: 0 — Not at all likely to 10 — Extremely likely

  4. Q2

    What's the main reason for the score you gave?

    Open text
  5. Q3

    What else, if anything, influenced your score? Include anything that influenced it positively or negatively. Select all that apply. If nothing else influenced your score, select only 'Nothing else'.

    Any that apply
    • Getting a quote and booking a shipment
    • The price of transport
    • Quaymark's booking fees or subscription charges
    • Collection and delivery reliability
    • Being able to see where a shipment is
    • Being told about delays
    • Getting help when I need it
    • Something else
    • Nothing else
  6. Q4

    Which one of these changes, if any, would be most useful to your business?

    One answer
    • More accurate, up-to-date shipment tracking online
    • Automatic notifications when a shipment is expected to be delayed
    • A named account manager you can contact for help
    • None of these would be useful
    • I'm not sure
  7. Q5

    Before Quaymark changed its pricing in March 2026, did your business use Quaymark on a monthly subscription?

    One answer
    • Yes
    • No
    • I'm not sure
  8. Q6

    Compared with the previous monthly subscription, how well does the current per-shipment booking fee work for your business?

    One answer
    • Much better
    • Somewhat better
    • About the same
    • Somewhat worse
    • Much worse
    • I don't have enough information to compare

    Routing Shown only if Q5 is: Yes

  9. Q7

    Quaymark used to charge a monthly subscription instead of a per-shipment booking fee. The subscription started at £25 a month for the smallest accounts. The current booking fee is £3.50 per shipment, with a lower fee for accounts that book 100 or more shipments a month. Which pricing option would your business prefer?

    One answer
    • Bring back the previous monthly subscription
    • Keep the current per-shipment booking fee
    • No preference
    • It depends on what my business would pay under each option
    • I'm not sure
  10. Q8

    If Quaymark could make only one of these changes, which should it prioritise for your business?

    One answer
    • Bring back the previous monthly subscription instead of charging a booking fee per shipment
    • Improve online shipment tracking and send automatic notifications about expected delays
    • Provide a named account manager you can contact for help
    • None of these
    • I'm not sure
Sample plan

Email a unique survey link to the main booking contact at each of the 4,720 active UK business accounts. Accept one response per eligible account. Aim for about 520 completes, with up to eight questions after screening and no more than six minutes; make Q2 optional and verify timing in a pilot. Use booking records to assign accounts to fewer than 20, 20–99, or 100 or more shipments per month. Use the Q1 size definition if documented and reproducible; otherwise use average monthly bookings over the three most recent complete calendar months. Before fieldwork, calculate the required completes from the actual eligible account count N_h in each band. For a 95% margin of error of ±5 percentage points at a worst-case 50% promoter share under simple random sampling without replacement, the benchmark per band is ceil(384.16 × N_h / (N_h + 383.16)). If the bands are roughly equal in size, that is about 309 completes per band, or 927 total; 520 evenly spread would give about ±7 points per band even under those sampling assumptions. Do not promise ±5 points at 520. Agree before launch whether to raise the target or accept wider intervals. Invite all accounts, do not cap responses by band, and target reminders to nonrespondents in bands furthest below their targets. The finite-population calculation is a planning benchmark, not a guarantee for a voluntary email survey. Randomise the substantive options in Q3, Q4, Q7 and Q8, keeping residual options such as 'None' and 'I'm not sure' last.