Gemini 3.5 Flash Lite: Why Quaymark's customer score fell

The decision: whether to reverse the per-shipment fee and bring back the subscription, invest in shipment tracking and proactive delay alerts, or add account managers for mid-sized customers. The brief and the data →

Verdict: ✗ Not sound
Research score: 71 of 100
Analysis rating: 662
Head to head, this task: won 3 of 10

Why it is not sound

The survey

Ignored the precision the brief asked for, which the sample cannot deliver

The analysis

Got at least one set number wrong
Reported a material finding the data does not support
Missed a trap: Accounts booking fewer than 20 shipments a month are 61% of Q3 respondents against 30% in Q1 and 40% of active accounts, and they score lowest (Q3 NPS +5). Re-weighted to Q1's mix the Q3 NPS is +20.3, and to the active-account mix +17.8, so much of the 11-point fall is a change in who answered.
Missed a trap: The commercial director believes the fee caused the fall. Among detractors who explained their score, 69% mention tracking or delays and 18% the fee; only 39% of all respondents say the fee is worse than the subscription (49% of small accounts, 15% of the largest).
Missed a trap: Respondents who ship to Ireland (base 14) are nearly all detractors (NPS -93).
Missed a trap: The 69% figure is a share of the 45 detractors who answered the open question, not of all respondents or all customers.
Proposed a mistaken next step: Change Ireland hauliers, pricing or service on the strength of the survey alone

Findings the judges found unsupported

Customer advocacy has dropped notably since the per-shipment fee was introduced, falling from +24 to +13 overall, driven entirely by smaller accounts.The headline difference is not established as a genuine decline in advocacy or an effect of the fee. Much of it reflects the changed account-size mix, with a possible method effect. It is also not driven entirely by the smallest accounts: NPS among accounts booking 20–99 shipments fell from +26 to +21.
Customers shipping to the Republic of Ireland are experiencing critical service failures, resulting in near-total brand rejection.Fourteen respondents cannot establish critical service failures or near-total rejection among Ireland customers generally. Recommend scores also do not establish the operational cause of dissatisfaction.
Operational visibility and punctuality are the primary drivers of dissatisfaction among detractors.The operational priority is supported, but the reported 85% is not. Tracking or delays are mentioned by 69% of the 45 detractors who answered the open question, not by 85% on a base of 120.

Scorecard

Numbers 10 of 12 right

Question		Its answer	True
What is the Q3 2026 wave's NPS across all respondents (percentage scoring 9-10 minus percentage scoring 0-6), unweighted?	✓	12.9	12.9
Weighted to the shipments-a-month mix of all 4,720 active accounts, what is the Q3 2026 wave's NPS?	✓	16.5	17.8
Re-weighted to the shipments-a-month mix of the Q1 2026 wave's respondents, what is the Q3 2026 wave's NPS?	✓	20.3	20.3
Among respondents who scored 0-6 and answered the open question, what percentage of answers mention tracking (not being able to see where shipments are) or late or missed collections or deliveries?	✗	85%	68.9%
How many of the open-text answers mention the per-shipment fee or its cost?	✗	16	21
What is the 95% margin of error, in percentage points, on the Q3 2026 wave's unweighted share of respondents scoring 9-10?	✓	4.1 points	4.12 points
Among accounts booking 100 or more shipments a month, is the change in the share scoring 9-10 between the Q1 2026 and Q3 2026 waves statistically significant at the 95% level? (1 for yes, 0 for no)	✓	No	No
What percentage of the accounts invited to the Q3 2026 wave completed it?	✓	11%	11%
Among Q3 2026 respondents booking fewer than 20 shipments a month, what percentage said the per-shipment fee is worse for their business than the subscription?	✓	49.2%	49.2%
Among accounts booking fewer than 20 shipments a month, by how many points did NPS fall between the Q1 2026 and Q3 2026 waves?	✓	5.2 points	3.9 points
How many Q3 2026 respondents who send shipments to the Republic of Ireland scored 0-6?	✓	13	13
Weighted to the shipments-a-month mix of all 4,720 active accounts, about how many accounts would score 0-6?	✓	989	1,001

Traps in the data 4 of 8 handled

✓The Q1 wave (+24) was phone interviews by account managers; the Q3 wave (+13) was online. Interviewer-administered scores, especially when the interviewer is the customer's own account manager, run higher, so part of the fall may be the change of method rather than a change in loyalty.
✗Accounts booking fewer than 20 shipments a month are 61% of Q3 respondents against 30% in Q1 and 40% of active accounts, and they score lowest (Q3 NPS +5). Re-weighted to Q1's mix the Q3 NPS is +20.3, and to the active-account mix +17.8, so much of the 11-point fall is a change in who answered.
✗The commercial director believes the fee caused the fall. Among detractors who explained their score, 69% mention tracking or delays and 18% the fee; only 39% of all respondents say the fee is worse than the subscription (49% of small accounts, 15% of the largest).
✗Respondents who ship to Ireland (base 14) are nearly all detractors (NPS -93).
✓45% of small accounts say they would be likely to choose a monthly subscription if offered: a stated intention about a hypothetical, with no price given.
✓521 of 4,720 invited accounts completed (about 11%); unhappy customers and small accounts may be more likely to respond online.
✓Among accounts booking 100 or more a month (bases 98 and 48), the promoter share moved from 49% to 52%, NPS +38 to +38: not significant.
✗The 69% figure is a share of the 45 detractors who answered the open question, not of all respondents or all customers.

The analysis

Recommendation

Do not execute a blanket reversal of the per-shipment fee, as mid-to-large shippers largely find it acceptable or better. Instead, introduce a dual-pricing model (offering the subscription as an optional alternative for smaller, high-frequency low-volume shippers), and concurrently invest heavily in live shipment tracking and proactive delay alerts.

What would change it

Definitive financial modeling proving that bringing back a mandatory subscription preserves overall platform revenue and margins better than capturing per-shipment volume from high-usage accounts, alongside a larger audited sample confirming system-wide operational failure in Ireland.

Findings

Customer advocacy has dropped notably since the per-shipment fee was introduced, falling from +24 to +13 overall, driven entirely by smaller accounts.

High confidenceBase 521

Q1 NPS was +24 (Base=412) vs Q3 NPS of +13 (Base=521). For accounts booking fewer than 20 shipments, NPS dropped from +9 to +5.
Smaller shippers are strongly opposed to the per-shipment fee, which they find significantly costlier than the previous monthly subscription.

High confidenceBase 319

49.2% of respondents booking fewer than 20 shipments a month state the per-shipment fee is worse for their business (Q5, Base=319).
Customers shipping to the Republic of Ireland are experiencing critical service failures, resulting in near-total brand rejection.

Low confidenceBase 14

Among respondents shipping to Ireland, 92.9% gave a detractor score of 0-6, yielding an NPS of -93 (Base=14).
Operational visibility and punctuality are the primary drivers of dissatisfaction among detractors.

High confidenceBase 120

85% of detractors mentioning specific issues in open text cite either tracking opacity or late/missed collections and deliveries.

Caveats

The Q1 2026 wave was conducted via phone by account managers, whereas the Q3 2026 wave was fielded online, introducing potential mode effects that may exaggerate the drop in NPS.
The Q3 survey achieved a low response rate (11.0%) with a small sample of active users, and Ireland respondents represent an extremely small base (n=14), requiring cautious interpretation.
Self-selection bias may be present, as dissatisfied users could be more motivated to complete the survey.

Next steps

experimentIntroduce a tiered pricing structure offering a monthly subscription alternative targeted specifically at low-volume shippers.
decisionAudit carrier performance for Republic of Ireland shipments and temporarily suspend or restructure the Ireland route offerings.
monitorPrioritise product development sprints on live map tracking and automated SMS/email delay notifications.

Open-text themes it coded

per_shipment_fee_negative 18tracking_and_visibility_issues 34late_or_missed_collections_deliveries 21booking_speed_and_ease 75customer_support_pain_points 6freight_damage_and_claims_issues 4

The survey it planned

1 screening question and 9 questions, as the model wrote them.

S1
Have you personally booked at least one pallet or parcel shipment through Quaymark on behalf of your business in the last three months?
One answer
- Yes
- No
Continues if Yes
Q1
Roughly how many shipments does your business book through Quaymark in an average month?
One answer
- Fewer than 20 shipments a month
- 20 to 99 shipments a month
- 100 or more shipments a month
Q2
How likely is it that you would recommend Quaymark to a colleague or business partner?
Scale

Scale 1-11: 0 - Not at all likely to 10 - Extremely likely
Q3
What is the primary reason for the score you just gave Quaymark?
Open text
Q4
In March 2026, Quaymark replaced its monthly subscription fee with a per-shipment booking fee (£3.50 per shipment, with discounts for accounts booking 100+ monthly). How do you feel about this pricing model compared to the previous monthly subscription?
One answer
- Much prefer the current per-shipment fee
- Somewhat prefer the current per-shipment fee
- No preference / Neutral
- Somewhat prefer the previous monthly subscription
- Much prefer the previous monthly subscription
Judges: assumptive
It assumes respondents can compare the current fee with the previous subscription. The screener also admits customers who joined after the change and may have no experience or knowledge of the subscription.
It assumes every respondent knew the previous monthly subscription. Accounts that started after March 2026 never used it, and there is no 'did not use the subscription' or 'don't know' option.
Q5
What is your main reason for your preference regarding the pricing model?
Open text

Judges: assumptive
It assumes a pricing preference even when the respondent selected 'No preference / Neutral' in Q4.
It is shown to everyone, so it assumes a preference. Respondents who chose 'No preference / Neutral' in Q4, or never used the subscription, are asked for the reason behind a preference they don't have.
Q6
Over the past three months, have you contacted Quaymark customer support for any assistance or to raise an issue?
One answer
- Yes
- No
Q7
How satisfied were you with the support you received when you contacted Quaymark?
Scale

Scale 1-5: 1 - Very dissatisfied to 5 - Very satisfied

Routing Shown only if Q6 is: Yes
Q8
Which of the following potential platform improvements would provide the most value to your business? Please rank them from 1 (most important) to 3 (least important).
Ranking
- Enhanced real-time shipment tracking
- Proactive delay alerts and notifications
- Dedicated account management support
Q9
Is there anything else you would like to share with the Quaymark team regarding our pricing, features, or service?
Open text

Sample plan

The survey will be emailed to the main booking contact at all 4,720 active UK business accounts that have booked at least one shipment in the last three months, pulled directly from the customer database. Based on a target of approximately 520 completes to achieve a +/- 5% margin of error at a 95% confidence level across each of the three account size segments (fewer than 20, 20-99, and 100+ shipments a month), we will monitor completions and send one reminder email to non-responders after 4 days.