Benchmarks / SurveyBench / Why Quaymark's customer score fell / minimax/minimax-m3.1-flash:free

Measured by Spring Prompt

minimax/minimax-m3.1-flash:free: Why Quaymark's customer score fell

The decision: whether to reverse the per-shipment fee and bring back the subscription, invest in shipment tracking and proactive delay alerts, or add account managers for mid-sized customers. The brief and the data →

Verdict
✗ Not sound
Research score
29 of 100
Analysis rating
441
Head to head, this task
won 0 of 7

Why it is not sound

The survey

  • A leading question: Q10

The analysis

  • Produced no usable analysis

Scorecard

The analysis

This model produced no usable analysis for this task.

The survey it planned

3 screening questions and 10 questions, as the model wrote them.

  1. Q1

    Is the business you use Quaymark for based in the United Kingdom?

    One answer
    • Yes
    • No

    Continues if Yes

  2. Q2

    Are you the main person responsible for booking shipments with Quaymark for this business?

    One answer
    • Yes
    • No

    Continues if Yes

  3. Q3

    In the last 3 months, have you personally booked or arranged at least one shipment collection with Quaymark?

    One answer
    • Yes
    • No

    Continues if Yes

  4. Q4

    How likely are you to recommend Quaymark to another business like yours? Use a scale from 0 to 10.

    Scale

    Scale 1-11: Not at all likely to Extremely likely

  5. Q5

    In a few words, what is the main reason you are so likely to recommend Quaymark?

    Open text

    Routing Shown only if Q4 is: 9; 10

  6. Q6

    In a few words, what is the main reason you were not more likely to recommend Quaymark?

    Open text

    Routing Shown only if Q4 is: 0; 1; 2; 3; 4; 5; 6; 7; 8

  7. Q7

    Which of the following aspects of your experience most influenced the score you just gave? Select up to three.

    Any that apply
    • Booking fees and value for money
    • How quickly and easily I can book a shipment
    • The prices I am quoted
    • The accuracy and usefulness of shipment tracking
    • How Quaymark communicates about delays
    • The quality of customer support
    • The reliability of collections and deliveries
    • The range of partner hauliers available
  8. Q8

    Since March 2026, how satisfied have you been with the support you have received from Quaymark?

    One answer
    • Very dissatisfied
    • Dissatisfied
    • Neither satisfied nor dissatisfied
    • Satisfied
    • Very satisfied
    • I have not contacted Quaymark for support since March 2026
  9. Q9

    Overall, how does the per-shipment booking fee you currently pay compare in value for your business with the previous monthly subscription?

    One answer
    • Much better value
    • Somewhat better value
    • About the same value
    • Somewhat worse value
    • Much worse value
    • I do not know enough to make this comparison
  10. Q10

    Which pricing model would you prefer for your business?

    One answer
    • A monthly subscription
    • The current per-shipment booking fee
    • A lower per-shipment booking fee
    • A combination of a monthly subscription and per-shipment fees
    • No preference between pricing models
    • Not sure
    Judges: leading

    Only the per-shipment model is offered with an explicitly lower price. This makes the alternatives unequal and steers preference towards a cheaper per-shipment option rather than measuring pricing-model preference alone.

    The options are unbalanced. Only the per-shipment model gets a 'lower' price option, while the subscription has no price at all. Anyone would pick a cheaper version of a model, so the answers mix preference for a pricing model with preference for paying less. Splitting per-shipment support across two options also distorts the comparison between models.

  11. Q11

    Thinking specifically about the March 2026 change from a monthly subscription to per-shipment booking fees, how has that change affected your likelihood of recommending Quaymark?

    One answer
    • Made me much more likely to recommend Quaymark
    • Made me somewhat more likely to recommend Quaymark
    • Has not affected my likelihood of recommending Quaymark
    • Made me somewhat less likely to recommend Quaymark
    • Made me much less likely to recommend Quaymark
    • I cannot tell whether it has affected my likelihood of recommending Quaymark
    Judges: assumptive

    It assumes the respondent was a customer before the March 2026 change. Accounts active only in the last three months may have joined after the change. 'I cannot tell' does not cover people who never had the subscription, so they have no suitable answer.

    Assumes the respondent was an active customer during the March 2026 change and experienced the transition.

  12. Q12

    Which of the following changes would make a meaningful difference to your overall Quaymark experience? Select all that apply.

    Any that apply
    • More accurate, real-time shipment tracking
    • Proactive alerts when a collection or delivery is likely to be delayed
    • A named account manager who can help with bookings and issues
    • Pricing that better suits my business (for example, a lower per-shipment fee, a subscription, or a combination)
    • None of these
  13. Q13

    Which one of those changes would make the biggest difference to your overall Quaymark experience?

    One answer
    • More accurate, real-time shipment tracking
    • Proactive alerts when a collection or delivery is likely to be delayed
    • A named account manager who can help with bookings and issues
    • Pricing that better suits my business (for example, a lower per-shipment fee, a subscription, or a combination)

    Routing Shown only if Q12 is: More accurate, real-time shipment tracking; Proactive alerts when a collection or delivery is likely to be delayed; A named account manager who can help with bookings and issues; Pricing that better suits my business (for example, a lower per-shipment fee, a subscription, or a combination)

Sample plan

Build a census invitation frame from the customer database: the 4,720 UK business accounts with at least one Quaymark shipment in the preceding three months. Email the named main booking contact for every account once, with up to two reminders at approximately 7 and 14 days; permit one response per account and do not substitute panel respondents. Classify account size from CRM data using actual average monthly shipments over the three complete months before fielding: fewer than 20, 20-99, and 100 or more; do not ask respondents to estimate this. Use an adaptive stratified design: treat approximately 520 completes as the first-stage target, then send targeted reminders to under-quota account-size groups and continue until each group has at least 385 effective completes. This minimum is based on the worst-case 50% promoter share: n = 1.96^2 x 0.5 x 0.5 / 0.05^2 = 384.16, rounded to 385, and therefore requires about 1,155 completes in total before any design-effect inflation. Weight oversampled groups back to the account-size distribution of the full 4,720-account frame and calculate design-adjusted 95% confidence intervals. The stated 520-completion total and the requirement of +/-5 percentage points for all three groups are mathematically incompatible: even an equal split of 520 gives about 173 per group and a worst-case margin of about +/-7.5 points. If 520 is a hard cap, the board must accept the wider achievable precision or extend fielding; the survey must not report that the +/-5-point requirement has been met when it has not.