Benchmarks / SurveyBench / Why Quaymark's customer score fell / Gemini 4 Neon

Measured by Spring Prompt

Gemini 4 Neon: Why Quaymark's customer score fell

The decision: whether to reverse the per-shipment fee and bring back the subscription, invest in shipment tracking and proactive delay alerts, or add account managers for mid-sized customers. The brief and the data →

Verdict
✗ Not sound
Research score
28 of 100
Analysis rating
629
Head to head, this task
won 0 of 6

Why it is not sound

The survey

  • A double-barrelled question: Q3
  • A double-barrelled question: Q4

The analysis

  • Produced no usable analysis

Scorecard

The analysis

This model produced no usable analysis for this task.

The survey it planned

2 screening questions and 10 questions, as the model wrote them.

  1. S1

    Are you the main person responsible for booking shipments on Quaymark for your business?

    One answer
    • Yes, I am the main person who books shipments on Quaymark
    • Yes, I am one of the main people who book shipments on Quaymark
    • No, I only book occasionally or someone else is the main contact
    • No, I am not involved in booking shipments

    Continues if Yes, I am the main person who books shipments on Quaymark; Yes, I am one of the main people who book shipments on Quaymark

  2. S2

    Has your business booked at least one pallet or parcel shipment through Quaymark in the last three months?

    One answer
    • Yes
    • No
    • Not sure

    Continues if Yes

  3. Q1

    On a scale from 0 to 10, how likely are you to recommend Quaymark to a colleague or another business?

    Scale

    Scale 1-11: 0 - Not at all likely to 10 - Extremely likely

  4. Q2

    What is the main reason for the score you gave?

    Open text
  5. Q3

    Which of the following do you see as Quaymark's biggest strengths? Please select up to three.

    Any that apply
    • Speed of getting a quote
    • Ease of booking online
    • Choice of partner hauliers
    • Reliability of collections and deliveries
    • Online shipment tracking
    • Pricing and fee structure
    • Customer support and account management
    • Other
    • None of these
    Judges: double-barrelled

    The option 'Customer support and account management' combines distinct services. Selecting it does not establish whether account management, customer support, or both are strengths.

    'Customer support and account management' asks about two distinct functions in one option.

  6. Q4

    Which of the following issues, if any, has your business experienced with Quaymark in the last three months? Please select all that apply.

    Any that apply
    • Higher overall booking costs
    • Confusion or frustration with the fee structure
    • Limited or inaccurate online shipment tracking
    • Delays to collections or deliveries without advance warning
    • Slow or unhelpful customer support
    • No dedicated contact to help manage our account
    • Website or technical problems
    • Other
    • None of these
    Judges: double-barrelled

    The option 'Slow or unhelpful customer support' combines response speed and helpfulness in one answer, so these distinct problems cannot be distinguished.

    'Higher overall booking costs' gives no reference point. It is unclear whether it means higher than before the March change, higher than expected or higher than competitors. Customers who joined after March have no 'before' to compare with.

  7. Q5

    When did your business start booking shipments through Quaymark?

    One answer
    • Before March 2026
    • In or after March 2026
    • Not sure
  8. Q6

    In March 2026, Quaymark replaced its monthly subscription with a per-shipment booking fee. Compared with the previous monthly subscription, how do you view the per-shipment booking fee?

    Scale

    Scale 1-5: 1 - Much worse than the subscription to 5 - Much better than the subscription

    Routing Shown only if Q5 is: Before March 2026

  9. Q7

    Going forward, which pricing structure would better suit your business?

    One answer
    • A per-shipment booking fee (£3.50 per shipment, reduced for 100 or more shipments a month)
    • A monthly subscription (from £25 a month, with no per-shipment booking fee)
    • No preference between the two
    • Not sure
  10. Q8

    Please rank the following potential improvements to Quaymark from 1 (most important to your business) to 5 (least important to your business).

    Ranking
    • Switching from a per-shipment booking fee to a monthly subscription
    • More detailed, real-time online shipment tracking
    • Proactive alerts when a collection or delivery is delayed
    • A dedicated account manager for your business
    • Faster customer support when problems occur
  11. Q9

    Roughly how many shipments does your business book through Quaymark in a typical month?

    One answer
    • Fewer than 20 shipments a month
    • 20 to 99 shipments a month
    • 100 or more shipments a month
  12. Q10

    If Quaymark could make just one change over the next six months to better support your business, what would it be and why? (Optional)

    Open text
Sample plan

Census email invitation to the main booking contact of all 4,720 active UK business accounts (booked at least one shipment in the last three months), using unique single-use survey links so CRM account-size tier, tenure, and recent support-ticket flags can be appended for weighting and non-response checks while keeping individual answers confidential from account managers. Randomise option order in Q3, Q4 (keeping Other and None of these fixed at the end), Q7 (first two options), and Q8 to prevent order bias. Field for 14 days with two email reminders to non-responders. CRITICAL PRECISION FLAG: Expecting ~520 completes (~11% response rate) will NOT achieve the board's target of +/-5 percentage points at 95% confidence for each of the three account-size tiers simultaneously. At 95% confidence (worst-case p=0.50), +/-5pp requires 385 completes per segment before finite population correction (FPC)—potentially 750 to 950+ total completes even after FPC, depending on exact CRM tier populations. With 520 total completes, natural proportional response (typically skewed toward <20 accounts) will leave 20-99 and especially 100+ segments with much wider margins of error (likely +/-8 to +/-15pp). Do not apply upper quotas that turn respondents away; instead set minimum stratum targets based on exact CRM tier counts with FPC, and actively boost response in the 20-99 and 100+ tiers via targeted reminders and a modest neutral incentive (e.g., charitable donation per completion or prize draw) without involving account managers. If final completes remain around 520, report exact FPC-adjusted confidence intervals per tier and apply post-stratification weighting by CRM account-size and tenure distribution for overall company-level metrics, while auditing non-response bias against recent support-complaint rates.