Find the cheapest model that passes your prompt.
Paste your prompt. We draft realistic test cases and checks, run several models side by side, check every answer against your rules, and show quality next to cost.
- Leaderboards don't know your task. This tests your prompt, on your cases.
- Cheaper only counts if it still passes. You see which answers pass, and what each one cost.
- Know before a model change reaches your users, not after one of them finds it.
Request an invite
We're opening the Prompt test a few people at a time, with test credits to start.
From a prompt to a decision.
- 01
Paste your prompt
Any prompt your product sends to a model. Parts that change, such as
{{product}}, become what each test case fills in. - 02
Check the cases and checks
We draft realistic test cases and the checks a good answer must pass. You can edit any of them, or paste your own examples.
- 03
Run models side by side
Several models from different labs answer every case. Each answer is checked against your rules.
- 04
Pick, or improve the prompt
See the cheapest model that passes. Then rewrite the prompt from what failed, or let "Evolve it" search for a better version.
What you get
- Every answer from every model, next to the checks it passed and failed
- The cheapest model that passes, with cost and time per answer
- Must-pass checks and stretch checks reported apart, so one average doesn't hide a failure
- A rewritten prompt, run on the same cases so you can compare it with yours
- Cases and checks you can run again when a new model comes out
How the judge is kept honest
- Rules such as word counts, required phrases and valid JSON are checked in code, the same way every time
- Everything else is a plain yes-or-no question. The judge quotes the words that decide it, and fails what it's unsure of
- Steady judging asks three times and takes the majority. Split votes are shown, not hidden
- You can pick a judge from a different lab than the models you're testing
- Disagree with a verdict and you see how often the judge agrees with you. A check can be rewritten from your verdicts
Our public benchmarks check their judges too. Read the methodology.
A better prompt, tested fairly.
A prompt tuned for one model can make the others look worse than they are. "Evolve it" tries many versions of your prompt for a model, learning from why answers failed. It keeps some test cases back, and offers a new prompt only if it clearly wins on cases it never saw. The most it can spend is shown before it starts.
Questions
Why not just pick from a leaderboard?
A leaderboard tells you how a model does on someone else's tasks. Use our benchmarks as a shortlist, then test the shortlist on your own prompt.
Do I need my own test data?
No. We draft test cases from your prompt, so you can start without exporting anything from production. If you have real examples, you can paste them in.
Which models can I test?
Hundreds of text models, from Anthropic, Google, OpenAI, DeepSeek, Mistral and many more. You can test up to six in one go.
What happens to my prompt?
It's stored privately and sent only to the models you choose and the models that judge and draft. Tests are deleted after 90 days, or sooner if you ask. The privacy policy has the details.
What does it cost?
Invites come with test credits to start. Every test shows what it cost, and a quick test stops at a set amount of credit.
Can you run the test for us?
Yes. If you'd rather we set it up and run it on your real data, talk to us about a custom evaluation.
Test your own prompt.
Tell us what you're building, and we'll send you an invite.