Promptfoo Alternatives: When to Stay, and When a Browser Test Fits
Last checked 18 November 2026.
Promptfoo is the tool many developers reach for first when they want LLM tests in a repo. In March 2026 it agreed to be acquired by OpenAI, which has led some teams to look at alternatives. This guide sets out what changed, what Promptfoo does better than a browser tool, who should stay with it, and where Spring Prompt fits.
Already have a Promptfoo config? Load it into a Prompt test and read every model's answers side by side. Test your prompt →
What's changed
On 9 March 2026, Promptfoo announced that it had “agreed to be acquired by OpenAI”. The post said Promptfoo “will remain open source” and “will continue to support a diverse range of providers and models”. Its GitHub README now reads: “Promptfoo is now part of OpenAI. Promptfoo remains open source and MIT licensed.”
OpenAI's own migration guide for its Evals platform, which shuts on 30 November 2026, is called “Moving from OpenAI Evals to Promptfoo”. We covered that in our OpenAI Evals guide.
Sources: Promptfoo's announcement, dated 9 March 2026; the Promptfoo repository on GitHub and Promptfoo's pricing page, both checked 18 November 2026.
Nothing we found says the open-source tool is closing or changing licence. If the reason you're looking is that a model developer now owns your test tool, that's a judgement for you; the tool itself works as before.
What Promptfoo does better
- Tests live in your repo. You write a YAML config of prompts, providers and tests with assertions, and run it from a terminal. Its README says evals run locally, so your prompts go only to the model providers you call.
- A wide set of assertions. Its assertions docs list deterministic checks such as
contains,regex,is-json, and your ownjavascriptorpythonfunctions, plus model-graded ones such asllm-rubric,factualityandsimilar. Any of them can be negated withnot-. - CI on every pull request. The
promptfoo/promptfoo-action@v1GitHub Action runs a before-and-after evaluation when a pull request edits a prompt and posts the results to the pull request. - Red teaming. Promptfoo also tests for prompt injection, jailbreaks, data leaks and insecure tool use. The free Community version includes up to 10,000 probes a month. Spring Prompt doesn't do security testing.
- Free and self-run. The Community version is free, and you can run it on your own machines. Spring Prompt is a hosted service; you can't self-host it.
Who should stay with Promptfoo
If your team writes tests as code, wants them in the repo next to the prompts, needs custom assertions in JavaScript or Python, or wants red teaming, Promptfoo is a good fit and we'd keep using it. It's also the cheaper choice for large suites: you pay only your model providers.
Where Spring Prompt fits
Spring Prompt is for the parts that are slow to do from a config file: choosing a model, rewriting a prompt, and showing the results to someone who doesn't read YAML.
- Prompt test. Paste a prompt in a browser. We draft test cases and checks for you to edit, run the models you pick side by side, check every answer against your rules, and show the cost and time of each answer next to whether it passed. A test takes up to 24 cases and 10 checks.
- Live. A two-line package,
springprompton PyPI and npm, that copies a sample of your app's real model calls so you can test them on other models. See the Live docs. - Check before merge. Our GitHub Action,
incremento/springprompt-action@v1, runs a changed prompt or model on a saved test's cases and checks and gives a pass, review or fail verdict. See how it works. - Independent benchmarks. Our benchmarks score current models on business tasks and show the real answers, which helps you pick which models to put in a test.
Promptfoo vs Spring Prompt
| Promptfoo | Spring Prompt | |
|---|---|---|
| Where you work | Config file and terminal; local web viewer | Browser, plus MCP and an API |
| Writing tests | You write them in YAML | We draft cases and checks from your prompt; you edit them |
| Checks | Many assertion types, including your own code | Phrases, patterns, JSON, word counts and judged checks; up to 10 |
| Test size | As large as you like | Up to 24 cases |
| Pull request checks | promptfoo/promptfoo-action@v1 | incremento/springprompt-action@v1 |
| Red teaming | Yes | No |
| Real traffic | No | Live package samples real calls |
| Licence and hosting | MIT; run it yourself | Hosted only |
| Cost | Community version free; you pay providers | Pay for the tests you run; free trial credit |
Move a Promptfoo config in, and back out
In a new Prompt test, choose Import from my app, then paste or upload promptfooconfig.yaml or its JSON version. It loads with no model call:
- Your prompts, including chat prompts written as a JSON list of messages.
- The first provider's model. If the config compares several, the import lists them so you can pick.
- Each test's
varsas a case. A variable given as a list expands into one case per value, as Promptfoo does. - Assertions we can run as checks:
contains,icontainsand their-all,-anyandnot-forms,regex,not-regex,starts-with,equals,is-json,llm-rubric,g-eval,model-graded-closedqaandfactuality. An assertion every test shares becomes a check on every answer; one that only some tests make goes into those cases' notes.
The import lists what it left out: javascript and python assertions, prompts or tests in file:// references, and scenarios. Provider settings, where configs often keep keys and endpoints, are never copied, and API keys anywhere in the file are removed before it's stored.
Going the other way, a finished Prompt test exports as a Promptfoo config, a springprompt.json or CSV files. Judged checks become llm-rubric, phrases become icontains-all and not-icontains-any, and JSON checks become is-json. So you can draft and compare in the browser, then commit the config and run it in CI with Promptfoo's action.
Coming from somewhere else? We have guides for Braintrust, LangSmith and Langfuse.
Bring your config, or start from a prompt. Run the models you're choosing between on the same cases and read every answer side by side. Test your prompt →
Quick answers
Is Promptfoo still open source after the OpenAI deal?
Yes. Promptfoo announced on 9 March 2026 that it had agreed to be acquired by OpenAI and said it would remain open source. Its GitHub README now says “Promptfoo is now part of OpenAI. Promptfoo remains open source and MIT licensed.” We checked on 18 November 2026.
Is Promptfoo free?
The open-source Community version is free and, according to Promptfoo's pricing page, includes all of its evaluation features and up to 10,000 red-teaming probes a month. Enterprise and On-Premise plans have custom pricing.
Does Promptfoo still support models from other labs?
Its March announcement said Promptfoo would continue to support a diverse range of providers and models, and its README lists OpenAI, Anthropic, Azure, Bedrock and Ollama among them.
Can I import a Promptfoo config into Spring Prompt?
Yes. Paste or upload promptfooconfig.yaml (or the JSON version) under Import from my app. Prompts, the first provider's model, tests and the assertions we can run come in without a model call. JavaScript and Python assertions, file:// references and scenarios are listed as left out.
Can I get my tests back out as a Promptfoo config?
Yes. A finished Prompt test exports as a Promptfoo config, a springprompt.json or CSV files, and the config reads back into a new Prompt test as it is.