Research

Promptfoo Alternatives: When to Stay, and When a Browser Test Fits

Last checked 18 November 2026.

Promptfoo is the tool many developers reach for first when they want LLM tests in a repo. In March 2026 it agreed to be acquired by OpenAI, which has led some teams to look at alternatives. This guide sets out what changed, what Promptfoo does better than a browser tool, who should stay with it, and where Spring Prompt fits.

Already have a Promptfoo config? Load it into a Prompt test and read every model's answers side by side. Test your prompt →

What's changed

On 9 March 2026, Promptfoo announced that it had “agreed to be acquired by OpenAI”. The post said Promptfoo “will remain open source” and “will continue to support a diverse range of providers and models”. Its GitHub README now reads: “Promptfoo is now part of OpenAI. Promptfoo remains open source and MIT licensed.”

OpenAI's own migration guide for its Evals platform, which shuts on 30 November 2026, is called “Moving from OpenAI Evals to Promptfoo”. We covered that in our OpenAI Evals guide.

Sources: Promptfoo's announcement, dated 9 March 2026; the Promptfoo repository on GitHub and Promptfoo's pricing page, both checked 18 November 2026.

Nothing we found says the open-source tool is closing or changing licence. If the reason you're looking is that a model developer now owns your test tool, that's a judgement for you; the tool itself works as before.

What Promptfoo does better

  • Tests live in your repo. You write a YAML config of prompts, providers and tests with assertions, and run it from a terminal. Its README says evals run locally, so your prompts go only to the model providers you call.
  • A wide set of assertions. Its assertions docs list deterministic checks such as contains, regex, is-json, and your own javascript or python functions, plus model-graded ones such as llm-rubric, factuality and similar. Any of them can be negated with not-.
  • CI on every pull request. The promptfoo/promptfoo-action@v1 GitHub Action runs a before-and-after evaluation when a pull request edits a prompt and posts the results to the pull request.
  • Red teaming. Promptfoo also tests for prompt injection, jailbreaks, data leaks and insecure tool use. The free Community version includes up to 10,000 probes a month. Spring Prompt doesn't do security testing.
  • Free and self-run. The Community version is free, and you can run it on your own machines. Spring Prompt is a hosted service; you can't self-host it.

Who should stay with Promptfoo

If your team writes tests as code, wants them in the repo next to the prompts, needs custom assertions in JavaScript or Python, or wants red teaming, Promptfoo is a good fit and we'd keep using it. It's also the cheaper choice for large suites: you pay only your model providers.

Where Spring Prompt fits

Spring Prompt is for the parts that are slow to do from a config file: choosing a model, rewriting a prompt, and showing the results to someone who doesn't read YAML.

  • Prompt test. Paste a prompt in a browser. We draft test cases and checks for you to edit, run the models you pick side by side, check every answer against your rules, and show the cost and time of each answer next to whether it passed. A test takes up to 24 cases and 10 checks.
  • Live. A two-line package, springprompt on PyPI and npm, that copies a sample of your app's real model calls so you can test them on other models. See the Live docs.
  • Check before merge. Our GitHub Action, incremento/springprompt-action@v1, runs a changed prompt or model on a saved test's cases and checks and gives a pass, review or fail verdict. See how it works.
  • Independent benchmarks. Our benchmarks score current models on business tasks and show the real answers, which helps you pick which models to put in a test.

Promptfoo vs Spring Prompt

PromptfooSpring Prompt
Where you workConfig file and terminal; local web viewerBrowser, plus MCP and an API
Writing testsYou write them in YAMLWe draft cases and checks from your prompt; you edit them
ChecksMany assertion types, including your own codePhrases, patterns, JSON, word counts and judged checks; up to 10
Test sizeAs large as you likeUp to 24 cases
Pull request checkspromptfoo/promptfoo-action@v1incremento/springprompt-action@v1
Red teamingYesNo
Real trafficNoLive package samples real calls
Licence and hostingMIT; run it yourselfHosted only
CostCommunity version free; you pay providersPay for the tests you run; free trial credit

Move a Promptfoo config in, and back out

In a new Prompt test, choose Import from my app, then paste or upload promptfooconfig.yaml or its JSON version. It loads with no model call:

  • Your prompts, including chat prompts written as a JSON list of messages.
  • The first provider's model. If the config compares several, the import lists them so you can pick.
  • Each test's vars as a case. A variable given as a list expands into one case per value, as Promptfoo does.
  • Assertions we can run as checks: contains, icontains and their -all, -any and not- forms, regex, not-regex, starts-with, equals, is-json, llm-rubric, g-eval, model-graded-closedqa and factuality. An assertion every test shares becomes a check on every answer; one that only some tests make goes into those cases' notes.

The import lists what it left out: javascript and python assertions, prompts or tests in file:// references, and scenarios. Provider settings, where configs often keep keys and endpoints, are never copied, and API keys anywhere in the file are removed before it's stored.

Going the other way, a finished Prompt test exports as a Promptfoo config, a springprompt.json or CSV files. Judged checks become llm-rubric, phrases become icontains-all and not-icontains-any, and JSON checks become is-json. So you can draft and compare in the browser, then commit the config and run it in CI with Promptfoo's action.

Coming from somewhere else? We have guides for Braintrust, LangSmith and Langfuse.

Bring your config, or start from a prompt. Run the models you're choosing between on the same cases and read every answer side by side. Test your prompt →

Quick answers

Is Promptfoo still open source after the OpenAI deal?

Yes. Promptfoo announced on 9 March 2026 that it had agreed to be acquired by OpenAI and said it would remain open source. Its GitHub README now says “Promptfoo is now part of OpenAI. Promptfoo remains open source and MIT licensed.” We checked on 18 November 2026.

Is Promptfoo free?

The open-source Community version is free and, according to Promptfoo's pricing page, includes all of its evaluation features and up to 10,000 red-teaming probes a month. Enterprise and On-Premise plans have custom pricing.

Does Promptfoo still support models from other labs?

Its March announcement said Promptfoo would continue to support a diverse range of providers and models, and its README lists OpenAI, Anthropic, Azure, Bedrock and Ollama among them.

Can I import a Promptfoo config into Spring Prompt?

Yes. Paste or upload promptfooconfig.yaml (or the JSON version) under Import from my app. Prompts, the first provider's model, tests and the assertions we can run come in without a model call. JavaScript and Python assertions, file:// references and scenarios are listed as left out.

Can I get my tests back out as a Promptfoo config?

Yes. A finished Prompt test exports as a Promptfoo config, a springprompt.json or CSV files, and the config reads back into a new Prompt test as it is.

More research
All posts →