Humanloop Alternatives: Where to Test Prompts After the Shutdown

Humanloop's platform closed on 8 September 2025. If you used it to manage prompts and run evaluations, you've had to find somewhere else to do that. This guide sets out what happened and when, what Humanloop's export tool saved, and the main routes for testing prompts now.

Want to test prompts in a browser? Paste a prompt and we'll draft the cases and checks, or bring in your own examples. Test your prompt →

What happened, and when

The dates for the email, billing and sunset come from Humanloop's guide, “How to export your data from Humanloop”, checked 8 November 2026. The announcement that “the Humanloop team is joining Anthropic” is on Humanloop's homepage, checked 8 November 2026. The page isn't dated; the Internet Archive's copy from 14 August 2025 is the earliest we found that shows it.

In its announcement, the team thanked its customers and said it was proud of what it had built: “the first development platform for LLM applications”.

1. Check what you saved

You can't export anything from Humanloop now. Its guide says all data became permanently inaccessible after 8 September 2025. If you ran its export tool before then, here's what you should have.

The tool is the open-source humanloop-export script. According to its README, each run made a dated folder with:

• Files and versions. Your prompts, tools, datasets, flows, agents and evaluators, with each version's definition, as JSON.
• Datapoints. Your dataset rows, as JSONL. Humanloop's docs describe each datapoint as inputs (the values for a prompt's {{variables}}), optional chat messages, and an optional target: the output you wanted.
• Evaluations. Each evaluation's metadata, statistics and runs, as JSON.
• Logs, as JSONL, only if you turned them on. They were off by default.

If you didn't export, start from what you still have: the prompts in your codebase, any test files, and a few real inputs your app handles.

2. Choose where to test next

There's no single right answer. It depends on who runs the tests, and whether you also need logs from production.

• The tools Humanloop named. Its guide suggested surveying the options, and named Langfuse and Braintrust as two it recommended in particular.
• Langfuse is an open-source platform for tracing, prompt management, datasets and experiments, with LLM judges among its evaluators. Its core features are MIT-licensed, and you can self-host it or use its cloud service. It suits teams who want production logs and tests in one place.
• Promptfoo is an open-source command-line tool for testing LLM apps. You write a config file listing prompts, providers and tests with assertions, and run it from a terminal or in CI. It suits developers who want tests in the repo next to the code.
• Your own script is fine for a handful of cases. It gets harder once you need several models, judged checks and a record of what changed.
• A browser tool suits people who want to test without setting up a project. Spring Prompt's Prompt test is one: paste a prompt, and it drafts test cases and checks for you, runs the models you pick side by side, and shows every answer with its cost. It runs models from many developers. It doesn't keep production logs, so it's for testing, not monitoring.

You can also use more than one. A config in CI catches regressions on every change; a browser test is quicker for choosing a model or rewriting a prompt.

3. Bring your prompts into a Prompt test

We don't have a Humanloop importer, but the pieces carry over.

• The prompt. In a new Prompt test, choose Import from my app and paste the prompt text, or the code that sends it. A model finds the system and user prompts and turns the parts filled in at runtime into {{variables}}. Humanloop templates already use {{variables}}, so the names can stay the same.
• The cases. Under Paste examples as JSON, paste a list with one object per case, keyed by your variable names. Put the output you wanted in an expected field and it becomes the case's notes. For a datapoint, that means its inputs as the keys and its target as expected.
• Or a dataset file. Rows shaped like OpenAI Evals data rows ({"item": {...}}, one per line) load straight in, with no model call. The fields become variables, and an answer field such as target becomes each case's expected answer.

The full list of files that load without a model call, including Promptfoo configs, is in the import table in our OpenAI Evals guide.

A Prompt test takes up to 12 cases and 10 checks, so a large dataset is cut to the first ones, and the import tells you. Pick the cases that matter most: the awkward inputs, not twelve easy ones. Evaluators written as code don't carry over; describe what each one checked and we can draft a check to replace it. API keys in anything you paste are removed before it's stored.

4. Re-check the model while you're there

The models your prompts used in mid-2025 may have changed or retired since. Our model retirements page lists upcoming dates with each official replacement and its source.

Moving your tests and your model at once makes it hard to tell which change moved a result. If you can, carry the tests over first, run them on the model you have now, then try the alternatives.

Bring your cases, or start from a prompt. Run the models you're choosing between on the same inputs and read every answer side by side. Test your prompt →

Quick answers