Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.helicone.ai/llms.txt

Use this file to discover all available pages before exploring further.

We are deprecating the Experiments feature and it will be removed from the platform on September 1st, 2025.

​
Feature Highlight

  • Create as many prompt versions as you like, without impacting production data.
  • Evaluate the outputs of your new prompt (and have data to back you up 📈).
  • Save cost by testing on specific datasets and making fewer calls to providers like OpenAI. 🤑

​
Running your first prompt experiment

To start an experiment, first, go to the Prompts tab and select a prompt.
1

Click `Start Experiment`

On the top right, click Start Experiment.
Start button in the Prompts tab for initiating an experiment in Helicone.
2

Select the base prompt

Select a base prompt and click Continue. You can edit the prompt in the next step.
To run an experiment on the production prompt, look for the production tag.
Selecting a base prompt to start an experiment in Helicone.
3

Edit the prompt

Your changes will not affect the original prompt, but rather create a new one to test your experiment on.
Editing a prompt without affecting the original prompt in production.
4

Configure your experiment

Select the dataset, model and provider keys.
To run your experiment on a random dataset, click Generate random dataset. We will pick up to 10 random data from your existing requests.
Configuring an experiment with a different dataset, model, and provider keys in Helicone.
5

Confirm and run

The Diff Viewer compares your new prompt to the base prompt that you selected.
Confirming changes to your prompt in Helicone's Diff Viewer before running an experiment.
6

View outputs

Once the experiment is finished, click on it to see a list of inputs and the associated outputs from the base prompt and the experiment.
Comparing the outputs of an experiment compared to the original prompt in Helicone.