Experiments is a spreadsheet-like experience designed to tune your LLM prompts for production.
Experiment
tab. Click on start from a prompt
, then choose the desired prompt version.
Requests
tab. Open the desired request, then click on the experiments icon. You should see an experiment being generated for the prompt associated with this request.
Experiment
tab, then click on Start from scratch
. A helper prompt will be generated for you; you can edit it by clicking on the cell.
Add column
and select a prompt that you want to fork from.
{{ input_name }}
to add input variables in your prompt. These input variables will appear in the Inputs
column.
Add manual inputs
: Manually enter values for each input variable you defined.Select an input set
: Select production request data that matches the same prompt ID.Random prod
: Randomly select any number of production request data. We wrote about why this approach is recommended.Add from a dataset
: Use datasets you’ve created in Helicone to test your prompt. This allows you to leverage curated collections of test cases for thorough prompt evaluation.Import from CSV
: Import data from a CSV file that contains columns matching your input variables. When importing, you can preview the data to ensure it matches your expected format. The preview shows the first few rows of your CSV data and highlights which columns will be used as input variables.Toggle on `Show scores`.
Under the dropdown, select 'Create new custom evaluators'
Create the evaluator
Run Evaluator
Run Evaluators
to see the scores in graph.(Optional) View score breakdown
Need more help?