Overview

FAQ Overview

Helicone OSS LLM Observability

Helicone is the open-source LLM observability platform for developers to monitor, debug, and improve **production-ready** applications.

Introduction

Introduction to Helicone - Open Source LLM Observability

Caching

LLM Caching

LLM Cache - Helicone OSS LLM Observability

Custom Properties

Custom Properties - Helicone OSS LLM Observability

Sessions

Sessions - Helicone OSS LLM Observability

User Metrics

User Metrics - Helicone OSS LLM Observability

Omit Logs

Omit Logs - Helicone OSS LLM Observability

Eval Scores

User Feedback

Datasets & Fine-Tuning

Helicone's AI Gateway is the open-source, lightweight Rust-based solution for intelligent routing to 100+ language models.

Get started with Helicone AI Gateway in 1 minute

Quickstart

Build your first custom router with load balancing, caching, and rate limiting in **5 minutes**

Create Your First Router

Choose the deployment method that works best for your infrastructure and requirements

Deploy Your Gateway

Authentication integration with the Helicone platform

Secure Your Gateway

Monitor your AI Gateway requests with Helicone

Enable Observability

Observability with Helicone

Complete reference for configuring your LLM Gateway

Configuration

Configuration Reference

Learn how Helicone calculates the cost per request for nearly all models, including both streamed and non-streamed requests. Detailed explanations and examples provided.

Cost Calculation

How We Calculate Cost

How We Calculate Cost - Helicone OSS LLM Observability

Helicone ensures top-tier data security and privacy through our SOC2 compliant cloud solution, with options for enhanced control and data ownership.

Data Security & Privacy

Data Security & Privacy - Helicone OSS LLM Observability

Comprehensive guide to all Helicone headers. Learn how to access and implement various Helicone features through custom request headers.

Header Directory

Helicone Header Directory

Helicone Header Directory - Open Source LLM Observability

Helicone minimizes latency for your LLM applications using Cloudflare's global network. Detailed benchmarking results and performance metrics included.

Latency

Latency Impact

Latency Impact - Helicone OSS LLM Observability

Understanding Helicone's open-source status and how to contribute

Open Source

Compare Helicone's Proxy and Async integration methods. Understand the features, benefits, and use cases for each approach to choose the best fit for your LLM application.

Proxy vs Async

Proxy vs Async Integration

Proxy vs Async Integration - Helicone OSS LLM Observability

Helicone ensures high availability for your LLM applications using Cloudflare's global network. Learn about our deployment practices and how we maintain reliability.

Reliability

Availability and Reliability

Availability - Helicone OSS LLM Observability

Retrieve all requests visible in the request table at Helicone.

Get Requests

Get Requests - Helicone OSS LLM Observability

Get Requests (Point Queries)

Get Requests (Point Queries) - Helicone OSS LLM Observability

Retrieve a single request visible in the request table at Helicone.

Get Single Request

Get Single Request - Helicone OSS LLM Observability

Get Requests by IDs

Get Requests by IDs - Helicone OSS LLM Observability

Submit Feedback

Submit Feedback - Helicone OSS LLM Observability

Create or update a property of a specific request.

Upsert Request Property

Upsert Request Property - Helicone OSS LLM Observability

Submit assets for a specific request. - If you don't know what this is, you probably don't need this.

Submit Request Assets

Submit Request Assets - Helicone OSS LLM Observability

Submit Score

Submit Score - Helicone OSS LLM Observability

Query Prompts

Get Prompts - Helicone OSS LLM Observability

Query Prompt

Get Prompt - Helicone OSS LLM Observability

Delete Prompt

Create Prompt

Modify the label of a specific prompt version

Edit Prompt Version Label

Modify the template of a specific prompt version

Edit Prompt Version Template

Retrieve a subversion of a specific prompt version

Get Prompt Version Subversion

Get Subversion of Prompt Version - Helicone OSS LLM Observability

Promote Prompt Version

Search through inputs of a specific prompt version

Query Prompt Version Inputs

Retrieve experiments associated with a prompt

Get Prompt Experiments

Search through versions of a specific prompt

Query Prompt Versions

Get Prompt Versions - Helicone OSS LLM Observability

Retrieve details of a specific prompt version

Get Prompt Version

Delete Prompt Version

Compile Prompt

Query Prompt Template

Get an overview of aggregated user metrics

Query User Metrics Overview

Search and filter through user-specific metrics

Query User Metrics

Retrieve user data based on specified user IDs and time filters

Get User Data

Get User Data - Helicone OSS LLM Observability

Search and filter through evaluation results

Query Evaluations

Get Evaluation Scores

Create a new evaluation for a specific request

Create Evaluation

Analyze distribution of evaluation scores

Query Score Distributions

Get Webhooks

Create Webhook

Delete Webhook

Create Evaluator

Get Evaluator

Modify an existing evaluator configuration

Update Evaluator

Remove an existing evaluator configuration

Delete Evaluator

Search and filter through evaluator configurations

Query Evaluators

Retrieve experiments associated with a specific evaluator

Get Evaluator Experiments

Retrieve all online evaluators for a specific evaluator

Get Online Evaluators

Add a new online evaluator to an existing evaluator

Create Online Evaluator

Remove an online evaluator from a specific evaluator configuration

Delete Online Evaluator

Initialize a new empty experiment using v2 API

Create Empty Experiment (v2)

Create a new experiment based on an existing request using v2 API

Create from Request (v2)

Create Experiment from Request (v2)

Create a new experiment with initial configuration using v2 API

Create New Experiment (v2)

List Experiments

Retrieve detailed information about a specific experiment

Get Experiment Details

Associate a prompt version with an experiment

Add Prompt Version

Add Prompt Version to Experiment

Retrieve all prompt versions used in an experiment

Get Experiment Prompt Versions

Retrieve input keys used in an experiment

Get Experiment Input Keys

Manually add a new row to an experiment using v2 API

Add Manual Row (v2)

Insert multiple rows into an experiment using v2 API

Batch Insert Rows (v2)

Update rows in an experiment using v2 API

Update Rows (v2)

Run a hypothesis for an experiment using v2 API

Run Hypothesis (v2)

Retrieve all evaluators associated with an experiment using v2 API

Get Experiment Evaluators (v2)

Add evaluators to an experiment using v2 API

Add Evaluators (v2)

Add Experiment Evaluators (v2)

Remove an evaluator from an experiment using v2 API

Delete Experiment Evaluator (v2)

Execute evaluators for an experiment using v2 API

Run Evaluators (v2)

Run Experiment Evaluators (v2)

Check if evaluators should be run for an experiment

Check Evaluator Run Status

Retrieve scoring metrics for a specific prompt version in an experiment

Get Experiment Scores

Retrieve a specific score for an experiment request

Get Experiment Score

Initialize a new empty experiment template

Create Empty Experiment

Create a new table for storing experiment data

Create New Table

Create New Experiment Table

Query Experiment Table

Search and filter experiment table metadata

Query Table Metadata

Search and filter through experiment tables

Query Tables

Query Experiment Tables

Create Table Cell

Modify the content of a specific cell in an experiment table

Update Experiment Table Cell

Create Table Column

Create New Table Row

Delete Experiment Table Row

Insert multiple rows into an experiment table at once

Getting Started

Integrations

Tracing

Prompts & Evals

AI Gateway

References

FAQ Overview

Categories

Using Helicone

Security and Compliance

Key Concepts

Fine-tuning