Community

GitHub

Helicone

Use Cases

Sign In

Discord

Github

Azure OpenAI

helicone

 Hundreds of organizations leverage Helicone's monitoring and tooling to make their Large-Language Model operations more efficient.  We experienced first-hand the pain of managing internal tools and monitoring for LLM's at scale - so we built Helicone to solve these problems for you.

Introduction

Helicone Documentation

Helicone is a sophisticated platform designed to streamline the use and management of large language models (LLMs) for developers and enterprises. 

Overview

Automate the monitoring and caching of LLM calls in your CI pipelines

Github Actions

Leverage custom segments to gain insights into costs and user behavior

Effortlessly track your environments with Helicone.

Environment Tracking

Helicone provides an efficient platform for identifying and rectifying errors in your code, offering insights into their occurrence.

Debugging

ETL / Data Extraction

Data Autonomy signifies the empowerment of individuals and organizations to retain full control over their data. Discover how Helicone's open-source software bolsters this essential facet of digital sovereignty.

Data Ownership and Autonomy

A quick start guide to get you up and running with Helicone in a few simple steps.

Quick Start

Determine when you should use a proxy or async function in Helicone.

Proxy vs Async

A quick start guide to get you up and running with a self-hosted instance of Helicone.

Self Deploy

Cloud Deploy

Self Deploy Docker Compose

The optimal method to integrate with Helicone is through our Gateway. This API provides a seamless interface for sending requests and responses to Helicone.

Gateway Integration

With the instability of some APIs and the similarity of many providers, we have developed a unified gateway. This gateway allows you to utilize any provider through a single endpoint.

Gateway Fallback (Beta)

OpenAI

Anthropic

Anyscale

LiteLLM

Custom Model (Beta)

Leverage the power of headers in your proxy request with Helicone to unlock advanced features and capabilities.

Predetermine Helicone's request ID for asynchronous utilization in other tasks

Request ID Predefinition

A Job is an assembly of nodes that can be interconnected and evaluated as clusters. This is particularly beneficial for assessing chains and monitoring your system holistically.

Utilizing Helicone to monitor our agent that generates courses

Course Creation Example

Helicone allows you to tag requests with any information you choose to segment, analyze, and visualize by.

Custom Properties

Get user metrics like number of requests, costs, and activity

User Metrics

Helicone now incorporates a feature that enables users to provide feedback on LLM responses, indicating their level of satisfaction with the provided response.

User Feedback

With Helicone's Vault, manage your provider API keys securely and simplify distribution using Helicone proxy keys.

Key Vault

DEPRECATED: Templated prompts are a way to separate the parameters from the prompt text

Templated Prompts

Reduce latency and save costs by caching on the edge

Caching

Intelligently retry requests to overcome rate limits and overloaded servers

Retries

Enforcing custom API usage restrictions with rate limiting

Custom Rate Limits

Omit Logs

Streaming

Webhooks allow you to build or set up integrations which subscribe to certain events on Helicone.ai. When one of those events is triggered, we'll send a HTTP POST payload to the webhook's configured URL. Webhooks can be used to update an external issue tracker, trigger CI builds, update a backup mirror, or even deploy to your production server.

Webhooks

Automatically capture your logs without changing any code.

Man-in-the-Middle Proxy

Use Helicone's GraphQL API to export all your data

Export Script

Getting Started

Python Example

We take the security of your OpenAI requests extremely seriously

How encryption works

We know latency of your Large-Language Model usage is important to your application's success, that's why we designed Helicone to impact latency as little as possible.

Latency Impact

An explanation of our process for calculating cost per request

Welcome

Getting Started

Integrations

Helicone Headers

Features

Tools

GraphQL API

FAQ

Azure OpenAI

Python

Node.js

Python w/package

LangChain JS

LangChain

cURL