LiteLLM Integration
Connect Helicone with LiteLLM, a unified interface for multiple LLM providers. Standardize logging and monitoring across various AI models with simple callback or proxy setup.
LiteLLM is a model I/O library to standardize API calls to Azure, Anthropic, OpenAI, etc. Hereβs how you can log your LLM API calls to Helicone from LiteLLM.
Note: Custom
Properties
are available in metadata
starting with LiteLLM version 1.41.23
.
Approach 1: Use Callbacks
1 line integration
Add HELICONE_API_KEY
to your environment variables.
export HELICONE_API_KEY=sk-<your-api-key>
# You can also set it in your code (See below)
Tell LiteLLM you want to log your data to Helicone
litellm.success_callback=["helicone"]
Complete code
from litellm import completion
## set env variables
os.environ["HELICONE_API_KEY"] = "your-helicone-key"
os.environ["OPENAI_API_KEY"], os.environ["COHERE_API_KEY"] = "", ""
# set callbacks
litellm.success_callback=["helicone"]
#openai call
response = completion(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Hi π - i'm openai"}],
metadata={
"Helicone-Property-Hello": "World"
}
)
#cohere call
response = completion(
model="command-r",
messages=[{"role": "user", "content": "Hi π - i'm cohere"}],
metadata={
"Helicone-Property-Hello": "World"
}
)
Approach 2: [OpenAI + Azure only] Use Helicone as a proxy
Helicone provides advanced functionality like caching, etc. which they support for Azure and OpenAI.
If you want to use Helicone to proxy your OpenAI/Azure requests, then you can -
2 line integration
litellm.api_url = ""https://oai.helicone.ai/v1"" # set the base url
litellm.headers = {"Helicone-Auth": f"Bearer {os.getenv('HELICONE_API_KEY')}"} # set your headers
Complete code
import litellm
from litellm import completion
litellm.api_base = "https://oai.helicone.ai/v1"
litellm.headers = {"Helicone-Auth": f"Bearer {os.getenv('HELICONE_API_KEY')}"}
response = litellm.completion(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "how does a court case get to the Supreme Court?"}],
metadata={
"Helicone-Property-Hello": "World"
}
)
print(response)
Feel free to check it out and tell us what you think π