Using Helicone as a proxy allows your LiteLLM requests to flow through Helicone’s infrastructure before reaching the LLM provider. This enables powerful features like:
Request Caching - Save money by reusing identical requests
Rate Limiting - Control your API usage
Retries - Automatically retry failed requests
Advanced Logging - Capture detailed metrics and request/response payloads
Unlike callback-based integration, proxy integration works at the network level and can provide more functionality for supported providers.
response = litellm.completion( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "how does a court case get to the Supreme Court?"}], metadata={ "Helicone-Property-Hello": "World" })print(response)
Gemini integration requires a special approach because of how the Vertex AI APIs are structured. We need to use a monkey patch with LiteLLM’s Router to correctly route requests through Helicone.
Gemini’s API structure differs from OpenAI’s, and LiteLLM’s default proxy handling doesn’t properly route Gemini requests through Helicone. The patch modifies LiteLLM’s internal URL handling to correctly work with Helicone’s Gemini proxy endpoints.
import osHELICONE_API_KEY = os.getenv("HELICONE_API_KEY")GEMINI_API_KEY = os.getenv("GEMINI_API_KEY")if not HELICONE_API_KEY or not GEMINI_API_KEY: print("Error: HELICONE_API_KEY and GEMINI_API_KEY must be set.")
For LLM providers beyond OpenAI, Azure, and Gemini, the integration approach varies by provider:
Each provider has a specific Helicone proxy URL format (e.g., OpenAI uses oai.helicone.ai/v1)
Some providers require the Helicone-Target-Url header, while others don’t
Copy
Ask AI
# Example pattern (will vary by provider)litellm.api_base = "<provider-specific-helicone-endpoint>"litellm.headers = { "Helicone-Auth": f"Bearer {os.getenv('HELICONE_API_KEY')}", # Additional headers may be required depending on the provider}
Consult the Helicone documentation for provider-specific proxy endpoints and required headers. If your provider isn’t explicitly supported, reach out to the Helicone team for guidance.For callback-based integration with LiteLLM, see our LiteLLM Callbacks Integration guide.