Overview

The Gateway serves as a unified entry point for all traffic, regardless of your provider. It enables you to dispatch requests to any provider through a single endpoint. This provides you with the advantage of utilizing all of Helicone’s features such as Caching, Monitoring, Rate Limiting, Vaults, Feedback, and many more for any provider.

Simple Integration

This is achieved by allowing you to determine the destination of your request by specifying the provider in the URL path. For instance, if you wish to send a request to LemonFox, you would send a request to https://gateway.helicone.ai instead of https://api.lemonfox.ai, but you would specify the provider and the endpoint in the URL path within the headers of your request.

Ex 1

api.groq.com/openai/v1" -> gateway.helicone.ai/openai/v1

Header: Helicone-Target-Url: https://api.groq.com

Ex 2

api.lemonfox.ai/v1 -> gateway.helicone.ai/v1

Header: Helicone-Target-Url: https://api.lemonfox.ai

How to Use the Gateway

import openai

openai.api_base = "https://gateway.helicone.ai"

# Create the completion request

openai.ChatCompletion.create(
  model="[DEPLOYMENT]",
  messages=[{"role": "User", "content": "Say hi!"}],
  headers={
    "Helicone-Auth": "Bearer [HELICONE_API_KEY]",
    "Helicone-Target-Url": "https://api.lemonfox.ai",
    "Helicone-Target-Provider": "LemonFox",
  }
)

Examples

Current gateway request

curl --request POST \
  --url https://gateway.ai.cloudflare.com/v1/[ACCOUNT-ID]/[GATEWAY-ID]/openai/chat/completions \
  --header 'Authorization: Bearer <API_KEY>' \
  --header 'Content-Type: application/json' \
  -d ' {
    "model": "gpt-3.5-turbo",
    "messages": [
      {
        "role": "user",
        "content": "What is Cloudflare?"
      }
    ]
  }
'

Simply change gateway.ai.cloudflare.com to gateway.helicone.ai and add the following headers

curl --request POST \
  --url https://gateway.helicone.ai/v1/[ACCOUNT-ID]/[GATEWAY-ID]/openai/chat/completions \
  --header 'Helicone-Auth: Bearer <Helicone_Auth>' \
  --header 'Helicone-Target-Url: https://gateway.ai.cloudflare.com' \
  --header 'Authorization: Bearer <API_KEY>' \
  --header 'content-type: application/json' \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {
        "role": "user",
        "content": "What is Cloudflare?"
      }
    ]
  }
'

curl --request POST \
  --url https://gateway.helicone.ai/v1/chat/completions \
  --header 'Authorization: Bearer <TOGETHERAI_API_KEY>' \
  --header 'Content-Type: application/json' \
  --header 'Helicone-Auth: Bearer <HELICONE_API_KEY>' \
  --header 'Helicone-Target-Provider: Together-AI' \
  --header 'Helicone-Target-Url: https://api.together.xyz' \
  --data '{
  "model": "meta-llama/Llama-2-70b-chat-hf",
  "max_tokens": 512,
  "messages": [
    {
      "role": "user",
      "content": "What is the capital of France?"
    }
  ]
}'

Gemini Pro

curl --request POST \
  --url https://gateway.helicone.ai/v1/projects/<PROJECT_ID>/locations/<LOCATION>/publishers/google/models/gemini-pro:streamGenerateContent \
  --header 'Authorization: Bearer <GEMINI_API_KEY>' \
  --header 'Content-Type: application/json' \
  --header 'Helicone-Auth: Bearer <HELICONE_API_KEY>' \
  --header 'Helicone-Target-Url: https://<LOCATION>-aiplatform.googleapis.com' \
  --data '{
  "contents": {
    "role": "user",
    "parts": {
      "text": "Which theaters in Mountain View show Barbie movie?"
    }
  },
}'

Gemini Pro Vision

curl --request POST \
  --url https://gateway.helicone.ai/v1/projects/<PROJECT_ID>/locations/<LOCATION>/publishers/google/models/gemini-pro-vision:streamGenerateContent \
  --header 'Authorization: Bearer <GEMINI_API_KEY>' \
  --header 'Content-Type: application/json' \
  --header 'Helicone-Auth: Bearer <HELICONE_API_KEY>' \
  --header 'Helicone-Target-Url: https://<LOCATION>-aiplatform.googleapis.com' \
  --data '{
  "contents": {
    "role": "user",
    "parts": [
      {
        "text": "What is the capital of France?"
      }
    ]
  }
}'

Approved Domains

Provider	Domain Name	Cost Support	Dedicated Domain
OpenAI	`api.openai.com`	✅	`oai.helicone.ai`
Anthropic	`api.anthropic.com`	✅	`anthropic.helicone.ai`
Together-AI	`api.together.xyz`	✅	`together.helicone.ai`
OpenRouter	`openrouter.ai`	✅	`openrouter.helicone.ai`
Fireworks	`api.fireworks.ai`	✅	`fireworks.helicone.ai`
Azure	`openai.azure.com`	✅	❌
Groq	`api.groq.com`	❌	`groq.helicone.ai`
Deepinfra	`api.deepinfra.com`	❌	`deepinfra.helicone.ai`
Anyscale	`api.endpoints.anyscale.com`	❌	❌
Cloudflare	`gateway.ai.cloudflare.com`	❌	❌
LemonFox	`api.lemonfox.ai`	❌	❌
Perplexity	`api.perplexity.ai`	❌	❌
Mistral	`api.mistral.ai`	❌	❌

Unapproved Domains

To protect our community from potential threats, we have certain restrictions for unapproved domains. To ensure a safe internet environment, you can use any unapproved domain, but you will be limited to the following:

10,000 requests per day
1 request per second

The integration is the same as the approved domains, but you will need to add the following header:

--header 'Helicone-Target-Url: https://[DOMAIN_NAME]'

Automated Mappers (“Model is pending mapping”)

Helicone will automatically determine how to map any unsupported model that you throw at it using our automated mapper. Helicone intelligently determines the schema of the model and maps it to the appropriate schema for the provider. This allows you to use any model you want without having to wait for us to manually map it.

Mappings are on a specialized job that runs every 24 hours, so your UI may not be mapped immediately. If you would like to expedite the mapping, please contact us at engineering@helicone.ai or connect with us on Discord!

How to Get a Domain Approved or Mapper Expedited

To get a domain approved, please contact us at engineering@helicone.ai or connect with us on Discord!

Subject: Domain Approval Request

Body: Is this a personal deployment? If not, what is the name of the company? What is the domain name?

If it is a personal deployment, please provide proof you own the domain by updating a text record with the following value: hconeai-verification=true

Getting Started

Dev Integrations

Other Integrations

Features

References

Gateway Integration

Overview

Simple Integration

Ex 1

Ex 2

How to Use the Gateway

Examples

Approved Domains

Unapproved Domains

Automated Mappers (“Model is pending mapping”)

How to Get a Domain Approved or Mapper Expedited

Getting Started

Dev Integrations

Other Integrations

Features

References

​Overview

​Simple Integration

​Ex 1

​Ex 2

​How to Use the Gateway

​Examples

​Approved Domains

​Unapproved Domains

​Automated Mappers (“Model is pending mapping”)

​How to Get a Domain Approved or Mapper Expedited

Overview

Simple Integration

Ex 1

Ex 2

How to Use the Gateway

Examples

Approved Domains

Unapproved Domains

Automated Mappers (“Model is pending mapping”)

How to Get a Domain Approved or Mapper Expedited