Deploy the AI Gateway using Docker for easy containerized deployment to any cloud provider or local environment.

​
Quick Start

1

Configure environment

Create a .env file with your provider API keys. You can use our template as a starting point:

# Download the template
curl -o .env https://raw.githubusercontent.com/Helicone/ai-gateway/main/.env.template

Then edit the .env file with your actual API keys:

# Required for server binding
AI_GATEWAY__SERVER__ADDRESS=0.0.0.0

# Add your provider API keys
OPENAI_API_KEY=sk-your-openai-key
ANTHROPIC_API_KEY=sk-ant-your-anthropic-key
# Add other provider keys as needed

The AI_GATEWAY__SERVER__ADDRESS=0.0.0.0 is required for Docker to work properly.

2

Run the container

Start the AI Gateway container:

docker run -it --rm --name ai-gateway -p 8080:8080 --env-file .env helicone/ai-gateway:main

The Gateway will be running on http://localhost:8080 with these routes:

  • /ai - Unified API that works out of the box
  • /router/{router-name} - Custom routing with load balancing
  • /{provider-name} - Direct provider access
3

Test your deployment

Make a test request to verify everything works:

import { OpenAI } from "openai";

const openai = new OpenAI({
  baseURL: "http://localhost:8080/ai",
  apiKey: "placeholder-key", // Gateway handles API keys
});

const response = await openai.chat.completions.create({
  model: "openai/gpt-4o-mini",
  messages: [{ role: "user", content: "Hello from Docker!" }],
});

console.log(response);

You should see a response from the AI model! 🎉

​
Using a Configuration File

For custom routing and advanced features, create a config.yaml file:

1

Create config file

Create a config.yaml file with your routing configuration:

routers:
  production:
    load-balance:
      chat:
        strategy: latency
        providers:
          - openai
          - anthropic
2

Mount config and run

Run the container with your configuration file:

docker run -it --rm --name ai-gateway -p 8080:8080 --env-file .env \
  -v $(pwd)/config.yaml:/helicone-config/config.yaml \
  helicone/ai-gateway:main ai-gateway -c /helicone-config/config.yaml
3

Test your router

Test your custom router:

import { OpenAI } from "openai";

const openai = new OpenAI({
  baseURL: "http://localhost:8080/router/production",
  apiKey: "placeholder-key", // Gateway handles API keys
});

const response = await openai.chat.completions.create({
  model: "openai/gpt-4o-mini",
  messages: [{ role: "user", content: "Hello from my custom router!" }],
});

console.log(response);

​
Next Steps

​
Secure Your Gateway

Secure Your Gateway

Learn how to secure your gateway with authentication and authorization

​
Deploy to the Cloud

The containerized AI Gateway can be deployed to various cloud platforms:

AWS ECS

One-click deployment to AWS ECS with Terraform

Fly.io

Simple container deployment platform

Porter

Container registry and app platform

Kubernetes

Deploy using Helm charts for production workloads