Deploy the AI Gateway to Fly.io for a simple, scalable cloud deployment with automatic scaling and global distribution.

Security Notice: Fly.io deployments are publicly accessible. You must enable authentication to prevent unauthorized access to your provider API keys. This guide includes authentication setup as a required step.

​
Quick Start

1

Install Fly CLI

Install the Fly.io CLI tool:

# macOS (with Homebrew)
brew install flyctl

# macOS/Linux (without Homebrew)
curl -L https://fly.io/install.sh | sh

# Windows
pwsh -Command "iwr https://fly.io/install.ps1 -useb | iex"

If you encounter any issues with installation, refer to the official Fly.io installation guide.

2

Get your Helicone API key

The AI Gateway configuration has authentication enabled by default to secure your public deployment. You’ll need a Helicone API key to authenticate requests.

Create your Helicone API key:

  1. Go to Helicone Settings
  2. Click “Generate New Key”
  3. Copy the key (starts with sk-helicone-)
  4. Save it securely - you’ll need it for deployment
3

Get the fly.toml configuration

Download the official fly.toml configuration from our GitHub repository:

curl -o fly.toml https://raw.githubusercontent.com/Helicone/ai-gateway/main/fly.toml

This configuration file contains all the necessary settings for deploying the AI Gateway to Fly.io.

4

Launch and deploy

Launch your app with Fly.io:

fly launch

This command will:

  • Automatically prompt you to log in if needed
  • Create the app with a unique name
  • Set up your deployment
  • Deploy your AI Gateway

The deployment will complete, but you’ll need to add your API keys next.

5

Configure authentication & API keys

Enable authentication to secure your public deployment, then add your provider API keys:

# REQUIRED: Enable authentication
fly secrets set HELICONE_CONTROL_PLANE_API_KEY="sk-helicone-your-key-from-step-2"

# Provider API keys (add the ones you need)
fly secrets set OPENAI_API_KEY="<your_openai_key>"
fly secrets set ANTHROPIC_API_KEY="<your_anthropic_key>"
fly secrets set GEMINI_API_KEY="<your_gemini_key>"

Use the Helicone API key you created in step 2. This enables secure authentication for your gateway.

6

Redeploy with secrets

After adding your environment variables, redeploy to apply them:

fly deploy

Your AI Gateway is now fully configured and ready to use!

​
Test Your Deployment

Once deployed, test your AI Gateway with authentication:

import { OpenAI } from "openai";

const openai = new OpenAI({
  baseURL: "https://your-app-name.fly.dev/ai",
  apiKey: "sk-helicone-your-api-key", // Your Helicone API key
});

const response = await openai.chat.completions.create({
  model: "openai/gpt-4o-mini",
  messages: [{ role: "user", content: "Hello from Fly.io!" }],
});

console.log(response);

Replace sk-helicone-your-api-key with your actual Helicone API key for secure access to your gateway.

​
Using a Configuration File

The default fly.toml includes a basic router configuration, but you can customize it for your specific needs.

1

Create a custom config file

Create a config.yaml file in your project directory:

server:
  address: 0.0.0.0
  port: 8080

routers:
  my-router:
    load-balance:
      chat:
        strategy: latency
        providers:
          - openai
          - anthropic

helicone:
  authentication: true
  observability: false # Set to true to enable observability
2

Update fly.toml

Modify the [[files]] section in your fly.toml to use your local config file:

[[files]]
  guest_path = '/app/config/config.yaml'
  local_path = './config.yaml'

Replace the existing raw_value line with local_path pointing to your config file.

3

Redeploy

Deploy to apply your changes:

fly deploy

For advanced routing options, learn about routers.

​
Next Steps

Secure Your Gateway

Learn more about authentication and observability features

Custom Configuration

Learn about advanced routing and caching options