Provider Configuration

The AI Gateway ships with built-in configurations for all major LLM providers, including their endpoints, API versions, and supported models. Most users never need to configure providers manually.

Provider configuration becomes necessary when you need custom endpoints, enterprise deployments, or want to add models not yet included in the defaults.

​

Getting Started

​

What is Provider Configuration?

Provider configuration defines how the AI Gateway connects to and communicates with LLM providers. Each provider has specific requirements for:

• Base URLs - The API endpoint to send requests to
• API Versions - Some providers require specific version headers
• Supported Models - Which models are available through each provider

The gateway includes comprehensive defaults that work out-of-the-box for standard deployments.

​

Why Configure Providers?

You’ll need to configure providers when you have:

• Custom endpoints - Self-hosted models, Amazon Bedrock, or enterprise deployments
• Regional requirements - Specific geographic endpoints for compliance
• API version control - Need to lock to specific provider API versions
• Limit available models - Override default provider config to restrict which models are accessible

​

How to Enable Providers?

Providers are enabled by simply setting their API keys as environment variables. See Secret Management for details on API key configuration.

​

Configuration Examples

providers:
  anthropic:
    base-url: "https://api.anthropic.com" 
    version: "2023-06-01"
    models:
      - claude-3-5-haiku  # Only allow cheaper, faster model

routers:
  production:
    load-balance:
      chat:
        strategy: latency
        providers:
          - anthropic

providers:
  anthropic:
    base-url: "https://api.anthropic.com" 
    version: "2023-06-01"
    models:
      - claude-3-5-haiku  # Only allow cheaper, faster model

routers:
  production:
    load-balance:
      chat:
        strategy: latency
        providers:
          - anthropic

providers:
  ollama:
    base-url: "http://192.168.1.100:8080"  # Custom host and port
    models:
      - llama3.2
      - deepseek-r1
      - custom-fine-tuned-model

routers:
  development:
    load-balance:
      chat:
        strategy: latency
        providers:
          - ollama

providers:
  bedrock:
    base-url: "https://bedrock-runtime.us-west-2.amazonaws.com"
    models:
      - anthropic.claude-3-5-sonnet-20241022-v2:0
      - anthropic.claude-3-haiku-20240307-v1:0

routers:
  production:
    load-balance:
      chat:
        strategy: latency
        providers:
          - bedrock

​

Reference

​

Supported Providers

The AI Gateway supports the following LLM providers with built-in configurations:

​

Supported Models

For current model support by provider, see the embedded provider configuration which is automatically updated as new models are released.

Model categories supported:

• Chat/Completion models - GPT-4, Claude, Gemini, etc.
• Embedding models - text-embedding-ada-002, etc.
• Vision models - GPT-4 Vision, Claude 3 Vision
• Local models - Any Ollama-compatible model