The AI Gateway ships with built-in configurations for all major LLM providers, including their endpoints, API versions, and supported models. Most users never need to configure providers manually. Provider configuration becomes necessary when you need custom endpoints, enterprise deployments, or want to add models not yet included in the defaults.

​
Getting Started

​
What is Provider Configuration?

Provider configuration defines how the AI Gateway connects to and communicates with LLM providers. Each provider has specific requirements for:
  • Base URLs - The API endpoint to send requests to
  • API Versions - Some providers require specific version headers
  • Supported Models - Which models are available through each provider
The gateway includes comprehensive defaults that work out-of-the-box for standard deployments.

​
Why Configure Providers?

You’ll need to configure providers when you have:
  • Custom endpoints - Self-hosted models, Amazon Bedrock, or enterprise deployments
  • Regional requirements - Specific geographic endpoints for compliance
  • API version control - Need to lock to specific provider API versions
  • Limit available models - Override default provider config to restrict which models are accessible
Provider configuration only defines endpoints and models. API key management is handled separately in Secret Management.

​
How to Enable Providers?

Providers are enabled by simply setting their API keys as environment variables. See Secret Management for details on API key configuration.

​
Configuration Examples

Use case: Limit available models for cost control or compliance.
providers:
  anthropic:
    base-url: "https://api.anthropic.com" 
    version: "2023-06-01"
    models:
      - claude-3-5-haiku  # Only allow cheaper, faster model

routers:
  production:
    load-balance:
      chat:
        strategy: latency
        providers:
          - anthropic
For complete configuration options and syntax, see the Configuration Reference.

​
Reference

​
Supported Providers

The AI Gateway supports the following LLM providers with built-in configurations:
ProviderDefault EndpointCustom Endpoint Support
OpenAIhttps://api.openai.com✅ (regional variants)
Anthropichttps://api.anthropic.com✅ (regional variants)
Geminihttps://generativelanguage.googleapis.com✅ (Gemini)
AWS BedrockRegional AWS endpoints✅ (cross-region)
VertexAIRegional GCP endpoints✅ (cross-region)
Ollamahttp://localhost:11434✅ (any host/port)

​
Supported Models

For current model support by provider, see the embedded provider configuration which is automatically updated as new models are released. Model categories supported:
  • Chat/Completion models - GPT-4, Claude, Gemini, etc.
  • Embedding models - text-embedding-ada-002, etc.
  • Vision models - GPT-4 Vision, Claude 3 Vision
  • Local models - Any Ollama-compatible model