This cloud AI Gateway is being deprecated. While it remains available for existing users, we are no longer adding new features or major updates to this version.

​
Migration Path

We’re building a new cloud-hosted AI Gateway based on our improved self-hosted AI Gateway. The new version offers:

  • Better performance with Rust-based architecture
  • More providers with 100+ LLM models supported
  • Advanced load balancing with intelligent routing
  • Enhanced caching for cost optimization
  • Improved reliability with automatic failover

Timeline:

  • Current: Legacy cloud gateway remains available
  • Coming Soon: New cloud gateway based on self-hosted version
  • Future: Gradual migration path for existing users

​
Legacy Cloud AI Gateway Features

While you’re using the current cloud gateway, here’s the complete documentation for all available features:

​
Core Gateway Features

Custom Rate Limits

Configure request rate limiting and spending controls for your applications

Gateway Fallbacks

Set up automatic fallback providers when your primary LLM provider fails

LLM Security

Implement security measures and content filtering for your LLM requests

Content Moderation

Automatically detect and filter inappropriate content in requests and responses

Automatic Retries

Configure retry logic for failed requests with exponential backoff