> ## Documentation Index
> Fetch the complete documentation index at: https://docs.helicone.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Prompt Caching
> Cache frequently-used context across LLM providers for reduced costs and faster responses
Prompt caching allows you to cache frequently-used context (system prompts, examples, documents) and reuse it across multiple requests at significantly reduced costs.
## Why Prompt Caching
Cached prompts are processed at significantly reduced rates by providers (up to 90% savings)
Providers skip re-processing cached prompt segments for faster response times
Works out-of-the-box with OpenAI compatible AI Gateway across all providers
***
## OpenAI and Compatible Providers
**Automatic caching** for prompts over 1024 tokens. Use the `prompt_cache_key` parameter for better cache hit control.
**Compatible providers:** OpenAI, Grok, Groq, Deepseek, Moonshot AI, Azure OpenAI
### Quick Start
```typescript theme={null}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://ai-gateway.helicone.ai",
apiKey: process.env.HELICONE_API_KEY,
});
const response = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [
{
role: "system",
content: "Very long system prompt that will be automatically cached..." // 1024+ tokens
},
{
role: "user",
content: "What is machine learning?"
}
],
prompt_cache_key: `doc-analysis-${documentId}` // Optional: control caching keys
});
```
### Pricing
OpenAI charges standard rates for cache writes and offers significant discounts for cache reads. Exact pricing varies by model.
View supported models and their caching capabilities
Official OpenAI prompt caching guide
***
## Anthropic (Claude)
Anthropic provides advanced caching with **cache control breakpoints** (up to 4 per request) and TTL control.
### Using OpenAI SDK with Helicone Types
The `@helicone/helpers` SDK extends OpenAI types to support Anthropic's cache control through the OpenAI-compatible interface:
```bash theme={null}
npm install @helicone/helpers
```
```typescript theme={null}
import OpenAI from "openai";
import { HeliconeChatCreateParams } from "@helicone/helpers";
const client = new OpenAI({
baseURL: "https://ai-gateway.helicone.ai",
apiKey: process.env.HELICONE_API_KEY,
});
const response = await client.chat.completions.create({
model: "claude-3.5-haiku",
messages: [
{
role: "system",
content: "You are a helpful assistant...",
cache_control: {
type: "ephemeral",
ttl: "1h"
}
},
{
role: "assistant",
content: "Example assistant message.",
cache_control: { type: "ephemeral" }
},
{
role: "user",
content: [
{
type: "text",
text: "This content will be cached.",
cache_control: {
type: "ephemeral",
ttl: "5m"
}
},
{
type: "image_url",
image_url: {
url: "https://example.com/image.jpg",
detail: "low"
},
cache_control: { type: "ephemeral" }
}
]
}
],
temperature: 0.7
} as HeliconeChatCreateParams);
```
### Cache Key Mapping
Anthropic uses `user_id` as a cache key on their servers. When using the OpenAI-compatible AI Gateway, these parameters automatically map to Anthropic's `user_id`:
* `prompt_cache_key`
* `safety_identifier`
* `user`
```typescript theme={null}
const response = await client.chat.completions.create({
model: "claude-3.5-haiku",
messages: [/* your messages */],
prompt_cache_key: "doc-analysis-v1", // Maps to Anthropic's user_id for cache keying
cache_control: {
type: "ephemeral",
ttl: "1h"
}
} as HeliconeChatCreateParams);
```
**Current Limitation**: Anthropic cache control is currently enabled for caching messages only. Support for caching tools is coming soon.
### Pricing Structure
Anthropic uses a simple multiplier-based pricing model for prompt caching.
| Operation | Multiplier | Example (Claude Sonnet @ \$3/MTok) |
| -------------------- | ---------- | ---------------------------------- |
| Cache Read | 0.1× | \$0.30/MTok |
| Cache Write (5 min) | 1.25× | \$3.75/MTok |
| Cache Write (1 hour) | 2.0× | \$6.00/MTok |
### Key Points
* **TTL Options**: 5 minutes or 1 hour
* **Providers**: Available on Anthropic API, Vertex AI, and AWS Bedrock
* **Limitation**: Vertex AI and Bedrock only support 5-minute caching
* **Minimum**: 1024 tokens for most models
### Calculation Example
```
Base input price: $3/MTok
5-min cache write: $3 × 1.25 = $3.75/MTok
1-hour cache write: $3 × 2.0 = $6.00/MTok
Cache read: $3 × 0.1 = $0.30/MTok
```
Anthropic Prompt Caching Documentation
***
## Google Gemini
Google uses a multiplier plus storage cost model for context caching.
### Pricing Structure
| Operation | Multiplier | Storage Cost |
| ----------- | ---------- | ------------- |
| Cache Read | 0.25× | N/A |
| Cache Write | 1.0× | + Storage fee |
**Storage Rates:**
* Gemini 2.5 Pro: \$4.50/MTok/hour
* Gemini 2.5 Flash: \$1.00/MTok/hour
* Gemini 2.5 Flash-Lite: \$1.00/MTok/hour
### Key Points
* **TTL**: 5 minutes only
* **Cache Types**: Implicit (automatic) and Explicit (manual)
* **Minimum**: 1024 tokens (Flash), 2048 tokens (Pro)
* **Discount**: 75% off input costs for cache reads
### Calculation Example
For Gemini 2.5 Pro (≤200K tokens):
```
Base input price: $1.25/MTok
Storage rate: $4.50/MTok/hour
Cache write (5 min):
- Input cost: $1.25 × 1.0 = $1.25
- Storage cost: $4.50 × (5/60) = $0.375
- Total: $1.625/MTok
Cache read: $1.25 × 0.25 = $0.31/MTok
```
### Tiered Pricing
Gemini 2.5 Pro has different rates for larger contexts:
| Context Size | Input Price | Cache Read | Cache Write (5 min) |
| ------------ | ----------- | ------------ | ------------------- |
| ≤200K tokens | \$1.25/MTok | \$0.31/MTok | \$1.625/MTok |
| >200K tokens | \$2.50/MTok | \$0.625/MTok | \$2.875/MTok |