Prompt caching works differently across OpenAI, Anthropic, and Google. Compare activation methods, TTLs, pricing, and token minimums to optimize costs across providers.
Three providers, three prompt caching implementations, three different philosophies about who decides what gets cached and for how long. The feature looks the same on the surface: store computed input tokens, reuse them on the next call, pay less. Underneath, OpenAI makes it invisible, Anthropic hands you the controls, and Google gives you both options. These differences matter the moment you send the same prompt through more than one provider.
Every time you send a prompt to a large language model, the transformer's self-attention layers compute key-value (KV) pairs for each input token. Prompt caching stores these KV tensors so subsequent requests with the same prefix skip the computation entirely.
The prerequisite is universal across all three providers: an exact prefix match. Change a single token in the cached portion and the match breaks. This is why every provider's documentation repeats the same structural advice: put static content (system instructions, tool definitions, few-shot examples) at the beginning of the prompt, variable content (user input, conversation tail) at the end.
The savings are substantial. OpenAI reports prompt caching can reduce latency by up to 80% and input token costs by up to 90%. But how you access those savings, and what you pay to write to the cache, diverges sharply.
OpenAI's approach is the simplest. Prompt caching works automatically on all API requests with no code changes required. There is no opt-in, no header, no configuration parameter. If your prompt is long enough, OpenAI caches it.
The minimum is 1,024 tokens. After that threshold, caching increases in 128-token increments. OpenAI launched the feature on October 1, 2024 with a 50% discount on cached input tokens and no additional fees for cache writes. What you can cache covers the full request structure: messages, images, tool definitions, and structured output schemas.
TTL and retention. Cached prefixes remain active for 5 to 10 minutes of inactivity, up to a maximum of one hour under the default in-memory policy. For newer models (gpt-5.5, gpt-5.4, gpt-5.1, gpt-5, gpt-4.1), extended prompt cache retention keeps prefixes active for up to 24 hours by offloading KV tensors to GPU-local storage. Only the KV tensors are persisted, not the original prompt text.
Routing hints. The one developer-facing control is prompt_cache_key, a parameter that combines with the prefix hash to influence routing and improve cache hit rates. This is useful when many requests share long common prefixes and you want to increase the probability they land on the same infrastructure.
Privacy. Prompt caches are never shared between organizations.
The tradeoff is clear: zero setup cost, but you cannot control TTL, cannot force a cache write, and cannot guarantee a hit. The infrastructure decides.
Anthropic gives you two modes. Automatic caching adds a single cache_control field at the top level of the request; the system applies the breakpoint to the last cacheable block and moves it forward automatically as conversations grow. Explicit caching places cache_control directly on individual content blocks for fine-grained control over exactly what gets cached.
The cache covers tools, system prompt, messages, and images, referenced as a full prefix in that order (tools, system, messages) up to and including the designated breakpoint.
Pricing is tiered, not flat. Cache read tokens cost 0.1x the base input price (a 90% discount). 5-minute cache write tokens cost 1.25x the base input price. 1-hour cache write tokens cost 2x the base input price.
TTL options. The default TTL is 5 minutes, refreshed each time the cached content is used. The optional 1-hour TTL costs more to write but keeps content available longer for workloads with sparser request patterns.
Zero Data Retention. Prompt caching is compatible with ZDR arrangements. Cached data is not stored after the API response is returned. This matters for regulated industries where data persistence is a compliance concern.
The economics here reward developers who understand their traffic patterns. High-frequency workloads that reuse cached content many times will see the 1.25x write cost amortized across dozens of 0.1x reads. Low-frequency workloads may find the write premium eats into savings.
Google splits prompt caching into two mechanisms: implicit (automatic) and explicit (manual). They serve different use cases and carry different guarantees.
Implicit caching is enabled by default for all Gemini 2.5 and newer models. There is nothing to configure. Google automatically passes on cost savings when a request hits the cache, but provides no guarantee that a hit will occur. To improve hit rates, Google recommends placing large, repeated content at the beginning of prompts and sending requests with similar prefixes close together in time.
Explicit caching requires creating a CachedContent object and passing its name in subsequent generate_content requests. This comes with a cost saving guarantee but imposes a minimum token count: 2,048 tokens for Gemini 2.5 Pro and Flash, 4,096 tokens for Gemini 3.5 Flash and 3.1 Pro.
The explicit cache defaults to a 1-hour TTL with no minimum or maximum bounds. You choose how long you want the cache to persist.
Billing model.Google charges based on both token count and storage duration — the time the cache is held. This is structurally different from OpenAI (no cache fees at all) and Anthropic (per-token write/read pricing). With Google, a large cached context held for several hours accumulates storage charges even if you never read from it again.
| Feature | OpenAI | Anthropic | Google Gemini |
|---|---|---|---|
| Activation | Fully automatic | Automatic or explicit breakpoints | Implicit (auto) + Explicit (manual) |
| Minimum tokens | 1,024 | Not specified | 2,048 (2.5 Pro/Flash) or 4,096 (3.5 Flash, 3.1 Pro) |
| TTL options | 5–10 min (in-memory), up to 24 hr (extended) | 5 min (default), 1 hr (optional) | Implicit: not guaranteed; Explicit: 1 hr default, configurable |
| Cache write cost | No additional fee | 1.25x (5-min) or 2x (1-hr) base input price | Token count + storage duration |
| Cache read discount | 50% off input tokens | 90% off input tokens (0.1x) | Savings passed on (implicit); guaranteed savings (explicit) |
| Storage billing | None | None (per-token only) | Token count + time held |
| Developer control | prompt_cache_key routing hint | Full breakpoint placement | CachedContent object with configurable TTL |
Each of these implementations is rational on its own. The complexity emerges when you use more than one provider.
Consider a multi-turn agent that routes requests to different models based on task complexity. The system prompt and tool definitions are identical across providers. But the caching behavior is not:
OpenAI caches automatically after 1,024 tokens with no write fee
Anthropic requires a cache_control parameter and charges 1.25x for the write
Google's explicit cache needs a separate CachedContent creation call with storage-duration billing
Three API patterns. Three cost structures. Three TTL behaviors. Without a normalization layer, the team maintaining this agent needs provider-specific caching logic in their routing code, provider-specific cost tracking for cache economics, and provider-specific monitoring to know whether caching is working.
This is the problem that an AI gateway solves at the middleware layer. A gateway can normalize prompt structure so static prefixes are consistently positioned for cache hits across providers, surface cache hit/miss metrics in a single dashboard regardless of which provider served the request, and apply provider-appropriate TTL strategy without the application code knowing which provider was selected.
The differences in prompt caching implementations are not converging. OpenAI added extended retention and routing hints. Anthropic added 1-hour TTL tiers. Google added explicit caching alongside its implicit layer. Each provider is building deeper into their own approach. The divergence is structural, and it is growing.
For teams running production workloads across multiple providers, keep each provider's native caching strategy explicit in application code. SHIM preserves the provider-native requests and does not add a response or semantic cache.
Yes. All three providers support prompt caching with streaming. The cache operates on the input (prompt) side, so the output delivery method does not affect it.
OpenAI caches messages, images, tool definitions, and structured output schemas. Anthropic caches tools, system prompt, messages, and images. Google's explicit caching accepts any content that can be passed to a generate_content call.
The cache miss is total. All three providers require an exact prefix match. A single token difference anywhere in the cached portion breaks the match, and the entire prefix is recomputed.
No. Prompt caching operates on exact token-level prefix matching at the provider's infrastructure layer. Semantic caching, implemented at the gateway level, matches requests by meaning and can return cached responses for similar (not identical) prompts. They address different problems and can be used together.