Prompt Caching

Prompt caching in LLMs means the provider keeps the work it did on the start of your prompt, so the next request that starts the same way does not pay for that work again. Most production prompts barely change between calls. The system prompt, the tool list, the examples and a long reference document go out on every request, and only the user's latest message differs. Without a cache the model reprocesses all of it each time at full input price. With one, the repeated part is billed at a fraction of the rate and the answer starts sooner.

What prompt caching stores: computed state, not text#

During prefill the model runs every prompt token through its attention layers and stores the keys and values for each layer, the KV cache. Prompt caching keeps that computed state between requests. Each token's entries depend on every token before it, so the saved state is only valid for an exact prefix that starts at the first token. Change one character early in the prompt and everything after it has to be computed again.

That one fact sets the layout of a cache-friendly prompt. Put stable content first: tools, system prompt, documents, examples. Put what varies last: the user's question and any passages retrieved for this query. The common self-inflicted miss is a timestamp at the top of the system prompt. "Current time: 14:32:07" makes every request a new prefix and turns caching off for everything behind it, with no error to tell you.

Prices as of October 2026#

On the Claude API, a cache read costs 0.1x the base input price on most models, 0.05x on Claude Opus 5.5 and Sonnet 5.5, and 0.025x on Claude Fable 5.1 and Mythos 5.1. Writing the cache costs 1.25x base input for the default 5-minute lifetime, or 2x for a 1-hour lifetime.

Take an agent with a 50,000-token stable prefix on Claude Opus 5 at $5 per million input tokens. Uncached, the prefix costs $0.25 per call. Cached, the first call writes it for about $0.31 and each later call reads it for $0.025. Over a 20-turn session the prefix costs $0.79 instead of $5.00, an 84% saving. On Opus 5.5, at $4 per million with reads at 0.05x, the same session costs about $0.44 instead of $4.00.

OpenAI changed its terms with GPT-5.6. On GPT-5.6 and later, caching is on by default, a read costs 0.1x the input rate (0.05x on GPT-6.1 Sol), and a write costs 1.25x for the default 30-minute lifetime or 2.5x for a 12-hour lifetime. GPT-5.5 and earlier models charge no write premium and give a cached-input rate that varies by model. The GPT-4o snapshots gpt-4o-2024-08-06 and gpt-4o-2024-11-20 leave the API on 14 January 2027, and the replacement OpenAI lists, gpt-6-luna, prices input at $0.10 per million, cache writes at $0.125 and cached input at $0.01. GPT-4o deprecation: dates, replacement and how to migrate is the cost check to run before a cache rate picks the model.

Gemini bills cached tokens at about 10% of the input price on most current models, for example $0.125 against $1.25 per million on Gemini 2.5 Pro, and 5% on Gemini 4 Argon ($0.10 against $2.00 until 30 November 2026). Explicit caches also bill storage per million tokens per hour, $4.50 on the Pro tiers and $1.00 on Gemini 2.5 Flash.

Our AI agent cost guide runs these rates through a ten-turn agent task, where caching the growing transcript takes the run from about $0.018 to about $0.006 on Luna. The latency gain comes from the same place, because a hit skips most of prefill. In Anthropic's announcement, a 100,000-token cached document cut time to first token from 11.5 seconds to 2.4, and the post claims up to 85% lower latency on long prompts.

Implicit or explicit, by provider#

OpenAI caches implicitly on every supported model. GPT-5.6 and later need at least 1,024 visible input tokens and also accept explicit breakpoints. On earlier models the minimum depends on request settings, hits count in 128-token steps, and prompt_cache_key helps route requests that share a prefix to the same machines, with OpenAI suggesting about 15 requests per minute per key.

Gemini caches implicitly on 2.5 and newer models, from 2,048 tokens on 2.5 Pro and Flash and 4,096 on the 3.x and 4 models, and applies the discount only when a hit happens. Explicit caching creates a cached-content object you reference on each request, which makes the discount predictable in exchange for managing the object and paying its storage. Gemini 4 Argon and later do not support explicit caching. Choose explicit caching where a cost model needs the discount to be predictable, and implicit caching where traffic is steady enough to hit anyway.

Anthropic lets you mark up to 4 breakpoints with cache_control on the blocks you want cached, or set one cache_control at the top of the request and let the API move the breakpoint forward as a conversation grows. The minimum is 512 tokens on the newest models and 1,024 to 4,096 on older ones. Below it the request runs uncached and returns no error.

Layout that keeps the hit rate up#

Order content by how often it changes. On the Claude API the invalidation order is tools, then system, then messages, and a change at one level invalidates that level and everything after it. Keep serialisation byte-stable too: reordered tool definitions or JSON keys that come out in a different order are a cache miss nobody sees in review. A conversation is friendly to the cache, because it only grows at the end, so each turn of an agent loop reads the existing prefix and adds one turn to it.

Then read the usage fields instead of assuming hits: cache_read_input_tokens and cache_creation_input_tokens on Claude, cached_tokens and cache_write_tokens on OpenAI. We recommend checking them on the first day of real traffic, because a miss caused by a stray timestamp or a reordered tool list shows up on the invoice rather than in any test.

When the cache costs more than it saves#

A cache entry has a lifetime. Anthropic's lasts 5 minutes by default and refreshes free on every read, so steady traffic keeps a prefix warm and idle prefixes expire. OpenAI's default on GPT-5.6 and later is 30 minutes. Caching can lose money through the write premium. Traffic that never repeats a prefix inside the lifetime pays the 1.25x write on Claude or GPT-5.6 and gets nothing back. A 1-hour Claude write at 2x needs at least two reads inside the hour to beat sending the prefix uncached, and a Gemini explicit cache held all day for a few queries can cost more in storage than its reads save. Every deploy that edits the system prompt also starts every cache cold at once. Pick the lifetime from the gaps in your traffic, not from the discount.

This page covers the provider's cache, which stores computed state for a prompt prefix. Caching your own results is a separate decision. Prospex, our prospect research engine, runs cache-first at the application layer, so a rerun reuses earlier extraction and retrieval. By our own estimate that cut repeated external API calls by 50 to 60 percent, and we track its cache hit rate as the cost metric. Neither cache replaces the other.