Anthropic Prompt Caching: Complete Implementation Guide

Anthropic prompt caching is the single highest-leverage cost and latency optimization available for Claude-based systems, and most teams leave it half-configured. The mechanism is simple: you mark a stable prefix of your request with a cache_control breakpoint, Claude stores the processed prefix, and subsequent requests that share that exact prefix read it back at roughly a tenth of the normal input price. The hard part is not the API call. It is designing your prompt assembly so the prefix actually stays stable, and then proving with the usage fields that the cache is being hit.

This guide covers how the cache works, what it costs, what silently invalidates it, how to structure multi-turn agents around it, and how it compares to OpenAI's automatic caching and Gemini's context caching.

Key Takeaways:

• Caching is a prefix match: the request is rendered as tools → system → messages, and any byte change before a breakpoint invalidates everything after it.
• Cache reads cost roughly 10% of the base input price; cache writes cost roughly 125% at the 5-minute TTL and more at the 1-hour TTL, so a prefix must be reused at least twice to pay for itself.
• There is a minimum cacheable prefix (model-dependent, on the order of 1,000-4,000 tokens); shorter prefixes silently do not cache.
• Put timestamps, request IDs, and per-user data after the last breakpoint, never inside the system prompt.
• Measure hit rate from usage.cache_read_input_tokens and usage.cache_creation_input_tokens on every response; a cache that reads zero across identical requests has a silent invalidator.
• For agents, place a breakpoint on the last block of the newest turn so each request reuses the entire prior conversation.

Table of Contents:

• Anthropic Prompt Caching Basics
• Cache Key Design
• Pricing and ROI
• Implementation Patterns
• Hit Rate Optimization
• Monitoring
• Best Practices
• Frequently Asked Questions

Anthropic Prompt Caching Basics

Every Messages API request is rendered into one long token sequence in a fixed order: tool definitions first, then the system prompt, then the messages array. Prompt caching lets you mark points in that sequence with cache_control: {"type": "ephemeral"}. When the API sees a request, it looks for the longest previously cached prefix that ends at one of your breakpoints and matches byte-for-byte. Everything up to that point is served from cache; everything after it is processed normally.

Three facts follow from this, and they drive every design decision in this article:

1. The cache is positional. If the tool list changes, the system prompt and the conversation history behind it are also invalidated, because they come later in the sequence.
2. Matching is exact. A single changed character, a reordered JSON key in a tool schema, or a different whitespace convention produces a different prefix.
3. Breakpoints are checkpoints, not ranges. A breakpoint on the last system block caches tools plus system together. You do not need a breakpoint on every block.

The official reference is the prompt caching documentation. Two constraints from it matter immediately:

• Maximum four breakpoints per request. Use them at stability boundaries (tools, system, shared context, latest turn), not on every message.
• Minimum cacheable prefix. Each model has a minimum token count below which a prefix will not be cached at all. The threshold is model-dependent (historically 1,024 tokens for Sonnet/Opus-class models and 2,048 for Haiku-class; newer models can be higher, so check the table in the docs for the model you are running). A 600-token system prompt with a breakpoint on it does nothing, and the API does not return an error. It just reports zero cache tokens.

Time-to-live: 5 minutes vs 1 hour

The default TTL is 5 minutes, refreshed on every cache hit. If your traffic keeps hitting a prefix at least once every five minutes, the entry effectively lives forever at no extra write cost. If traffic is bursty or sparse, you can request a 1-hour TTL with cache_control: {"type": "ephemeral", "ttl": "1h"}. The 1-hour write is priced higher than the 5-minute write, so it only makes sense when the gap between reuses is regularly longer than five minutes but shorter than an hour: nightly batch jobs, low-traffic internal tools, or agent sessions where a human takes a long time to respond.

You can mix TTLs in one request, but the 1-hour entries must precede the 5-minute entries in the prefix order.

Cache Key Design

Because the cache key is the rendered byte sequence, "cache key design" really means "prompt assembly order design". The workflow we use on every engagement:

1. Trace the assembly path. Find every input that flows into tools, system, and messages.
2. Classify each input by how often it changes: never (product instructions, tool schemas), per-tenant (customer config, retrieved knowledge base), per-session (conversation history), per-request (user question, timestamp, request ID).
3. Order the prompt by stability. Never-changing content first, per-request content last.
4. Place breakpoints at the boundaries between those tiers.

What invalidates the cache

These are the invalidators we find most often in code review. All of them are silent; the only symptom is cache_read_input_tokens: 0.

Changing the system prompt invalidates the system and messages cache but not the tools cache. Changing tools invalidates everything. Changing only the messages tail leaves tools and system intact, which is exactly why the tools → system → messages ordering exists.

Pricing and ROI

Per the Anthropic pricing page, cached tokens are billed at a multiplier of the model's base input price. In round terms: cache reads cost roughly 10% of the input price, and cache writes cost roughly 125% at the 5-minute TTL (1-hour writes are priced higher still). Output tokens are unaffected.

That gives a simple break-even model. Let P be the base input price per token and N the number of times a prefix is reused within its TTL:

• Uncached cost: N × P
• Cached cost: 1.25P + (N − 1) × 0.1P

At N = 1 you lose 25%. At N = 2 you are roughly even (1.35P vs 2P, so already a win). At N = 10 you pay about 2.15P instead of 10P, a 78% reduction on that prefix. The savings asymptote at 90% as N grows.

The illustrative table below assumes a 20,000-token stable prefix, a 500-token per-request tail, and a base input price of P. Numbers are relative, not dollar figures:

Latency improves alongside cost. The cached prefix does not need to be re-processed by the model's prefill stage, so time-to-first-token drops substantially on long prompts. For RAG systems that stuff tens of thousands of tokens of retrieved context into every call, this is often the difference between a usable and an unusable interactive experience. See our guide to reducing LLM costs with techniques that actually work for where caching sits relative to model routing and batching.

Implementation Patterns

Pattern 1: Large shared system prompt

The simplest case. Tools and system are stable; only the user message varies. One breakpoint on the last system block caches tools and system together.

import anthropic

client = anthropic.Anthropic()

SYSTEM_PROMPT = open("prompts/support_agent_v12.md").read()  # frozen at import
TOOLS = [...]  # deterministic list, defined once

def answer(question: str) -> anthropic.types.Message:
    return client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=1024,
        tools=TOOLS,
        system=[
            {
                "type": "text",
                "text": SYSTEM_PROMPT,
                "cache_control": {"type": "ephemeral"},
            }
        ],
        messages=[{"role": "user", "content": question}],
    )

Note that SYSTEM_PROMPT is read once at import time. Reading it per request is fine as long as the file does not change, but reading a template and interpolating a timestamp into it is the most common way teams break caching without noticing.

Pattern 2: Shared context with a varying question

Many requests share a large fixed preamble (a document, few-shot examples, a schema) but differ in the final question. The breakpoint goes at the end of the shared portion, not the end of the whole prompt. If you put it after the question, every request writes a distinct entry and nothing is ever read.

def ask_about_document(document: str, question: str):
    return client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=1024,
        system="You answer questions strictly from the provided document.",
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": f"<document>\n{document}\n</document>",
                        "cache_control": {"type": "ephemeral"},
                    },
                    {"type": "text", "text": question},  # no marker: varies per call
                ],
            }
        ],
    )

Pattern 3: Multi-turn agents

For agents, the conversation itself is the expensive prefix. Place a breakpoint on the last content block of the most recently appended turn. Each new request finds the previous request's entry, reads the entire history, and processes only the new turn. Earlier breakpoints remain valid read points, so hits accrue incrementally as the conversation grows.

def run_agent_turn(history: list[dict], new_user_content: list[dict]) -> anthropic.types.Message:
    for msg in history:
        if isinstance(msg["content"], list):
            for block in msg["content"]:
                block.pop("cache_control", None)

    new_user_content[-1]["cache_control"] = {"type": "ephemeral"}
    history.append({"role": "user", "content": new_user_content})

    response = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=4096,
        tools=TOOLS,
        system=[{"type": "text", "text": SYSTEM_PROMPT,
                 "cache_control": {"type": "ephemeral"}}],
        messages=history,
    )
    history.append({"role": "assistant", "content": response.content})
    return response

Two agent-specific gotchas from the docs:

• The lookback window is limited. Each breakpoint only searches back a bounded number of content blocks (20 at the time of writing) for a prior cache entry. A turn that appends dozens of tool results can push the previous entry out of the window. For long tool loops, add an intermediate breakpoint partway through the turn.
• Parallel fan-out does not share writes. A cache entry becomes readable only after the first response begins streaming. If you fire ten identical-prefix requests concurrently, all ten pay the write price. Send one, wait for its first streamed token, then fire the rest.

This is directly relevant to multi-agent orchestration patterns: N workers with slightly different prompts over the same context write N entries and read none of each other's. Fewer lanes over a byte-identical prefix win.

Pattern 4: Mid-conversation instruction changes

The tempting way to change agent behaviour mid-session is to edit the system prompt. That invalidates the entire history. On models that support it, append a {"role": "system", ...} message to the messages array instead; it sits after the cached history and leaves the prefix intact. Where that is not available, put the instruction in the newest user turn.

Hit Rate Optimization

Once the basics are in place, the remaining wins come from increasing how many requests share a prefix:

• Freeze prompt versions. Tie the system prompt to a version identifier and deploy it like code. Every edit is a full cache rewrite for every user, so batch prompt changes rather than trickling them out. Our prompt versioning guide covers the release process.
• Consolidate small context. If your system prompt is below the minimum cacheable size, move stable reference material (glossaries, policy text, schema descriptions) into it rather than into per-request messages.
• Pre-warm before traffic. A request with max_tokens=1 against the stable prefix writes the cache before users arrive. Useful after deploys and for the 1-hour TTL.
• Align tenant context with breakpoints. In a multi-tenant system, order the prompt as global instructions → tenant config → session history, with a breakpoint after each. Global content is shared by everyone; tenant content by that tenant's sessions.
• Do not route within a conversation. LLM routing is a powerful cost lever, but caches are model-scoped. Route at conversation start, not per turn.

Monitoring

Every response carries four usage fields. Log all of them on every call:

u = response.usage
print(u.input_tokens)                 # uncached tokens processed at full price
print(u.cache_creation_input_tokens)  # tokens written to cache this request
print(u.cache_read_input_tokens)      # tokens served from cache
print(u.output_tokens)

Derive two metrics per route and per prompt version:

• Hit rate = cache_read / (input + cache_creation + cache_read). For a stable system prompt route this should approach the fraction of the prompt that is prefix, often 80-95%. Anything below 50% on a route you expected to cache means an invalidator.
• Write ratio = cache_creation / cache_read. A high write ratio with a low hit rate means prefixes are being written but never reused: TTL too short, too many distinct prefixes, or fan-out without pre-warming.

Wire these into the same tracing you use for the rest of the pipeline. If you already run OpenTelemetry tracing for LLM calls, add the four fields as span attributes and alert when the per-route hit rate drops after a deploy, which is the signature of a prompt change that broke prefix stability.

def cache_metrics(usage) -> dict:
    total = usage.input_tokens + usage.cache_creation_input_tokens + usage.cache_read_input_tokens
    return {
        "llm.cache.hit_rate": usage.cache_read_input_tokens / total if total else 0.0,
        "llm.cache.write_tokens": usage.cache_creation_input_tokens,
        "llm.cache.read_tokens": usage.cache_read_input_tokens,
    }

Best Practices

How prompt caching compares across providers

Anthropic's approach is explicit; the other major providers made different trade-offs.

OpenAI's automatic prompt caching is zero-effort but also zero-control: you cannot pin a prefix, choose a TTL, or pay to keep something warm. Gemini's context caching is closer to a managed object store for context: excellent for a 500,000-token corpus reused all day, heavier to operate for chat-style traffic. Anthropic's model sits between them, which is why the prompt-assembly discipline in this guide matters more on Claude than elsewhere: you get the savings only if you do the ordering work.

Where it fits in a caching stack

Prompt caching reduces the cost of processing a prefix. It does not eliminate the call. For repeated questions, layer a semantic cache in front of the model to skip the call entirely, and use prompt caching for the calls that get through. For non-interactive workloads, the Batch API stacks its 50% discount on top of cache reads.

If you are building agents on Claude and want the prompt assembly, tracing, and cost controls designed in from the start, our AI agent development service does exactly this work.

Frequently Asked Questions

How much does Anthropic prompt caching save?

Cache reads cost roughly 10% of the base input price, so the cached portion of a prompt is about 90% cheaper on every hit after the first. Real-world total savings depend on what fraction of each request is stable prefix and how often it is reused; systems with large system prompts or retrieved context commonly see 50-80% lower input bills. Output tokens are not discounted.

Does prompt caching work across API calls?

Yes. The cache is keyed on the rendered prefix, not on a session, so any request from the same organization with the same model and the same byte-identical prefix within the TTL reads the entry. The default 5-minute TTL is refreshed on every hit, so steady traffic keeps the entry warm indefinitely.

What is the minimum prompt size for caching?

It is model-dependent, on the order of 1,024-4,096 tokens depending on the model family. Prefixes below the threshold are silently not cached and the response reports zero cache tokens. Check the per-model table in the official docs and consolidate stable context into the system prompt until it clears the bar.

Does changing the system prompt invalidate cached tools?

No. Tools render before the system prompt, so editing the system prompt invalidates the system and messages portion but leaves a cached tools prefix readable. Changing the tool list, however, invalidates everything after it, including system and history.

How many cache breakpoints can I use?

Four per request. Place them at stability boundaries: after tools/system, after shared tenant context, and on the newest conversation turn. You do not need a breakpoint on every message because the API searches backwards for the longest matching cached prefix.

Should I use the 1-hour TTL?

Only when reuse gaps are regularly longer than five minutes but shorter than an hour. The 1-hour write is priced above the 5-minute write, so on steady traffic it costs more for no benefit. It pays off for sparse agent sessions, low-traffic internal tools, and pre-warmed scheduled jobs.

Can I measure the cache hit rate?

Yes, from the usage object on every response: cache_read_input_tokens divided by the sum of input_tokens, cache_creation_input_tokens, and cache_read_input_tokens. Log it per route and prompt version and alert on drops after deploys.

Conclusion

• Prompt caching is a prefix match over tools → system → messages; order your prompt by stability and put volatile content last.
• Reads cost roughly 10% of input price and writes roughly 125% at 5 minutes, so any prefix reused twice or more is a net win.
• The most common failures are silent: timestamps in system prompts, unstable JSON serialization, per-user tool lists, and prefixes below the minimum size.
• For agents, mark the newest turn and keep history append-only; pre-warm before fan-out.
• Log the four usage fields on every call and alert on hit-rate regressions.

If you want a second pair of eyes on your prompt assembly and cache hit rates, talk to our engineering team.

Resources:

• Anthropic prompt caching documentation
• Anthropic pricing
• OpenAI prompt caching guide
• Gemini context caching