Prompt caching

Prompt caching is the single highest-leverage cost optimization available in the Anthropic API. It is also frequently misconfigured in ways that look correct but produce zero savings.

What gets cached

Prompt caching stores a prefix of your request on Anthropic's servers. When the next request begins with an identical prefix, the cached tokens are read at roughly 10% of the normal input token price instead of being reprocessed. The cache lives for five minutes — the TTL resets on each cache hit.

The prefix can include the system block, large documents, few-shot examples, or any stable content you send on every request. The marker tells Anthropic where the cacheable prefix ends.

Where to place the cache marker

Add "cache_control": {"type": "ephemeral"} to the last content block in the prefix you want cached. The key word is last — the marker must be at the boundary where the stable content ends and the dynamic content begins.

System prompt with caching:

# scripts/cache_demo.py
import anthropic

client = anthropic.Anthropic()

SYSTEM = [
    {
        "type": "text",
        "text": "You are a technical writing assistant. " * 200,  # artificially long for demo
        "cache_control": {"type": "ephemeral"},
    }
]

def ask(question):
    response = client.messages.create(
        model="claude-opus-4-5",
        max_tokens=256,
        system=SYSTEM,
        messages=[{"role": "user", "content": question}],
    )
    u = response.usage
    print(
        f"input={u.input_tokens} "
        f"cache_create={getattr(u, 'cache_creation_input_tokens', 0)} "
        f"cache_read={getattr(u, 'cache_read_input_tokens', 0)} "
        f"output={u.output_tokens}"
    )
    return response.content[0].text

ask("What is an API?")   # first call — cache miss, creates entry
ask("What is REST?")     # second call — cache hit
ask("What is JSON?")     # third call — cache hit

First call: cache_creation_input_tokens is non-zero, cache_read_input_tokens is zero. Second and third calls: cache_creation_input_tokens is zero, cache_read_input_tokens is the size of the cached prefix.

Why the five-minute TTL matters for your architecture

Cache entries expire after five minutes of inactivity. Applications with consistent, frequent traffic — a support bot handling dozens of requests per minute — maintain warm caches and pay the low cache-read price continuously.

Batch jobs with irregular timing — something that runs once an hour — will cache-miss on every batch run. Caching is still worth enabling because the batch itself may run many requests within five minutes, but the savings model is different. A batch of 50 requests in a two-minute window will hit the cache on requests 2 through 50.

The most common misconfiguration

Placing the cache marker inside dynamic content. If the user's current message is part of the cached prefix, the prefix changes every turn and the cache never hits.

Wrong — marks the user message as the cache boundary:

messages=[{
    "role": "user",
    "content": [{"type": "text", "text": user_input, "cache_control": {"type": "ephemeral"}}]
}]

Every new message produces a different prefix. Zero hits. The marker belongs on the system prompt or the last stable document block, not on the user's message.

Minimum token threshold

Caching only activates when the cached prefix exceeds 1024 tokens for Claude Sonnet and Haiku and 2048 tokens for Claude Opus. A short system prompt will not populate the cache regardless of the marker. In the demo above, the system string is repeated to hit this threshold. In production, a real system prompt plus reference documents will easily clear it.

If you get stuck

• cache_read_input_tokens is always zero: the most likely cause is a prefix shorter than the minimum threshold. Check the token count. Second cause: the prefix is not identical between calls — a timestamp or per-user variable in the cached section breaks the cache key.
• cache_creation_input_tokens is zero on the first call: the prefix may be below the minimum threshold, or the marker is missing. Confirm the system string is long enough by counting tokens using the Anthropic tokenizer or a rough 4-characters-per-token estimate.
• Cost is not dropping after adding caching: confirm you are reading cache_read_input_tokens from the actual response, not from the first call. The savings only appear on the second and subsequent calls within the TTL window.
• Fields not present on the usage object: access them with getattr(u, 'cache_read_input_tokens', 0) as shown above — older SDK versions may not surface them as direct attributes.

Cross-reference

l5 (Rate limits and the cost of context) shows where the input token costs come from. Caching is the primary tool for reducing those costs when the stable prefix is large. l8 (Add it to a real app) is where caching should be enabled — any route that sends a long system prompt or document on every request is a candidate.

Deliverable

Run cache_demo.py. Confirm the second and third calls show cache_read_input_tokens greater than zero. Calculate the percentage savings on input tokens compared to a call without caching.