Prompt caching and sessions

When to use this pageCopy Anchor Link

You send the same long prefix more than once (a system prompt, documents, tool definitions, or a growing conversation) and want to pay less for it. This page covers cache lifetimes, cache_control on the Anthropic-format API, automatic caching on the OpenAI-compatible API, sessions, and how to check that the cache works.

What caching savesCopy Anchor Link

A prompt cache stores the processed start of a prompt. When the next request starts with exactly the same content, the model reads it from the cache instead of processing it again.

The prices for each model are on Pricing. Caching pays off as soon as a cached prefix is read back once or twice. A prefix written and never read again costs more than sending it uncached.

• The cache matches an exact prefix, in the order tools, system, messages, up to the block marked with cache_control. A change anywhere before that point (a timestamp in the system prompt, a reordered tool) misses the cache.
• Short prompts aren't cached. Each model has a minimum cacheable length (a few hundred to a few thousand tokens). Below it, the request runs normally and is billed as plain input, with no error.
• Each read renews the lifetime, at no extra cost.

Cache lifetimesCopy Anchor Link

The lifetime counts from the start of the request that wrote or last read the entry. The features column on Models shows which models offer the 1-hour cache.

On the Anthropic-format APICopy Anchor Link

Put cache_control on the last block of the prefix you want cached: the last tool in tools, a block in system, or a content block in messages. Up to 4 breakpoints per request.

import os

import anthropic

client = anthropic.Anthropic(base_url="https://api.router.ezat.io", api_key=os.environ["EZAT_API_KEY"])

# A long, unchanging system prompt: the part worth caching.
system = [
    {
        "type": "text",
        "text": "You answer questions about the Example Ltd product manual. " + "Manual text. " * 400,
        "cache_control": {"type": "ephemeral"},
    }
]

for question in ["How do I reset my password?", "How do I change my email address?"]:
    message = client.messages.create(
        model="anthropic/claude-sonnet-5-5",
        max_tokens=64,
        system=system,
        messages=[{"role": "user", "content": question}],
        # Different questions are different sessions; one session id keeps them on one deployment.
        extra_headers={"x-ezat-session": "manual-assistant-v1"},
    )
    usage = message.usage
    print("write", usage.cache_creation_input_tokens, "read", usage.cache_read_input_tokens)

The first request writes the cache and the second one reads it, as long as the prefix is above the model's minimum length.

Anthropic's top-level cache_control (one field on the request that places the breakpoint for you) is passed through as well, but Claude Opus 4.6 rejects it. Breakpoints on blocks work on every model that caches.

On the OpenAI-compatible APICopy Anchor Link

OpenAI's format has no way to mark a breakpoint, so EZAT caches automatically. On a request with no breakpoints of its own, to a model that caches, it places two 5-minute breakpoints:

1. after the stable prefix: the system prompt, or the last tool when there is no system prompt;
2. on the newest message, which the next turn of the conversation reads back.

A multi-turn conversation then reads everything it has already sent from the cache.

• Turn it off for one request with the header x-ezat-cache: off. Do this for one-off prompts above the minimum length that you won't send again: they would pay a cache write that is never read.
• Place your own breakpoints by putting cache_control on content parts (text, image_url, file, or text parts of system messages) or on a tool, as OpenRouter does. EZAT then adds none. The ttl field works here too.
• prompt_cache_key names the session, like x-ezat-session (below).

{
  "model": "anthropic/claude-sonnet-5-5",
  "messages": [
    {
      "role": "system",
      "content": [
        { "type": "text", "text": "Long instructions…", "cache_control": { "type": "ephemeral", "ttl": "1h" } }
      ]
    },
    { "role": "user", "content": "First question" }
  ]
}

Sessions: one conversation, one deploymentCopy Anchor Link

EZAT serves each model from several deployments, and a prompt cache lives in only one of them. A request sent to a different deployment misses the cache and writes it again. So EZAT keeps the requests of one conversation on the deployment that served it.

• The session is your organization plus the system prompt plus the first message, ignoring cache_control (clients move breakpoints every turn). Every turn of a normal conversation has the same session, with nothing for you to do.
• EZAT remembers the deployment for 1 hour after the session's last successful request.
• A session moves when its deployment is unhealthy, out of capacity, or taken out of service. The next request then writes the cache again on the new deployment.

Name the session yourself when the automatic one doesn't fit:

Session ids only group your own organization's requests. Don't send one id with all your traffic: it would all go to one deployment while that deployment has room.

Check that the cache worksCopy Anchor Link

On the Anthropic-format API, input_tokens counts only the uncached input; the total input is input_tokens + cache_creation_input_tokens + cache_read_input_tokens. On the OpenAI-compatible API, prompt_tokens is already the total.

If reads stay at zero:

• the prefix changes between requests, or is below the model's minimum length;
• the requests are more than 5 minutes apart (use the 1-hour lifetime);
• the requests belong to different sessions (send x-ezat-session);
• the model doesn't cache (next section).

Models with caveatsCopy Anchor Link

capabilities for every model is in GET https://api.router.ezat.io/api/models (no key needed). The features column on Models shows "Prompt caching" and "1 h cache".

Claude CodeCopy Anchor Link

Claude Code places its own breakpoints, and its system prompt and first message stay the same for the whole conversation, so caching and sessions work with no setting. See Claude Code.

For AI agentsCopy Anchor Link

1. Anthropic-format API: put {"type": "ephemeral"} (or with "ttl": "1h") on the last tool, system block or message block of the stable prefix. At most 4 breakpoints.
2. OpenAI-compatible API: caching is automatic; don't add breakpoints unless you need a 1-hour lifetime. Send x-ezat-cache: off for one-off prompts.
3. For one conversation, send the same x-ezat-session (or prompt_cache_key) on every turn if its system prompt or first message changes. Never use one value for all traffic.
4. Check cache_read_input_tokens (Anthropic format) or prompt_tokens_details.cached_tokens (OpenAI format) on the second request to confirm reads.