# Prompt caching and sessions

URL: https://docs.router.ezat.io/guides/prompt-caching

> How prompt caching saves money on EZAT Router, how to use it on both APIs, and why a conversation stays on one deployment.

## When to use this page

You send the same long prefix more than once (a system prompt, documents, tool definitions, or a growing conversation) and want to pay less for it. This page covers cache lifetimes, `cache_control` on the Anthropic-format API, automatic caching on the OpenAI-compatible API, sessions, and how to check that the cache works.

## What caching saves

A prompt cache stores the processed start of a prompt. When the next request starts with exactly the same content, the model reads it from the cache instead of processing it again.

| Token kind | Price |
| --- | --- |
| Input | The model's input price |
| Cache write | More than input. The 1-hour cache costs more to write than the 5-minute one. |
| Cache read | Much less than input |

The prices for each model are on [Pricing](https://docs.router.ezat.io/pricing.md). Caching pays off as soon as a cached prefix is read back once or twice. A prefix written and never read again costs more than sending it uncached.

- **The cache matches an exact prefix,** in the order `tools`, `system`, `messages`, up to the block marked with `cache_control`. A change anywhere before that point (a timestamp in the system prompt, a reordered tool) misses the cache.
- **Short prompts aren't cached.** Each model has a minimum cacheable length (a few hundred to a few thousand tokens). Below it, the request runs normally and is billed as plain input, with no error.
- **Each read renews the lifetime,** at no extra cost.

## Cache lifetimes

| Lifetime | `cache_control` | Use it when |
| --- | --- | --- |
| 5 minutes (default) | `{"type": "ephemeral"}` | Requests come within a few minutes of each other: chat, agent loops, back-to-back jobs |
| 1 hour | `{"type": "ephemeral", "ttl": "1h"}` | Gaps between requests are longer than 5 minutes |

The lifetime counts from the start of the request that wrote or last read the entry. The features column on [Models](https://docs.router.ezat.io/models.md) shows which models offer the 1-hour cache.

## On the Anthropic-format API

Put `cache_control` on the last block of the prefix you want cached: the last tool in `tools`, a block in `system`, or a content block in `messages`. Up to 4 breakpoints per request.

```python tab="Python" test
import os

import anthropic

client = anthropic.Anthropic(base_url="https://api.router.ezat.io", api_key=os.environ["EZAT_API_KEY"])

# A long, unchanging system prompt: the part worth caching.
system = [
    {
        "type": "text",
        "text": "You answer questions about the Example Ltd product manual. " + "Manual text. " * 400,
        "cache_control": {"type": "ephemeral"},
    }
]

for question in ["How do I reset my password?", "How do I change my email address?"]:
    message = client.messages.create(
        model="anthropic/claude-sonnet-5-5",
        max_tokens=64,
        system=system,
        messages=[{"role": "user", "content": question}],
        # Different questions are different sessions; one session id keeps them on one deployment.
        extra_headers={"x-ezat-session": "manual-assistant-v1"},
    )
    usage = message.usage
    print("write", usage.cache_creation_input_tokens, "read", usage.cache_read_input_tokens)
```

The first request writes the cache and the second one reads it, as long as the prefix is above the model's minimum length.

Anthropic's top-level `cache_control` (one field on the request that places the breakpoint for you) is passed through as well, but Claude Opus 4.6 rejects it. Breakpoints on blocks work on every model that caches.

## On the OpenAI-compatible API

OpenAI's format has no way to mark a breakpoint, so EZAT caches automatically. On a request with no breakpoints of its own, to a model that caches, it places two 5-minute breakpoints:

1. after the stable prefix: the system prompt, or the last tool when there is no system prompt;
2. on the newest message, which the next turn of the conversation reads back.

A multi-turn conversation then reads everything it has already sent from the cache.

- **Turn it off** for one request with the header `x-ezat-cache: off`. Do this for one-off prompts above the minimum length that you won't send again: they would pay a cache write that is never read.
- **Place your own breakpoints** by putting `cache_control` on content parts (`text`, `image_url`, `file`, or text parts of system messages) or on a tool, as OpenRouter does. EZAT then adds none. The `ttl` field works here too.
- **`prompt_cache_key`** names the session, like `x-ezat-session` (below).

```json
{
  "model": "anthropic/claude-sonnet-5-5",
  "messages": [
    {
      "role": "system",
      "content": [
        { "type": "text", "text": "Long instructions…", "cache_control": { "type": "ephemeral", "ttl": "1h" } }
      ]
    },
    { "role": "user", "content": "First question" }
  ]
}
```

## Sessions: one conversation, one deployment

EZAT serves each model from several deployments, and a prompt cache lives in only one of them. A request sent to a different deployment misses the cache and writes it again. So EZAT keeps the requests of one conversation on the deployment that served it.

- **The session** is your organization plus the system prompt plus the first message, ignoring `cache_control` (clients move breakpoints every turn). Every turn of a normal conversation has the same session, with nothing for you to do.
- **EZAT remembers the deployment for 1 hour** after the session's last successful request.
- **A session moves** when its deployment is unhealthy, out of capacity, or taken out of service. The next request then writes the cache again on the new deployment.

Name the session yourself when the automatic one doesn't fit:

| Situation | What to send |
| --- | --- |
| The system prompt or first message changes during a conversation (a date in the system prompt, trimmed or summarised history) | One id per conversation |
| Many separate requests share a long prefix (one system prompt, the same documents) | One id per shared prefix, as in the example above |
| Everything else | Nothing |

| API | How |
| --- | --- |
| Both | Header `x-ezat-session: <id>`. Any string; only the first 256 characters count. |
| OpenAI-compatible | Body field `prompt_cache_key`, up to 256 characters (longer is refused with `400`). The header wins if you send both. |

Session ids only group your own organization's requests. Don't send one id with all your traffic: it would all go to one deployment while that deployment has room.

## Check that the cache works

| Where | Cache writes | Cache reads |
| --- | --- | --- |
| Anthropic-format `usage` | `cache_creation_input_tokens` (split by lifetime in `cache_creation`, when the provider reports it) | `cache_read_input_tokens` |
| OpenAI-format `usage` | `prompt_tokens_details.cache_write_tokens` | `prompt_tokens_details.cached_tokens` |
| Dashboard, Usage page | Cache (write / read) | Cache (write / read) |

On the Anthropic-format API, `input_tokens` counts only the uncached input; the total input is `input_tokens + cache_creation_input_tokens + cache_read_input_tokens`. On the OpenAI-compatible API, `prompt_tokens` is already the total.

If reads stay at zero:

- the prefix changes between requests, or is below the model's minimum length;
- the requests are more than 5 minutes apart (use the 1-hour lifetime);
- the requests belong to different sessions (send `x-ezat-session`);
- the model doesn't cache (next section).

## Models with caveats

| Model | What's different |
| --- | --- |
| Models with `capabilities.promptCaching` `false` (today: Claude Fable 5) | Their cache never reads back, so EZAT removes `cache_control` from requests to them on both APIs and skips automatic caching. You pay plain input, not cache writes that would never be read. `usage` shows no cache tokens. |
| Claude Opus 4.6 | Rejects Anthropic's top-level `cache_control`. Use breakpoints on blocks. |

`capabilities` for every model is in `GET https://api.router.ezat.io/api/models` (no key needed). The features column on [Models](https://docs.router.ezat.io/models.md) shows "Prompt caching" and "1 h cache".

## Claude Code

Claude Code places its own breakpoints, and its system prompt and first message stay the same for the whole conversation, so caching and sessions work with no setting. See [Claude Code](https://docs.router.ezat.io/tools/claude-code.md).

## For AI agents

1. Anthropic-format API: put `{"type": "ephemeral"}` (or with `"ttl": "1h"`) on the last tool, system block or message block of the stable prefix. At most 4 breakpoints.
2. OpenAI-compatible API: caching is automatic; don't add breakpoints unless you need a 1-hour lifetime. Send `x-ezat-cache: off` for one-off prompts.
3. For one conversation, send the same `x-ezat-session` (or `prompt_cache_key`) on every turn if its system prompt or first message changes. Never use one value for all traffic.
4. Check `cache_read_input_tokens` (Anthropic format) or `prompt_tokens_details.cached_tokens` (OpenAI format) on the second request to confirm reads.