Prompt caching

Long conversations, big system prompts and agent tools repeat the same tokens on every request. Prompt caching lets those repeated input tokens be read from a cache. Cached input is billed at the lower cached input rate shown on each model page.

It is automaticCopy Anchor Link

You don't need to turn anything on, and you don't need cache_control breakpoints. They are accepted for compatibility and ignored. Every request is checked against recent requests: the longest matching prefix of the prompt is read from the cache.

Hits are best-effort. An identical prompt can still miss, for example after a period of inactivity, and there is no fixed retention time.

What must stay the sameCopy Anchor Link

The cache matches the beginning of the prompt exactly, so these must be identical, in the same order:

• the model
• the system or developer instructions
• the tool definitions
• every earlier message

Appending a new message at the end keeps the earlier prefix cacheable. Editing or reordering earlier messages, changing tools or instructions, or putting a timestamp at the top of the system prompt breaks the match from that point on.

Rule of thumb: put what never changes first (instructions, tools, long documents), and what changes last.

Routing hintsCopy Anchor Link

Requests are spread across many servers. A routing hint sends all requests of one conversation to the same cache:

Both hints are scoped to your organization. The API never forwards the value you send: it forwards a hash that is specific to your organization. Caching therefore works across all the keys of your organization, and your values can never collide with another customer's.

Use one stable, opaque value per conversation or workload, for example your conversation id. prompt_cache_key takes any string up to 1024 characters, and a value that is not a string fails with 400 invalid_request. Shorter is better: 128 characters or fewer is plenty. Don't reuse one value across unrelated traffic, and use an id rather than prompts, personal data or secrets. If you send both hints, give them the same value.

curl https://api.interodin.com/v1/chat/completions \
  -H "Authorization: Bearer $INTERODIN_API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-interodin-session-id: conv_7d2a9f" \
  -d '{
    "model": "odin-v1",
    "prompt_cache_key": "conv_7d2a9f",
    "messages": [
      {"role": "system", "content": "You are a concise coding assistant."},
      {"role": "user", "content": "Review this function."}
    ]
  }'

The same goes for the end-user fields user and safety_identifier (Chat Completions and Responses) and metadata.user_id (Messages). You can send your own end-user id in them. They take a string of up to 1024 characters, and the API forwards only a hash that is specific to your organization.

When you send many requests in parallel with the same long prefix, let the first one start streaming before you send the rest. That gives the cache entry time to form.

Reading cached tokensCopy Anchor Link

When streaming, read these from the final usage event:

• Chat Completions: the last chunk.
• Responses: response.completed.
• Messages: message_start.

A value of 0 means that request had no cache hit. Your usage in the console may show cached tokens too.

Coding agentsCopy Anchor Link

Tools like Claude Code and Codex send several API requests per visible turn: tool calls, retries and context compaction. Each request is billed separately, and each benefits from the cache. Claude Code needs no extra setup, because caching is automatic.