Bedrock prompt caching: agent cost cliffs come from replaying history, not model price

AI agents live a Groundhog Day: each tool call must re-send the whole story—tool definitions, instructions, prior calls and results—because the model has no durable memory between requests. By turn 20 you pay again for turns 1–19.

An AWS Machine Learning Blog post on OpenAI models on Amazon Bedrock argues the cost cliff is mostly this replay, not sticker price. Prompt caching stores the prompt’s KV working state after prefill and reloads it when a later request starts with the same tokens.

Caching can be implicit (Bedrock chooses the reusable prefix end) or explicit (your code marks prompt_cache_breakpoint). The post’s core recipe is to use both: an explicit marker after the stable tools-and-instructions block, while request-level mode stays implicit so the service also caches through the latest eligible conversation message.

Static prefixes need enough tokens to qualify (1,024 in the examples). prompt_cache_key decides which requests share an entry. Keep serialization byte-stable; put dates, user names and request IDs after the cached region. For growing history, either implicit rolling endpoints or explicit markers on the newest tool-result batch work—AWS measured ~62% and ~61% input-cost cuts on a six-turn GPT-6 Astra workload for those two styles.

Concrete patterns matter more than the metaphor. When tools change mid-session, mutating the initial TOOLS list invalidates the prefix; prefer allowed_tools / tool_choice over rewriting the catalog, or append additional_tools after the cached history. Different roles should use separate tool profiles, each with its own cached prefix.

For subagents, warm the shared prefix once, confirm cached_tokens / cache_write_tokens, then fan out helpers that append their task after the shared beginning. Side calls (summarizer, judge) deepcopy completed history rather than rebuilding it. TTL is at least 30 minutes and refreshes on reuse; session-specific keys isolate even matching prompts.

The steelman is that framework-level memory, summarization, or smaller tool schemas can shrink tokens without depending on provider cache semantics—and cache hits still require identical text, same model, account and Region. If your agent randomly reorders tools, injects timestamps into the system prompt, or rotates keys per session, you will pay for repeated writes and call it “caching.”

Also thin: how often production agents keep prefixes stable under MCP tool discovery; security notes that cached tools/prompts can leak and need governance; and how savings transfer when tool results dominate history length. The dollar figures are modeled Bedrock on-demand prices for a support-agent example, not a fleet A/B test.

For teams shipping coding or support agents on Bedrock, the decision is architectural: treat the shared prefix as a versioned artifact (support-agent:v7:…), not as whatever the framework serialized today. Observability starts with usage counters—if cached_tokens stay near zero, you do not have a caching problem, you have a stability problem.

Combined explicit+implicit is presented as the pattern that survives new tools and helper fan-out; explicit-only static breakpoints leave growing history uncached, which the post’s relative-cost chart flags as the common under-save.

Watch whether agent frameworks start emitting stable cache keys and breakpoints by default, or keep regenerating prompts each turn. If the industry standardizes on shared-prefix plus rolling history, agent unit economics tilt toward longer sessions. If not, every multi-tool agent will keep rediscovering Groundhog Day in the bill.