Native Bedrock Codex GPT-5.6 Sol lacks explicit cache controls, producing high cache-write spend

For the telemetry part of this request, one semantic detail is worth making explicit in tests/UI: cache_write_input_tokens should be shown as its own diagnostic bucket, but it should not automatically be added on top of provider input_tokens when deriving total usage/cost unless the provider contract says those buckets are disjoint. The same caution already applies to cached_input_tokens, which is a subset of Codex input.

I keep cache-write/read as separate observability fields in a small local Codex rollout monitor: https://github.com/tylerchen0123-sudo/CODEX-Inspection-Guidelines-for-Dosage/blob/main/collector.py

A useful cross-provider fixture here would contain nonzero input + cache-read + cache-write and assert both: (1) the raw provider fields survive serialization unchanged, and (2) the presentation/cost layer does not accidentally double-count a subset bucket. That would make the new Bedrock cache controls measurable without baking one provider’s bucket semantics into every downstream consumer.

@rka-oai, tagging you since you worked on #36981, which appears related to this issue.

After upgrading from 0.146.0 to 0.147.0 with the Amazon Bedrock provider, the cache write-to-read ratio increased from 0.08 to 8.84, and my daily cost rose to at least 5x its previous level.

After rolling back to 0.146.0, the ratio immediately returned to 0.025 with a 97% cache-read rate.

Would you mind taking a look at whether remote compaction could be invalidating the Bedrock prompt cache?

Potential patch implementing the request-body side of this issue (explicit prompt_cache_options + a prompt_cache_breakpoint on the last cacheable developer-prefix block):

https://github.com/openai/codex/compare/main...devonpmack:codex:feat/gpt-56-explicit-prompt-cache

I had Sol put together a workaround/fix for this: https://github.com/scottleibrand/codex/pull/1. Mantle implicit caching was not advancing across growing session history, causing Codex to repeatedly write most of the prompt instead of reading the previously cached prefix. So this sets explicit cache breakpoints at stable and recent-history boundaries, reproducing the same desired input caching behavior seen with the direct OpenAI inference.

Testing against production workloads on Mantle, we saw:

• 35 Bedrock/Luna rollouts
• 1,486 provider responses
• 30 multi-request sessions: all cache-healthy
• 97.771% aggregate cache-read ratio after seed requests
• 99.958% aggregate read-or-write coverage
• Lowest per-session coverage: 99.849%

Since direct upstream PRs are by invitation only, I had it PR this on my own fork: https://github.com/scottleibrand/codex/pull/1

I don't necessarily expect anyone to want to merge this workaround upstream as-is, but affected users should be able to test/use this until a proper fix is in place.

This does not directly solve https://github.com/openai/codex/issues/35300, which appears to involve cross-session cache-key/reuse behavior. The explicit cache-control plumbing may nevertheless be relevant, so I’m referencing it here.

Confirmed the request-shape gap on main (1f41cc5d92): the Responses request structs carry only prompt_cache_key (codex-api/src/common.rs#L41, #L270, threaded into both HTTP and WS request builders) — no prompt_cache_options, no per-block breakpoint field anywhere in codex-api. So on Bedrock, where caching for Sol is explicit-opt-in rather than implicit, every request re-writes the full prefix, which is exactly the 85%-cache-write / zero-cache-read profile you measured.

Two observations for the implementation:

1. Codex already knows where the breakpoint belongs. The request assembly has a clean seam between the stable prefix (base instructions, tool specs, environment context — constant across a session's turns) and the per-turn tail; the natural placement is after the last stable item, and codex constructs that boundary itself, so "safe placement strategy" doesn't need heuristics — just a marker emitted at assembly time when the provider capability gate is on.
2. Your item 4 (usage telemetry) has a cheap first step: cache_write_input_tokens/cached_input_tokens already come back in Bedrock usage payloads (your local session read them), but codex's token accounting only tracks cached_input from first-party responses — surfacing both fields per-turn in /status + exec --json usage would let users see the miss pattern without Cost Explorer archaeology. That also helps first-party users diagnose cache-invalidation churn (#38524's compaction-thrash pattern produces the same signature).

The capability gate can hang off the provider config rather than model info, since this is a Bedrock-Mantle contract detail — which also keeps the change orthogonal to the catalog-staleness problems in #37813.

this is most likely caused by cache miss caused by newly turned on web search tool and we are working with bedrock team for the fix. In the meantime, feel free to disable the tool as a workaround (in config, add web_search = "disabled"). note codex will still be able to perform web search via other tools such as shell commands

this is most likely caused by cache miss caused by newly turned on web search tool and we are working with bedrock team for the fix. In the meantime, feel free to disable the tool as a workaround (in config, add web_search = "disabled"). note codex will still be able to perform web search via other tools such as shell commands

thanks for the reply. Will try this out and confirm. Do we expect a real fix at some point?

I tested this on 0.148.0 with Bedrock and the workaround works. With web_search = "cached", cache reads stayed at zero; with web_search = "disabled", caching resumed normally.

I feel like this should be treated as a critical bug. It can quickly chew up thousands of dollars of tokens and you only notice if you are paying close attention. Whatever caused the regression should be reverted or fixed asap.

I feel like this should be treated as a critical bug. It can quickly chew up thousands of dollars of tokens and you only notice if you are paying close attention. Whatever caused the regression should be reverted or fixed asap.

Yep.

I've a bill for $1200+ now that would have normally been 300USD.

Thankfully I checked a day later or it would have been far worse.

But still, really annoying when I didn't do anything wrong.

I lump myself into $1k+ bill territory due to this bug. I wonder if AWS would be amenable to a discount or something 😩

the deployment of the fix has gone through on the bedrock side for web search tool. I verified that the fix works in us-east-2, us-west-2 and us-east-1. Closing this out - feel free to open a new one if the issue persists.

promptCacheKey tells the provider where to route the request. cache_control is
what marks the prefix as cacheable. Send the first where the second is expected
and every request comes back fine while nothing gets cached at all.

A read costs 0.1 of a normal input token. So on a stable prefix that mistake
means paying about ten times what you should, on every repeated token, until
somebody reads the raw request body.

Translation layers keep doing this. There are open issues right now with the
same shape against an Anthropic to Bedrock translator in one of the open source
gateways, and against a vendor orchestration layer in a big enterprise
suite.