Intermittent zero prompt-cache hits despite stable prompt_cache_key and repeated/same-prefix Codex requests

What issue are you seeing?

We are seeing intermittent zero prompt-cache hits for Codex requests despite stable prompt_cache_key usage and repeated/same-prefix requests.

This may be expected best-effort cache behavior, but we would like to confirm whether this pattern is normal or indicates a cache eligibility/routing issue.

Sanitized observations:

row input_tokens cached_tokens output_tokens total_time_ms status
━━━━━━ ━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━
1001 68,026 64,000 572 23,034 success
────── ────────────── ─────────────── ─────────────── ─────────────── ───────────────
1002 74,311 0 0 180,116 local timeout
────── ────────────── ─────────────── ─────────────── ─────────────── ───────────────
1056 74,371 0 6,848 249,649 success
────── ────────────── ─────────────── ─────────────── ─────────────── ───────────────
1058 74,371 0 1,130 43,072 success
────── ────────────── ─────────────── ─────────────── ─────────────── ───────────────
1059 74,371 0 342 14,984 success

Rows 1056, 1058, and 1059 are replays of the same dumped request after fixing our local timeout behavior. Our local request fingerprint was stable for those
replays, but cached_tokens remained 0.

We also observed adjacent traffic where cache clearly works and then drops unexpectedly:

row input_tokens cached_tokens output_tokens status
━━━━━━ ━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━ ━━━━━━━━━
1054 100,093 97,792 830 success
────── ────────────── ─────────────── ─────────────── ─────────
1055 101,367 0 406 success
────── ────────────── ─────────────── ─────────────── ─────────
1057 101,789 0 954 success

What steps can reproduce the bug?

1. Send large Codex requests through the Responses API / Chat Completions compatibility path.
2. Include a stable prompt_cache_key.
3. Repeat the same request, or send same-prefix continuation requests with the same tools and stable prompt cache key.
4. Inspect returned usage metadata, especially prompt_tokens_details.cached_tokens.

In our telemetry, repeated/same-prefix requests sometimes return high cached_tokens, but other repeated or expected-cache requests return cached_tokens=0.

What is the expected behavior?

We expected repeated or same-prefix requests with a stable prompt_cache_key to usually receive non-zero cached tokens, especially when the same session
is used and adjacent requests have already shown high cache hits.

If Codex prompt caching is intentionally best-effort or non-deterministic, the expected behavior may simply be that cached_tokens=0 can happen. In that case,
we would like confirmation and any guidance on what request fields affect cache eligibility.

Additional information

This was initially discovered while debugging a local router timeout issue. That local issue has been fixed: our router had incorrectly applied a 180s timeout to
the full streamed body rather than only to the initial upstream response/header wait.

After fixing that local timeout behavior, the large replayed requests complete successfully, but prompt-cache hits remain inconsistent.

What we have ruled out locally:

• prompt_cache_key is present and stable.
• The issue is not global cache disablement, because adjacent traffic can show high cache hits.
• In repeated replay rows, our local request fingerprint was stable while cached_tokens stayed at 0.

Questions:

1. Are zero cached tokens expected for repeated or same-prefix Codex requests even with a stable prompt_cache_key?
2. Can tool schema, tool count, large tool outputs, final message role, or long context size affect prompt-cache eligibility?
3. Are there additional request fields or response headers we should log to debug cache eligibility?
4. Is prompt_cache_key expected to influence Codex prompt caching in this API path, or is it only a best-effort hint?

Redacted by @codex