Low cache hit rate when Codex integrates with GPT-5.5

Open 💬 17 comments Opened Apr 30, 2026 by pyfdtic

💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

What version of Codex CLI is running?

0.125.0

Which model were you using?

gpt-5.5

What platform is your computer?

WSL2

What terminal emulator and version are you using (if applicable)?

Windows Terminal

What issue are you seeing?

When Codex integrates with the GPT-5.5 model, its cache hit rate is very low, which causes costs to be consumed rapidly.

In contrast, OpenCode integrated with GPT-5.5 has no such problem and maintains a high cache hit rate.

Besides, Codex paired with GPT-5.4 also achieves a high cache hit rate normally.

What steps can reproduce the bug?

We tested Codex integrated with GPT-5.4 and GPT-5.5 respectively. By checking the session logs of both versions, we can see there is a significant difference in the frequency of cached_input_tokens.

What is the expected behavior?

The expected behavior is that Codex with GPT-5.5 should maintain a normal high cache hit rate, consistent with the performance of Codex + GPT-5.4 and OpenCode + GPT-5.5, to avoid excessive and rapid cost consumption.

Additional information

_No response_

View original on GitHub ↗

17 Comments

github-actions[bot] contributor · 2 months ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

#20292
#20227
#20153
#20169
#19585

Powered by Codex Action

etraut-openai contributor · 2 months ago

Thanks for the feedback. We track cache miss rates closely on the server, and we don't see any anomalous behavior related to Codex and gpt-5.5. It's possible that the session you analyzed wasn't representative.

etraut-openai contributor · 2 months ago

We're receiving similar reports from other sources, so I'm going to reopen this issue until we're able to reconcile these reports with our server-side stats.

kkondaurov · 2 months ago

Can confirm. Codex Desktop on Mac OS with an API key.

Other users in our org with API keys also have this problem, although they're TUI/OpenCode users. So it's probably not about the harness.

Had to switch back to 5.4, where caching is totally fine. On my personal subscription, 5.5 is totally fine.

@etraut-openai I can share session ids with cache misses if it helps.

hsir · 2 months ago

Yes, I also confirm this. I am using the API Key billing, not a subscription. Whether using the CLI or the desktop version, when using GPT-5.5, the cache hit rate is almost always 0%. Once I switch back to GPT-5.4, the cache ratio returns to normal, at 90% or higher. I didn’t make any changes other than switching the model in Codex settings. I even checked the request body JSON shape, nothing wrong at all. I think there must be something wrong in OpenAI server

AetherMagee · 2 months ago

Can confirm too, GPT-5.5 cache hit rates are close to zero in both Codex and Opencode for me. GPT-5.4 is doing fine with otherwise identical configs.

etraut-openai contributor · 2 months ago

If you're seeing this problem, please use /feedback to upload the session and post the session ID here. Thanks!

kkondaurov · 2 months ago

If you're seeing this problem, please use /feedback to upload the session and post the session ID here. Thanks!

Hey @etraut-openai , reported these sessions via /feedback:

019df1e4-9669-76a1-bdde-280bd0d241ac
019df1e7-bf0a-7543-b3ec-30074cd7d528
019df1ed-1dc4-7221-9888-e5f75488ffde

ryanmadsenn · 2 months ago

Also been seeing this issue. Cache hit rate with 5.4 was ~94% and it dropped to 85% with 5.5. In certain instances I saw 8 consecutive cache misses within my session.

darinkishore · 2 months ago

I uploaded several affected sessions with /feedback / feedback/upload.

The pattern I see locally is not a steady low cache hit rate. The aggregate hit
rate is high, but there are isolated catastrophic collapses inside otherwise
healthy high-context sessions: 200k-240k input-token requests with 1-3% cache
hit rate, seconds after 98-100% hits, then immediately recovering.

Uploaded thread IDs and representative miss moments:

019dda83-e45e-7740-89ec-3c6c789c398d: 2026-04-29T18:49:55.375Z, rollout line 748, input 230,330, hit 2.8%, uncached 223,802. Prior token events were 99.6/99.8/99.7/99.9/99.8/99.5/99.8/99.8%.
019dda86-583a-7430-9b0f-af1a6dd3c755: 2026-04-29T19:34:29Z, rollout lines 826/839, input 229,641, hit 2.8%, uncached 223,113.
019dda9a-2e7f-7af0-a9cb-bcb63cec8b77: 2026-05-03T21:59:09.713Z, rollout line 119649, input 240,110, hit 1.4%, uncached 236,654. Prior token events: 98.6/98.3/97.3/99.7/99.1/99.8/99.9/99.8%; next events recovered to 99.6/99.7/99.7/98.9/99.8% after a 10.6s gap and only 178 new chars.
019def0d-178f-79d1-ad2d-5213fa7245c6: 2026-05-03T22:31:08.296Z, rollout line 8418, input 238,349, hit 2.7%, uncached 231,821. Prior token events: 99.9/99.6/99.7/99.1/99.7/99.8/99.5/99.5%; next recovered to 99.1/99.3% before compaction.
019ded0c-78f5-7cf2-9645-110e2c40bf47: 2026-05-05T00:00:35.489Z, rollout line 14782, input 218,023, hit 1.6%, uncached 214,567. Prior: 94.5/99.8/99.6/97.0/94.8/99.7/99.9/98.5%; next recovered to 99.9/99.8/99.7%.

Bulk local scan, 2026-04-29 through 2026-05-05 UTC:

160 rollout files, 138,559 token events.
75,142 fresh-ish token events after local replay/fork filtering.
gpt-5.5 fresh-ish input: 11.107B tokens.
gpt-5.5 fresh-ish uncached: 239.6M tokens.
gpt-5.5 fresh-ish hit rate: 97.8%.
Severe tail: 109 fresh-ish events with input >= 100k and hit < 10%.
Very severe tail: 28 fresh-ish events with uncached >= 200k.

This does not look like cache expiry: several examples happen 5-30s after
strong hits and recover immediately. It also does not look like the documented
">15 requests/min with the same prefix" cache-key overflow case; in the
uncached>=200k band I found 0 events with 15+ previous same-thread token
events in the preceding minute.

Related live-session observation while uploading feedback: two of the long
threads (019dda9a... and 019ded0c...) were not inert when resumed just to
submit /feedback; they resumed/continued paused work and had to be
interrupted again before or after upload. That may be related to the "old active
mega threads still updating" behavior users are seeing.

kkondaurov · 2 months ago

Still seeing these cache misses for 5.5, which make session cost skyrocket.

Submitted a couple more sessions via feedback, here are correlation ids:
930e9e7b-1f1e-40ff-acad-e709a6371646
2c22e6d4-a83a-49ed-82dd-a514936d918d

Raw1mage · 2 months ago

Codex Responses API — Prompt Cache Stuck at Tools-Only Floor

Status: Open. Likely server-side regression. Filed pending OpenAI response.

Reporter: OpenCode (third-party CLI/daemon built on top of @opencode-ai/codex-provider, a custom AI SDK V2 adapter that speaks the same Responses API wire format as codex-rs).

Affected: Codex Responses API via ChatGPT-subscription auth. Multiple GPT-5.4 and GPT-5.5 sessions show the same floor. Replicates across fresh sessions and multiple working directories.

---

Executive summary

usage.input_tokens_details.cached_tokens consistently caps at 4608 across an entire codex session, regardless of:

Model (gpt-5.5 and gpt-5.4 both affected)
Working directory / repo
Session length (turns 2–200+ all return 4608)
WS chain delta mode (delta=true with previous_response_id) vs full re-send (delta=false)
Instructions content (driver-only ~640 tok, or full system blob ~16k tok — same floor)
Wire shape of bundle items (raw-string content vs ContentPart[] array — same floor)
prompt_cache_key value (composite vs pure ${threadId} — same floor)
Bundle insertion position in input[] (mid-chain at lastUserIdx vs head at index 0 — same floor)

The 4608 figure ≈ instructions + tools schema token count for our setup. It looks like the prefix cache hit ends precisely where tools ends and the conversation chain begins.

A historical session under the same OpenCode binary on 2026-05-10 22:48–23:56 showed cache_read growing 35,840 → 49,152 → 117,248 → 137,216 → 181,248 across consecutive turns under delta=true AND delta=false. Cache worked until something changed between then and our next sessions.

Sessions starting 2026-05-11 all stuck at 4608. Same daemon binary lineage (we restarted the daemon several times for unrelated reasons), same @opencode-ai/codex-provider code path, same upstream codex-cli reference at f7e8ff8e50 (then 76845d716b after pull).

---

Reproduction

ChatGPT subscription account, OAuth credentials, codex Responses API endpoint via WebSocket transport (or HTTP fallback — same result)
model: "gpt-5.5" (also reproduces on gpt-5.4)
store: false, prompt_cache_key: "<sessionId>", previous_response_id: <chained>
instructions: stable text (we tested driver-only ~2.5KB and monolithic ~65KB; both hit 4608)
Send a turn, observe usage.input_tokens_details.cached_tokens = 0 (fresh)
Send 2nd turn with previous_response_id set
Observe cached_tokens = 4608 regardless of input_tokens (which grows naturally as conversation extends)
Send 100+ more turns. cached_tokens stays at 4608. Occasional one-turn jumps to 50,688 / 71,168 right after a chain reset, then back to 4608.

---

Diagnostic data

Sample USAGE telemetry across 49 turns of a single session

[CODEX-WS] USAGE input_tokens=18512 cached_tokens=0     hasPrevResp=false  ← turn 1
[CODEX-WS] USAGE input_tokens=22098 cached_tokens=4608  hasPrevResp=true
[CODEX-WS] USAGE input_tokens=22988 cached_tokens=4608  hasPrevResp=true
[CODEX-WS] USAGE input_tokens=31345 cached_tokens=4608  hasPrevResp=true
[CODEX-WS] USAGE input_tokens=42290 cached_tokens=4608  hasPrevResp=true
... 30+ more turns same pattern ...
[CODEX-WS] USAGE input_tokens=105448 cached_tokens=4608 hasPrevResp=false  ← chain reset
[CODEX-WS] USAGE input_tokens=97730  cached_tokens=50688 hasPrevResp=true   ← peak (1 turn)
[CODEX-WS] USAGE input_tokens=101093 cached_tokens=4608 hasPrevResp=false   ← back to floor

Historical comparison (working session 2026-05-10)

[CODEX-WS] USAGE input_tokens=36197  cached_tokens=0      hasPrevResp=false  ← turn 1
[CODEX-WS] USAGE input_tokens=36610  cached_tokens=35840  hasPrevResp=true
[CODEX-WS] USAGE input_tokens=49562  cached_tokens=36352  hasPrevResp=true
[CODEX-WS] USAGE input_tokens=50162  cached_tokens=49152  hasPrevResp=true
[CODEX-WS] USAGE input_tokens=53760  cached_tokens=53760  hasPrevResp=true   (effectively 100%)
... cache continues to grow as chain extends ...
[CODEX-WS] USAGE input_tokens=215840 cached_tokens=181248 hasPrevResp=false  ← 84% on full re-send

Request body fingerprint (current broken sessions)

{
  "model": "gpt-5.5",
  "instructions": "<driver text, ~640 tok, byte-stable across turns, sha256:24c40934fe30...>",
  "input": [
    { "role": "developer", "content": [{"type":"input_text","text":"<role_identity + opencode_protocol bundle, 3850 tok, byte-stable>"}] },
    { "role": "user",      "content": [{"type":"input_text","text":"<AGENTS.md + environment_context bundle, 3940 tok, byte-stable>"}] },
    /* ... conversation history reconstructed from previous_response_id chain ... */
  ],
  "tools": [/* 11 function tools, schema byte-stable */],
  "tool_choice": "auto",
  "parallel_tool_calls": true,
  "prompt_cache_key": "ses_1eca32cc1ffelyxaN1FlERPqRS",
  "store": false,
  "previous_response_id": "resp_0af46a9adff..."
}

driverHash, developerBundle.totalChars, userBundle.totalChars, prompt_cache_key, model, tools — all verified byte-stable across turns via local telemetry. No drift.

---

What we tried (none lifted the floor)

✅ Persona alignment: replaced our 27-line custom driver with upstream BaseInstructions::default() (275 lines from refs/codex/codex-rs/protocol/src/prompts/base_instructions/default.md, md5 7a62de0a7552d52b455f48d9a1e96016). No change.

✅ prompt_cache_key = thread_id (matching upstream client.rs:713): switched our adapter from a composite key to pure threadId. Verified the value is byte-stable across all turns. No change.

✅ Wire structure aligned to upstream: instructions = driver only; input[] opens with one developer bundle item + one user bundle item, mirroring build_initial_context() in core/src/session/mod.rs:2553-2761. No change.

✅ Bundle position at input[0]: confirmed bundles always at chain head, not mid-conversation (transport delta-slice correctly trims them out as already-in-chain). No change.

✅ Bundle content as raw string vs ContentPart[]: tested both shapes. Raw string matched pre-break (May 9) wire that historically cached well. Both forms hit 4608. Reverted to ContentPart[] for upstream alignment.

✅ finishReason fallback fix: when terminal response.completed event fails to arrive but text was emitted, we now default to "stop" instead of "unknown". Stops a runloop empty-round guard from issuing chain resets every few turns. Visibly stabilized chain (see #6 in upstream impact below) but did not improve cache_read floor.

✅ Daemon restart: clean process state. Same floor.

✅ New session: cache=0 on turn 1 (correct), cache=4608 on turn 2 (stuck), confirmed across 4 fresh sessions.

✅ gpt-5.5 → gpt-5.4: same floor.

✅ Re-auth / new credentials: re-authenticated the ChatGPT subscription credential and started a fresh session — same floor on the new credential.

---

Hypothesis status

Wire-shape mismatch (raw string vs ContentPart[], bundle position, instructions size): Falsified by experiments 4–5 above.
Account / credential state: Plausible — the historical working session used a credential that has since gone into 5h cooldown, so we cannot directly retest with it.
Server-side regression: Strongly supported by three open OpenAI issues:
openai/codex#20301 — "Low cache hit rate when Codex integrates with GPT-5.5"
openai/codex#21756 — "Conversation cache unexpectedly drops to nearly zero during short continuous sessions" (filed 2026-05-08, exact match for our symptom)
openai/codex#21796 — Codex's own engineers observe ~55% cache hit on byte-identical prefixes, indicating a broader cache anomaly even under ideal conditions

All three issues open, no assignee, no resolution.

---

Operational impact

A single chat turn under cache=4608 with input ≈ 80k tokens costs full price for ~75k uncached tokens. The 5h ChatGPT subscription quota gets exhausted in roughly 10 minutes of active multi-turn use, where the same workload before the regression was sustainable for hours.

For any third-party tool relying on the Codex Responses API for non-trivial multi-turn work — agent loops, repo-wide code edits, long debug sessions — the cache regression is a hard blocker.

---

What we would like from OpenAI

Acknowledge the regression in one of the three open issues.
Confirm or refute server-side cache changes between 2026-05-10 and 2026-05-11.
Document the actual prefix cache prefix calculation for Responses API requests with previous_response_id set — the public docs do not address chain-mode interaction with prefix cache, leaving a black-box where third-party clients cannot self-debug.
Surface an extended_cache_signature or similar explicit field clients can use to confirm what prefix the server hashed for cache lookup. Right now we have no way to tell whether our prompts are byte-stable from the server's perspective.

---

Repro environment

OpenCode daemon (TypeScript/Bun), @opencode-ai/codex-provider ^x.x
refs/codex submodule pinned at 76845d716b (rust-v0.0.2504301132-6092)
Linux x64, WSL2 Ubuntu
ChatGPT subscription auth (OAuth flow)
Tested: gpt-5.5, gpt-5.4
WebSocket transport (also reproduces on HTTP SSE fallback)
store: false
service_tier: undefined (default)
reasoning.effort: configured (no impact tested)

---

Related references

Upstream wire layout: refs/codex/codex-rs/core/src/session/mod.rs:2553-2761 (build_initial_context())
Upstream cache key: refs/codex/codex-rs/core/src/client.rs:713 (prompt_cache_key = self.state.thread_id.to_string())
Upstream BaseInstructions default: refs/codex/codex-rs/protocol/src/prompts/base_instructions/default.md
OpenAI prompt-caching guide: https://developers.openai.com/api/docs/guides/prompt-caching

MrLightspeed · 2 months ago

019e2e4d-e84b-78e3-bfb1-0b0c923200d5

danielsan1 · 2 months ago

This might be connected https://github.com/openclaw/openclaw/issues/84110 Started with 2026.5.12 update

makoto-soracom · 1 month ago

019e4e09-2723-74f0-ad76-496e30ab9a70

makoto-soracom · 1 month ago

I am not attaching the raw payload JSON publicly because it contains full instructions, tool schemas, tool outputs, local paths, and request metadata. I checked the relevant payloads locally and did not find API keys,
bearer tokens, cookies, refresh tokens, or access tokens, but the raw files still contain more context than should be posted in a public issue.

The interesting part is a sequence of three consecutive Responses requests in the same thread:

previous request
started: 2026-05-22T04:56:09.758Z
completed: 2026-05-22T04:56:16.306Z
input_tokens=29658
cached_input_tokens=26496
low-cache request
started: 2026-05-22T04:56:18.856Z
completed: 2026-05-22T04:56:21.451Z
input_tokens=30168
cached_input_tokens=3456
following request
started: 2026-05-22T04:56:23.098Z
completed: 2026-05-22T04:56:25.136Z
input_tokens=30394
cached_input_tokens=30080

The middle request is the low-cache one. Comparing it with the previous request:

instructions are identical
tools are identical
model is the same: gpt-5.5
prompt_cache_key is the same
Codex WebSocket diagnostics report outcome=incremental
request_without_input_hash is stable
request_tools_hash is stable

Timing also makes cache expiry unlikely:

previous request start -> low-cache request start: about 9.1s
previous response complete -> low-cache request start: about 2.6s

So this looks like a single intermittent prompt-cache collapse inside an otherwise healthy same-thread incremental Responses API chain. The following request recovered immediately to cached_input_tokens=30080.

kyle7zhang · 1 month ago

I’ve encountered the same issue before.

It seems to happen when the WebSocket of the backend-api/codex/responses reconnects. If the user continues speaking after being idle for a while, the connection reconnects, and the cached_tokens counter resets and starts over. I suspect this is related to the new retention/eviction policy in GPT-5.5, and possibly to the request no longer being routed to the same backend instance after reconnection (just an assumption).

It seems much more noticeable when using Azure Codex, since we typically use HTTPS/SSE instead of WebSockets.

For example,

Previous WebSocket request: input_tokens 183426, cached_tokens 152448 (expected)
id:"resp_0442f7dc3e2b541c016a1a095214588199bbeabf8cf52eef1e"
prompt_cache_key:"019e74ff-6cf1-7d40-80ce-0c8baa3ad6cf"

New WebSocket request: input_tokens 184739, cached_tokens 91520 (dropped a lot)
id:"resp_0fd6c96548b5311c016a1a099e911081938f04ba42de133e60"
prompt_cache_key:"019e74ff-6cf1-7d40-80ce-0c8baa3ad6cf"