Running out of room in the Codex context window is immediately fatal to that chat thread

Are you using OpenAI models through the OpenAI endpoint, or are you using a custom model provider? If you're using the latest OpenAI model defaults, you should be seeing auto-compaction when your context window usage approaches 90%.

I am only using the provided OpenAI models.

In the last sesssion, I'm pretty sure I did see the token counter reset a lot at one point. But just now it filled the token window again, and locked the chat. Somewhat ironically I used ChatGPT to attach the session JSONL and got it to give me a progress summary and a kick off for the next chat.

Interestingly I did change the model to GPT 5.1 (not codex) and it was while using that model that the token counter reset. In many ways the GPT 5.1 model feels... nicer. I have very little to compare these experiences to, so I mention it just because...

And just now when the token window ran out and locked the chat, I was using GPT 5.1 Codex... So... I only have really two data points, so co-incidence is a real possibility.

(I cannot get any custom model to appear in the list. No matter how hard I try. Frankly your docs about it are impossibly confusing. I have been using ChatGPT 5.1 to try to get it working, but nothing it tries actually works.

but that problem is really for another bug report.)

I had a similar issue. What ended up being fixing my issue was just there was do much text in my prompt. If I removed some of the logs I wanted chat to search through and split it up into sections it would work find but one big block wouldn't. Idk the technical details but worth a shot if your still stuck on this.

Following up on @etraut-openai's note that auto-compaction should fire near 90% — that pre-turn path works, but there are three failure modes where the sampling request still reaches the hard context limit before the threshold trips:

1. Single large user message — a paste/attach pushes the prompt past the window in one shot, so the 90% pre-sampling auto-compact never has a chance to run before the overflow.
2. Tool output explosion mid-turn — a shell / apply_patch / MCP call returns much more than expected and the conversation lands past the window inside the turn.
3. Model-reported token undercount — the 90% trigger uses total_token_usage, which can lag the real prompt size when the provider rounds down or batches token accounting.

In all three cases the user lands on CodexErr::ContextWindowExceeded and sees "Codex ran out of room in the model's context window." with no in-place recovery, even though run_auto_compact(MidTurn, BeforeLastUserMessage) is generally enough to free the budget and let the same turn continue.

Proposed safety net (prototyped locally)

run_turn catches CodexErr::ContextWindowExceeded once per turn, runs the existing mid-turn auto-compaction, resets the WS session if the inner compaction asked for it, and retries the sampling request. A turn-scoped flag prevents recovery from running more than once, so we cannot loop on repeated overflows — subsequent failures fall through to the existing generic-error path that emits EventMsg::Error. The user-visible warning event already emitted by compact.rs ("Heads up: Long threads and multiple compactions...") still fires, so the compaction is visible.

This is additive: pre-turn 90% auto-compact and the existing pre-turn context-window-failure behavior are unchanged.

Tests in the local branch

• Updated context_window_error_sets_total_tokens_to_model_window (client suite) — asserts the recovery + retry end-to-end.
• New auto_compact_recovers_after_sampling_context_window_error — non-OpenAI provider path (4-step SSE sequence: turn1, CW error, recovery compact, retry).
• New remote_auto_compact_recovers_after_sampling_context_window_error — remote ChatGPT/responses path with dedicated compact mock.

cargo fmt --check -p codex-core and cargo clippy -p codex-core --tests --no-deps -- -D warnings are clean locally.

Happy to send a PR with this approach if the team would like to invite it — per the contributing guide I'm holding off until there's an explicit go-ahead, and would also welcome direction if a different shape (e.g. surfacing the recovery as a user-visible event rather than relying on the existing compaction warning) would land more cleanly.

Following up on the original comment after finding #22141 (merged then reverted by #22170 on 2026-05-11) — I refined the local branch so the patch addresses the three Codex Review P2 concerns raised on that earlier attempt.

How this version differs from #22141

| Codex Review P2 from #22141 | How the refined patch handles it |
|---|---|
| "Do not rely on compacted text for the current turn" — build_compacted_history_with_limit applies the 20k cap to all messages including the just-submitted overflowing turn, silently truncating it. | New LastUserMessagePolicy { Cap, AlwaysInclude } plumbed through run_inline_auto_compact_task → run_compact_task_inner → build_compacted_history. Only the recovery callsite passes AlwaysInclude — pre-turn auto-compact, manual /compact, downshift, and rollout reconstruction continue to use Cap (no behavior change). With AlwaysInclude, the latest user message bypasses the cap entirely; the cap still bounds earlier history. |
| "Rebuild tool routing after recovery compaction" — router is built once in the sampling retry loop, so build_prompt reuses a stale router after compaction inside that loop. | The recovery is placed in run_turn's outer loop (around run_sampling_request), not inside the inner retry. Each continue re-enters run_sampling_request, which re-runs built_tools(...) against the post-compaction input — router naturally fresh. |
| "Pass hooks the recovered request input" — sampling_request_input_messages is captured pre-recovery, so AfterAgent hooks see the pre-compaction input while the model answered the post-compaction one. | sampling_request_input_messages is computed inside the run_turn loop, after sampling_request_input is rebuilt from sess.clone_history(). Each iteration after recovery recomputes both — hooks always see what the model actually answered. |
| Multimodal preservation (Vec<String> → Vec<Vec<UserInput>> refactor introduced in #22141) | Same refactor applied locally: collect_user_messages → Vec<Vec<UserInput>>, new user_message_text and truncate_user_message helpers, build_compacted_history* accept &[Vec<UserInput>], images are preserved verbatim while only text segments are subject to the cap. |

Test coverage in the local branch

Unit tests in compact_tests.rs:

• collect_user_messages_preserves_user_content — image + text content survives the collector.
• truncate_user_message_preserves_text_segment_order_around_images — truncation only shortens text, preserves image position.
• build_token_limited_compacted_history_truncates_overlong_user_messages — regression-fence: Cap policy still truncates as before.
• build_compacted_history_with_always_include_preserves_full_current_turn — AlwaysInclude keeps the latest oversized message verbatim.
• build_compacted_history_with_always_include_still_caps_earlier_history — AlwaysInclude still caps the rest of history.
• build_compacted_history_with_always_include_preserves_current_turn_images — multimodal current turn survives.

Integration tests in tests/suite/compact.rs:

• auto_compact_recovers_after_sampling_context_window_error — end-to-end recovery (non-OpenAI provider).
• auto_compact_recovery_preserves_multimodal_user_turn — image + text overflow, asserts the retry preserves both exactly once and the body never contains a tokens truncated marker.

Integration test in tests/suite/compact_remote.rs:

• remote_auto_compact_recovers_after_sampling_context_window_error — equivalent end-to-end coverage on the remote/ChatGPT responses path.

Plus the existing context_window_error_sets_total_tokens_to_model_window (client suite) updated to assert the recovery flow.

cargo fmt --check -p codex-core and cargo clippy -p codex-core --tests --no-deps -- -D warnings are clean locally.

What I'm asking

Would the team be willing to invite this as a PR? Happy to also (a) split it into two commits (multimodal refactor + recovery policy) for easier review, (b) port the additional bounded-retry tests normal_loop_context_window_error_stops_after_sample_retry_budget and normal_loop_context_window_error_stops_after_remote_compaction_failure from #22141 if the team wants the same termination guarantees, or (c) take a different shape entirely if there's an internal direction I'm missing.

Following up on my two earlier comments: I rebased the branch onto current main and it's ready for review whenever the team has bandwidth. I tried to open a PR but openai/codex blocks CreatePullRequest for non-collaborators (invitation-only, enforced at the GitHub permission level), so I'm linking the branch here instead:

https://github.com/emersonbusson/codex/tree/fix/context-window-recovery-compact

Since the last update I also ported the two bounded-retry tests I'd offered in option (b), adapted to this design's turn-scoped flag rather than #22141's stream-retry budget:

• remote_auto_compact_recovery_stops_after_single_retry_on_repeated_overflow — proves recovery runs exactly once per turn (one compaction + one retry, then the terminal error) and cannot loop.
• remote_auto_compact_recovery_stops_when_compaction_fails — proves a failed recovery compaction ends the turn without retrying sampling.

Local checks on the rebased branch:

• cargo fmt --check ✓
• cargo clippy -p codex-core --tests --no-deps -- -D warnings ✓
• cargo test -p codex-core -- compact ✓ (new + existing recovery/termination tests)

The three Codex Review P2 concerns from #22141 are still addressed exactly as described in my previous comment (recovery in run_turn's outer loop, LastUserMessagePolicy::AlwaysInclude for the current turn, hooks/router rebuilt from post-compaction input). The change stays additive to the existing pre-turn 90% auto-compaction.

Happy to open the PR the moment it's invited, split the multimodal refactor from the recovery policy into two commits, or reshape it if there's an internal direction I'm missing.

<img width="862" height="352" alt="Image" src="https://github.com/user-attachments/assets/ba4c89fc-776a-4918-83bc-c33d7372f427" />

I have just ran into issue 😄 In codex app, spark model

Ive faced the error exactly for one of the reasons listed above:
-Auto compaction mid execution, running a powerful "Continue" continued the session
-On very long threads like mine truncating the conversation history directlt from ~./codex chat history gives you some more room to wiggle not the best but it does work

*For some extra troubleshooting agentic level, i make codex cli truncate it for me, i have a massive set of MD files to restore short and mid term context of the project so losing that thread length is affordable

i fix this in: https://github.com/emersonbusson/codex

Still an ongoing issue, and seems reported by many for months.

Sent feedback:
019eccf6-1871-7201-92d4-cf0b69975590

I’ve burned through nearly 3 billion tokens in my lifetime using Codex, but over the last two days, I hit the exact same error twice. First time ever. Weird.

Adding a multi-agent consequence of this bug that is not captured by the thread-level UX alone.

When a spawned subagent receives ContextWindowExceeded, the error currently bubbles out of the subagent turn immediately. The supervisor receives the failed/closed subagent result and may close it and spawn a replacement. That replacement has none of the previous subagent’s working context, so the work accumulated by the original subagent is lost.

The problematic sequence is:

1. A long-running subagent accumulates context and sends a Responses request.
2. The backend rejects it with ContextWindowExceeded before the local threshold triggers compaction.
3. The subagent turn returns the error immediately.
4. The supervisor observes a failed child and starts a new subagent instead of the original child recovering.

For this path, the subagent should get one bounded recovery attempt: run auto-compaction, then retry the failed sampling request once. Only if compaction fails, or the retried request also exceeds the context window, should the error bubble to the supervising agent. The attempt must be bounded to avoid an overflow/compaction retry loop.

This appears to be the same underlying recovery gap discussed in this issue, but the multi-agent impact is more severe because bubbling the first error can discard a substantial amount of in-progress child-agent work rather than merely ending an interactive turn.