Auto-compaction uses stale token usage after tool output, causing unrecoverable context overflow
Summary
In a long-running app-server turn, a large tool result can push the next sampling request beyond the model context window without triggering auto-compaction. The compaction decision uses the last server-reported token usage, which does not include the newly appended tool output. Once the request fails, small follow-up turns can remain unrecoverable because pre-turn compaction also does not account for incoming context updates and user input.
Observed behavior
From a production app-server rollout (identifiers and prompt contents omitted):
- Effective model context window: 258,400 tokens
- Last server-reported input usage: 242,701 tokens
- The next
function_call_outputpayload was 24,143 JSON bytes - No
context_compactedevent occurred - The next sampling request failed with an HTTP 400
- A tiny follow-up prompt on the resumed thread failed immediately with the same error
- Changing the selected model on resume did not recover the thread
The rollout contained 149 successful MCP tool calls and 164 token-usage updates before the failure, so this was not a model availability or authentication problem.
Suspected cause
Current main computes estimated_token_count after sampling, but compaction is gated only by token_status.token_limit_reached, which is derived from server-reported usage:
The estimate is currently only included in tracing. It is not used to decide whether the newly appended tool result makes another sampling request unsafe.
Pre-turn compaction has the same blind spot for newly incoming items; the source already calls this out:
The actual pre-turn gate checks only the persisted token status before recording incoming context/user input:
Expected behavior
- Before a follow-up sampling request, compaction should consider the estimated model-visible context after newly appended tool outputs.
- Before the first sampling request of a resumed turn, compaction should include pending context reinjection/diffs and the new user input.
- A thread near the context boundary should compact and remain resumable instead of becoming permanently stuck on 400 responses.
Suggested regression test
Use a deliberately small context/auto-compaction limit:
- Return server token usage just below the compaction threshold.
- Append a tool result large enough that the estimated next request exceeds the threshold/window.
- Assert that compaction runs before the next sampling request.
- Persist/resume the same near-limit thread with a small user input and assert that pre-turn compaction runs before sampling.
Possible fix direction
Use the estimated post-append token count in the mid-turn compaction decision (conservatively taking the maximum of server usage and the local estimate), and estimate pending incoming items in run_pre_sampling_compact before deciding whether to compact.
4 Comments
Potential duplicates detected. Please review them and close your issue if it is a duplicate.
Powered by Codex Action
Adding user-facing Codex Desktop evidence from duplicate issue #34017:
Codex ran out of room in the model's context window, short follow-up messages continued to fail with the same error./compactalso failed to recover the conversation; the thread remained unusable.stream disconnected before completion: Incomplete response returned, reason: max_output_tokensbefore the context-overflow message.This confirms the same recovery failure described here: stale or incomplete compaction accounting can leave a near-limit conversation permanently unrecoverable. Issue #34017 is being closed as a duplicate; its additional value is the Desktop
/compactreproduction and screenshot evidence.I found and validated an additional accounting edge case while implementing a candidate fix for this issue based on
rust-v0.147.0.A successful provider response can omit numeric token usage. A fix that keeps the last numeric usage value but moves its local-history boundary to the latest model-generated item is still unsafe: items after the last usage-bearing response and before the usage-less response disappear from the estimated growth, even though the provider baseline never included them.
The root cause is allowing the trustworthy numeric baseline and its corresponding local-history boundary to advance independently. Let
S0be the last trustworthy provider usage total andB0its corresponding boundary. If a later successful response omits usage, retainingS0while advancing the boundary causes the items between the two boundaries to disappear from locally estimated growth. For a compaction thresholdT, this can skip a required compaction whenever the partial estimate remains belowTbut the full growth afterB0would cross it. This accounting invariant is independent of the configured context-window size.The candidate fix treats a trustworthy usage sample and its history boundary as one atomic pair:
The regression fixture used a numeric 4,800-token baseline, a usage-less successful response, and enough locally appended data to cross a synthetic 5,000-token limit only when the baseline is retained. It also covered a newer numeric provider sample moving the boundary, legacy zero-valued usage placeholders on resume, fork reconstruction, rollback/history-replacement invalidation, and one-compaction loop prevention. A packaged loopback test made exactly four requests: two ordinary samples, the required compaction request, and the post-summary sample.
Current
mainstill bases the decision onget_total_token_usage()while the fresh post-sampling estimate is trace-only:This does not change the original report's fix direction; it adds a persistence requirement that prevents a partial fix from losing growth whenever a successful completion omits usage.
Your boundary is unusually concrete: a 24 KB tool result landed after a 242,701-token server baseline, no compaction occurred, and even tiny prompts on the resumed thread kept failing with HTTP 400.
Codex Rescue cannot fix Codex’s token accounting or perform speculative general compaction. I’m trying to validate whether the surviving rollout is diagnosed conservatively and whether a safe repository-state handoff is possible when the original thread cannot continue. It leaves the rollout untouched, never replays unknown calls, and verification may return
REVIEW_REQUIRED.If the affected session still exists, could you run:
pipx install codex-rescue==0.1.0a3codex-rescue sessionscodex-rescue doctor --latestPlease share sanitized output only—no rollout, database, tool contents, prompts, paths, or secrets.
https://github.com/shleder/codex-rescue