bug: Compaction can overwrite concurrent history writes and loop indefinitely

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• #13279

Powered by Codex Action

I traced both issues and have a proposed fix. Sharing here since external PRs are restricted.

Root cause analysis

Finding 1: Race condition in compact.rs and compact_remote.rs

The window between the final snapshot and replace_compacted_history() is larger than just snapshot ②→replacement — the compaction task itself takes 2–10 seconds (model API call). Any items appended during that entire window (e.g. subagent notifications via inject_user_message_without_turn, background events) are silently dropped.

Fix: take one more snapshot immediately before calling replace_compacted_history() and merge any items that appeared since the original snapshot:

// compact.rs (same pattern applies to compact_remote.rs)
// add before the CompactedItem { ... } block
let latest_history_snapshot = sess.clone_history().await;
let latest_history_items = latest_history_snapshot.raw_items();
if latest_history_items.len() > history_items.len() {
    new_history.extend_from_slice(&latest_history_items[history_items.len()..]);
}

This assumes append-only semantics during compaction (new items only grow the tail), which matches the current architecture.

Finding 2: Mid-turn compaction infinite loop in codex.rs

run_turn() retries compaction in a loop with no cap. If the summary itself is large enough to keep token usage above auto_compact_limit, the loop never exits.

Fix: add a counter and abort after 3 attempts:

// add before the mid-turn compaction block in run_turn()
let mut mid_turn_compaction_attempts: u32 = 0;

// inside the if token_limit_reached && needs_follow_up block:
mid_turn_compaction_attempts += 1;
if mid_turn_compaction_attempts > 3 {
    error!(
        turn_id = %turn_context.sub_id,
        mid_turn_compaction_attempts,
        "mid-turn compaction exceeded retry limit"
    );
    return None;
}

Working implementation

Full implementation with both fixes: https://github.com/Lordanakun/codex/commit/a460195acabbcaa7238ef886a1264c654e5544ef

3 files changed, 25 insertions. Happy to adjust the approach if maintainers prefer a different strategy (e.g. holding a write lock vs. the merge-on-replace approach).

Update: Just validated by building from source on Apple M4 (Rust 1.93.0 / aarch64). One borrow checker fix needed in compact_remote.rs — the temporary clone needs to be bound to a named variable:

// compact_remote.rs — corrected version
let history_clone = history.clone();
let history_items = history_clone.raw_items();
let ghost_snapshots: Vec<ResponseItem> = history_items
    .iter()
    .filter(|item| matches!(item, ResponseItem::GhostSnapshot { .. }))
    .cloned()
    .collect();

// ... (later, before replace_compacted_history)
let latest_history_snapshot = sess.clone_history().await;
let latest_history_items = latest_history_snapshot.raw_items();
if latest_history_items.len() > history_items.len() {
    new_history.extend_from_slice(&latest_history_items[history_items.len()..]);
}

The same split is not needed in compact.rs because that file already uses history_snapshot (a prior clone) rather than moving history directly.

Updated branch: https://github.com/Lordanakun/codex/tree/fix/compaction-race-and-infinite-loop

Hey there was a bug in the Responses API /compact endpoint that did not properly truncate replacement_history, causing compaction to barely reduce token % in longrunning conversations. This was causing lots of reports of compaction looping or happening very frequently. This should be fixed by tomorrow

However I think the 2 code issues you flagged here are probably still work fixing

https://github.com/openai/codex/pull/14563 would be the PR if we want to fix these edge cases

Thanks for reporting this problem. Until recently, Codex used a separate endpoint and server-side logic for compaction. We recently switched to a more robust approach where the compaction logic is moved locally using the same endpoint as normal turns. This eliminates "remote compaction" errors and increases the stability and reliability of compaction operations. If you see further problems with compaction, please use /feedback and open a new bug report.