Codex remote compaction deadlocks when input_image payloads remain in compacted replacement history

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• #23257
• #24100
• #24191

Powered by Codex Action

Fix: Remote Compaction Deadlock When input_image Payloads Remain in Compacted Replacements

Issue: openai/codex #24388
Component: codex-rs/core/src/compact_remote_v2.rs
Severity: High — causes infinite compaction retry loops on conversations containing image inputs

---

1. Root Cause Analysis

1.1 Symptom

When a conversation thread contains user messages with input_image content items (base64-encoded image data URLs), remote compaction (v2) enters a deadlock cycle:

1. Compaction runs and produces a replacement history
2. The replacement history retains original user messages that include large InputImage payloads
3. The next inference request exceeds the model's context window because of the retained image data
4. Auto-compaction is triggered again
5. The same images are retained in the new replacement history
6. Steps 3–5 repeat indefinitely → deadlock

1.2 Technical Root Cause

The deadlock originates in codex-rs/core/src/compact_remote_v2.rs in the retained-message truncation logic:

// compact_remote_v2.rs — current behavior

fn message_text_token_count(item: &ResponseItem) -> usize {
    let ResponseItem::Message { content, .. } = item else {
        return 0;
    };

    content
        .iter()
        .map(|item| match item {
            ContentItem::InputText { text } | ContentItem::OutputText { text } => {
                approx_token_count(text)
            }
            ContentItem::InputImage { .. } => 0,  // ← BUG: images cost 0 tokens
        })
        .sum()
}

message_text_token_count returns 0 for ContentItem::InputImage items. This is intentional for text token budgeting (images are billed differently by the API), but it creates a critical problem in the truncation loop:

fn truncate_retained_messages_for_remote_compaction(
    items: Vec<ResponseItem>,
    max_tokens: usize,
) -> Vec<ResponseItem> {
    let mut remaining = max_tokens;
    let mut truncated_reversed = Vec::with_capacity(items.len());
    for item in items.into_iter().rev() {
        if remaining == 0 {
            continue;  // ← only triggered when remaining == 0
        }

        let token_count = message_text_token_count(&item).max(1);
        // token_count is 1 for image-only messages (due to .max(1))
        // But if a message has text + image, token_count only counts the text
        if token_count <= remaining {
            truncated_reversed.push(item);  // ← image payloads are always kept
            remaining = remaining.saturating_sub(token_count);
        } else if let Some(truncated_item) =
            truncate_message_text_to_token_budget(item, /*max_tokens*/ remaining)
        {
            truncated_reversed.push(truncated_item);
            remaining = 0;
        }
    }
    truncated_reversed.reverse();
    truncated_reversed
}

And critically, truncate_message_text_to_token_budget always preserves InputImage items:

fn truncate_message_text_to_token_budget(
    item: ResponseItem,
    max_tokens: usize,
) -> Option<ResponseItem> {
    // ...
    for mut content_item in content {
        match &mut content_item {
            ContentItem::InputText { text } | ContentItem::OutputText { text } => {
                // text is truncated when budget is exceeded
            }
            ContentItem::InputImage { .. } => truncated_content.push(content_item),
            //                                       ^^^^^^^^^^^^^^^^^^^^^^^^
            //                                       Image is ALWAYS preserved
        }
    }
    // ...
}

1.3 Deadlock Sequence

Turn N: User sends text + image (50KB base64)
  → Model responds normally
  → History: [user(text+image), assistant(reply)]

Context grows → Auto-compaction triggered
  → build_v2_compacted_history retains the user message
  → truncate_retained_messages_for_remote_compaction keeps the image
    (token_count = 0 for the image portion, .max(1) = 1 token)
  → Replacement history: [user(text+50KB image), compaction_summary]

Turn N+1: Next request sent with replacement history
  → Request size: base_instructions + user(text+50KB image) + new_input
  → Context window exceeded! (50KB base64 ≈ thousands of tokens)
  → Auto-compaction triggered AGAIN
  → Same images retained → Same overflow → Infinite loop

1.4 Why V1 Doesn't Have This Problem

The V1 remote compaction (compact_remote.rs) uses the /responses/compact endpoint directly, which returns an encrypted compaction summary. The retained messages are filtered more aggressively:

// compact_remote.rs — V1
pub(crate) fn should_keep_compacted_history_item(item: &ResponseItem) -> bool {
    match item {
        ResponseItem::Message { role, .. } if role == "user" => {
            matches!(
                crate::event_mapping::parse_turn_item(item),
                Some(TurnItem::UserMessage(_) | TurnItem::HookPrompt(_))
            )
        }
        // ...
    }
}

The V1 path relies on the server-side compact endpoint to produce a clean summary, and the client-side filtering is applied to the server's output, not the original input. The V2 path (compact_remote_v2.rs) uses a streaming approach that retains original user messages from prompt_input, which includes the raw image data.

---

2. Step-by-Step Fix Approach

2.1 Overview

The fix strips InputImage content items from retained messages during remote compaction v2, replacing them with a text placeholder. This ensures:

1. The compacted replacement history never contains large base64 image payloads
2. The model still knows an image was present (via the placeholder)
3. Context window overflow from retained images is eliminated
4. The compaction deadlock is broken

2.2 Changes Required

File: codex-rs/core/src/compact_remote_v2.rs

Three functions need modification:

1. message_text_token_count — No change needed (correctly returns 0 for images; the fix is elsewhere)
2. truncate_message_text_to_token_budget — Replace InputImage with placeholder instead of preserving
3. truncate_retained_messages_for_remote_compaction — Add a pre-pass to strip images from all retained messages

Additionally, add a new helper function to strip images from a single message.

2.3 Detailed Fix

Step 1: Add image-stripping helper

Add a new function to replace InputImage content items with a text placeholder:

/// Placeholder text inserted when image content is stripped during compaction.
const COMPACTED_IMAGE_OMITTED_PLACEHOLDER: &str = "[Image omitted during compaction]";

/// Strip all InputImage content items from a message, replacing them with
/// a text placeholder. Returns None if the resulting message would be empty.
fn strip_image_content_from_message(item: ResponseItem) -> Option<ResponseItem> {
    let ResponseItem::Message {
        id,
        role,
        content,
        phase,
    } = item
    else {
        return Some(item);
    };

    let mut has_content = false;
    let stripped_content: Vec<ContentItem> = content
        .into_iter()
        .filter_map(|content_item| match content_item {
            ContentItem::InputImage { .. } => {
                has_content = true;
                Some(ContentItem::InputText {
                    text: COMPACTED_IMAGE_OMITTED_PLACEHOLDER.to_string(),
                })
            }
            ContentItem::InputText { text } if !text.is_empty() => {
                has_content = true;
                Some(ContentItem::InputText { text })
            }
            ContentItem::OutputText { text } if !text.is_empty() => {
                has_content = true;
                Some(ContentItem::OutputText { text })
            }
            _ => None,
        })
        .collect();

    if stripped_content.is_empty() {
        return None;
    }

    Some(ResponseItem::Message {
        id,
        role,
        content: stripped_content,
        phase,
    })
}

Step 2: Modify truncate_message_text_to_token_budget

Replace the line that preserves InputImage items:

// BEFORE:
ContentItem::InputImage { .. } => truncated_content.push(content_item),

// AFTER:
ContentItem::InputImage { .. } => {
    truncated_content.push(ContentItem::InputText {
        text: COMPACTED_IMAGE_OMITTED_PLACEHOLDER.to_string(),
    });
}

Step 3: Apply image stripping in build_v2_compacted_history

Add image stripping to the retained message pipeline:

fn build_v2_compacted_history(
    prompt_input: &[ResponseItem],
    compaction_output: ResponseItem,
) -> Vec<ResponseItem> {
    let retained = prompt_input
        .iter()
        .filter(|item| is_retained_for_remote_compaction_v2(item))
        .filter(|item| should_keep_compacted_history_item(item))
        .cloned()
        // NEW: Strip image content from all retained messages
        .filter_map(strip_image_content_from_message)
        .collect::<Vec<_>>();
    let mut retained =
        truncate_retained_messages_for_remote_compaction(retained, RETAINED_MESSAGE_TOKEN_BUDGET);
    retained.push(compaction_output);
    retained
}

2.4 Alternative Approach (Pre-pass)

If modifying build_v2_compacted_history is undesirable, apply stripping as a pre-pass in truncate_retained_messages_for_remote_compaction:

fn truncate_retained_messages_for_remote_compaction(
    items: Vec<ResponseItem>,
    max_tokens: usize,
) -> Vec<ResponseItem> {
    // Strip images from all items before token budgeting
    let items: Vec<ResponseItem> = items
        .into_iter()
        .filter_map(strip_image_content_from_message)
        .collect();

    // ... rest of function unchanged
}

---

3. Code Snippets

3.1 Full Patch for compact_remote_v2.rs

--- a/codex-rs/core/src/compact_remote_v2.rs
+++ b/codex-rs/core/src/compact_remote_v2.rs
@@ -30,6 +30,9 @@ use codex_utils_output_truncation::truncate_text;
 // Mirror the current /responses/compact retained-message default while the
 // server-side path remains the reference implementation.
 const RETAINED_MESSAGE_TOKEN_BUDGET: usize = 64_000;
+/// Placeholder text inserted when image content is stripped during compaction.
+const COMPACTED_IMAGE_OMITTED_PLACEHOLDER: &str = "[Image omitted during compaction]";
+
 // Compact attempts can run much longer than normal turns, so keep the per-transport
 // retry budget smaller than the general Responses stream retry budget.
 const MAX_REMOTE_COMPACTION_V2_STREAM_RETRIES: u64 = 2;
@@ -348,6 +351,7 @@ fn build_v2_compacted_history(
         .filter(|item| is_retained_for_remote_compaction_v2(item))
         .filter(|item| should_keep_compacted_history_item(item))
         .cloned()
+        .filter_map(strip_image_content_from_message)
         .collect::<Vec<_>>();
     let mut retained =
         truncate_retained_messages_for_remote_compaction(retained, RETAINED_MESSAGE_TOKEN_BUDGET);
@@ -429,6 +433,10 @@ fn truncate_message_text_to_token_budget(
                     remaining = 0;
                 }
                 if !text.is_empty() {
                     truncated_content.push(content_item);
                 }
             }
-            ContentItem::InputImage { .. } => truncated_content.push(content_item),
+            ContentItem::InputImage { .. } => {
+                truncated_content.push(ContentItem::InputText {
+                    text: COMPACTED_IMAGE_OMITTED_PLACEHOLDER.to_string(),
+                });
+            }
         }
     }
@@ -448,6 +456,39 @@ fn truncate_message_text_to_token_budget(
     })
 }
 
+/// Strip all InputImage content items from a message, replacing them with
+/// a text placeholder. Returns None if the resulting message would be empty.
+fn strip_image_content_from_message(item: ResponseItem) -> Option<ResponseItem> {
+    let ResponseItem::Message {
+        id,
+        role,
+        content,
+        phase,
+    } = item
+    else {
+        return Some(item);
+    };
+
+    let stripped_content: Vec<ContentItem> = content
+        .into_iter()
+        .filter_map(|content_item| match content_item {
+            ContentItem::InputImage { .. } => Some(ContentItem::InputText {
+                text: COMPACTED_IMAGE_OMITTED_PLACEHOLDER.to_string(),
+            }),
+            ContentItem::InputText { text } if !text.is_empty() => {
+                Some(ContentItem::InputText { text })
+            }
+            ContentItem::OutputText { text } if !text.is_empty() => {
+                Some(ContentItem::OutputText { text })
+            }
+            _ => None,
+        })
+        .collect();
+
+    if stripped_content.is_empty() {
+        return None;
+    }
+
+    Some(ResponseItem::Message {
+        id,
+        role,
+        content: stripped_content,
+        phase,
+    })
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;

3.2 New Tests

#[test]
fn strip_image_content_from_message_replaces_images_with_placeholder() {
    let item = ResponseItem::Message {
        id: Some("msg-1".to_string()),
        role: "user".to_string(),
        content: vec![
            ContentItem::InputText {
                text: "describe this image".to_string(),
            },
            ContentItem::InputImage {
                image_url: "data:image/png;base64,abc123".to_string(),
                detail: None,
            },
        ],
        phase: None,
    };

    let result = strip_image_content_from_message(item).expect("should not be None");

    let ResponseItem::Message { content, .. } = result else {
        panic!("expected message");
    };
    assert_eq!(content.len(), 2);
    assert!(matches!(&content[0], ContentItem::InputText { text } if text == "describe this image"));
    assert!(matches!(&content[1], ContentItem::InputText { text } if text == COMPACTED_IMAGE_OMITTED_PLACEHOLDER));
}

#[test]
fn strip_image_content_from_message_returns_none_for_image_only() {
    let item = ResponseItem::Message {
        id: None,
        role: "user".to_string(),
        content: vec![ContentItem::InputImage {
            image_url: "data:image/png;base64,abc123".to_string(),
            detail: None,
        }],
        phase: None,
    };

    let result = strip_image_content_from_message(item);

    // Image-only messages become text-only with the placeholder, so they are NOT None
    assert!(result.is_some());
    let ResponseItem::Message { content, .. } = result.unwrap() else {
        panic!("expected message");
    };
    assert_eq!(content.len(), 1);
    assert!(matches!(&content[0], ContentItem::InputText { text } if text == COMPACTED_IMAGE_OMITTED_PLACEHOLDER));
}

#[test]
fn build_v2_compacted_history_strips_images_from_retained_messages() {
    let input = vec![
        ResponseItem::Message {
            id: None,
            role: "user".to_string(),
            content: vec![
                ContentItem::InputText {
                    text: "look at this".to_string(),
                },
                ContentItem::InputImage {
                    image_url: "data:image/png;base64,large_payload".to_string(),
                    detail: None,
                },
            ],
            phase: None,
        },
        ResponseItem::Message {
            id: None,
            role: "user".to_string(),
            content: vec![ContentItem::InputText {
                text: "follow up".to_string(),
            }],
            phase: None,
        },
    ];
    let output = ResponseItem::Compaction {
        encrypted_content: "new".to_string(),
    };

    let history = build_v2_compacted_history(&input, output.clone());

    // Verify no InputImage items remain in the history
    for item in &history {
        if let ResponseItem::Message { content, .. } = item {
            for ci in content {
                assert!(
                    !matches!(ci, ContentItem::InputImage { .. }),
                    "compacted history must not contain InputImage items"
                );
            }
        }
    }

    // Verify placeholder text is present
    let history_str = format!("{history:?}");
    assert!(
        history_str.contains(COMPACTED_IMAGE_OMITTED_PLACEHOLDER),
        "compacted history should contain image omission placeholder"
    );
}

#[test]
fn build_v2_compacted_history_no_deadlock_with_many_images() {
    // Simulate a conversation with many image-heavy messages that would
    // previously cause a compaction deadlock
    let mut input = Vec::new();
    for i in 0..10 {
        input.push(ResponseItem::Message {
            id: None,
            role: "user".to_string(),
            content: vec![
                ContentItem::InputText {
                    text: format!("message {i}"),
                },
                ContentItem::InputImage {
                    image_url: format!("data:image/png;base64,payload_{i}"),
                    detail: None,
                },
            ],
            phase: None,
        });
    }

    let output = ResponseItem::Compaction {
        encrypted_content: "new".to_string(),
    };

    let history = build_v2_compacted_history(&input, output.clone());

    // All messages should be retained (within budget) but with images stripped
    for item in &history {
        if let ResponseItem::Message { content, .. } = item {
            for ci in content {
                assert!(
                    !matches!(ci, ContentItem::InputImage { .. }),
                    "no InputImage should survive compaction"
                );
            }
        }
    }
}

---

4. Testing Strategy

4.1 Unit Tests

| Test | Purpose |
|------|---------|
| strip_image_content_from_message_replaces_images_with_placeholder | Verify images are replaced with placeholder text |
| strip_image_content_from_message_returns_none_for_image_only | Verify image-only messages become placeholder-only (not dropped) |
| strip_image_content_from_message_preserves_non_message_items | Verify non-Message ResponseItems pass through unchanged |
| build_v2_compacted_history_strips_images_from_retained_messages | Verify end-to-end image stripping in compacted history |
| build_v2_compacted_history_no_deadlock_with_many_images | Verify many image messages don't cause deadlock |
| retained_history_truncation_strips_images_before_budgeting | Verify images are stripped before token budget is applied |

4.2 Integration Tests

| Test | Purpose |
|------|---------|
| remote_compact_v2_with_image_input | Full compaction flow with image-containing user messages |
| remote_compact_v2_no_context_overflow_after_compaction | Verify post-compaction request fits within context window |
| remote_compact_v2_multiple_compactions_with_images | Verify multiple sequential compactions don't deadlock |
| remote_compact_v2_image_placeholder_preserves_semantics | Verify model can understand the placeholder text |

4.3 Regression Tests

The existing tests in compact_remote_v2.rs need updates:

1. retained_history_truncation_preserves_images_and_truncates_later_text_parts — This test currently expects images to be preserved. It must be updated to expect the placeholder instead.

1. retained_history_truncation_charges_image_only_messages — Update to verify image-only messages become placeholder-only messages.

1. retained_history_truncation_drops_image_only_messages_after_budget_is_spent — This test's behavior changes: image-only messages will now become placeholder-only messages, which have a non-zero token cost.

4.4 Test Execution

# Run unit tests for the affected module
cd codex-rs
just test -p codex-core -- compact_remote_v2

# Run all compaction-related tests
just test -p codex-core -- compact

# Run integration tests
just test -p codex-core -- suite::compact

---

5. Impact Assessment

5.1 Breaking Changes

None for end users. The fix is internal to the compaction pipeline. The placeholder text [Image omitted during compaction] is visible to the model but is designed to be semantically clear.

5.2 Behavioral Changes

| Aspect | Before | After |
|--------|--------|-------|
| Image payloads in compacted history | Retained as-is (base64 data URLs) | Replaced with [Image omitted during compaction] |
| Token cost of image-containing messages | Underestimated (images counted as 0 tokens) | Accurate (placeholder text has real token cost) |
| Compaction deadlock with images | Possible | Eliminated |
| Model awareness of images | Full image data available | Placeholder indicates image was present |

5.3 Performance Impact

• Positive: Reduced request sizes after compaction (no large base64 payloads)
• Positive: Fewer context window overflow errors → fewer retry cycles
• Neutral: Placeholder text adds ~5-7 tokens per image (negligible)
• Positive: Faster serialization/deserialization (smaller payloads)

5.4 Risk Assessment

| Risk | Likelihood | Mitigation |
|------|-----------|------------|
| Model confusion from placeholder | Low | Placeholder text is explicit and clear |
| Loss of image context for follow-up turns | Medium | Compaction summary should capture image semantics; the placeholder signals an image was present |
| Existing tests failing | High | Tests must be updated to expect new behavior |
| Rollout compatibility | Low | Change is backward-compatible; old rollouts will simply have images stripped on next compaction |

5.5 Migration Path

No migration needed. The fix is applied at compaction time:

1. Existing conversations with image payloads in history are unaffected
2. On the next compaction, images are stripped from the replacement history
3. Subsequent turns use the cleaned history

5.6 Monitoring Recommendations

After deployment, monitor:

1. Compaction success rate — Should increase (fewer context window errors)
2. Average request size after compaction — Should decrease for image-heavy threads
3. Model quality metrics — Watch for any degradation in responses following image-related turns (expected to be minimal)
4. Compaction retry count — Should drop to near-zero for image-containing threads

---

6. Files Modified

| File | Change |
|------|--------|
| codex-rs/core/src/compact_remote_v2.rs | Add strip_image_content_from_message, modify build_v2_compacted_history, modify truncate_message_text_to_token_budget |
| codex-rs/core/src/compact_remote_v2.rs (tests) | Update existing tests, add new tests for image stripping |

---

7. Appendix: Related Code Paths

7.1 V1 Remote Compaction (compact_remote.rs)

The V1 path uses the /responses/compact endpoint and does not retain original user messages with image payloads in the same way. The should_keep_compacted_history_item filter operates on the server's output, not the client's input. No changes needed for V1.

7.2 Local Compaction (compact.rs)

Local compaction uses collect_user_messages which extracts text via UserMessageItem::message(). This method already ignores image content (returns empty string for non-text items). No changes needed for local compaction.

7.3 Context Manager Normalization (context_manager/normalize.rs)

The strip_images_when_unsupported function already handles stripping images when the model doesn't support them. This is orthogonal to the compaction fix — it applies at request time, not compaction time. No changes needed.

7.4 Token Estimation (context_manager/history.rs)

The estimate_response_item_model_visible_bytes function already applies a heuristic for image token costs (RESIZED_IMAGE_BYTES_ESTIMATE = 7373 bytes ≈ 1844 tokens). This is used for context window management but not for compaction retention decisions. No changes needed, but confirms that images have significant token cost.

I can confirm this failure mode on Windows / Codex Desktop as well.

In my case, a long-running Codex Desktop thread failed remote compaction with the same class of error:

stream disconnected before completion: unknown variant auto, expected high or original

The relevant local evidence was in the latest type:"compacted" JSONL record. Its payload.replacement_history still contained historical input_image entries with large data:image... payloads.

A narrow local mitigation worked:

• create byte-identical backup of the rollout JSONL
• inspect only the latest compacted record
• replace only content items with type:"input_image" inside payload.replacement_history
• use a small input_text placeholder instead
• preserve all other lines, ids, timestamps, ordering, and text
• validate the full JSONL
• reopen/compact normally

After this, the thread could compact again. This was not solved by treating it as a generic context-window overflow; the image payloads in replacement history were the decisive factor.

I’m seeing what appears to be the same failure mode: a Codex thread that involved reading image inputs eventually hit a hard model context limit during continuation/compaction.

Environment:

• Surface: Codex CLI
• Platform: Linux
• Codex version: 0.136.0
• Model: gpt-5.5 on bedrock
• Thread type: local thread
• Input pattern: thread included image-reading / screenshot-style inputs

Observed error:

<img width="854" height="151" alt="Image" src="https://github.com/user-attachments/assets/8244d884-0231-4574-9b10-3da71ef5939a" />

prompt tokens (375980) exceed model maximum (278528) for base model 412fd7ff-914f-4066-b102-b3d5aeaa3e06

Thanks for reporting this problem. Until recently, Codex used a separate endpoint and server-side logic for compaction. We recently switched to a more robust approach where the compaction logic is moved locally using the same endpoint as normal turns. This eliminates "remote compaction" errors and increases the stability and reliability of compaction operations. If you see further problems with compaction, please use /feedback and open a new bug report.

Codex Desktop on Windows still stalls for about 3–4 seconds after switching to a large existing thread. The thread becomes visible first, but the input box remains unresponsive during that period.

Environment:

• Codex Desktop package: 26.707.9981.0
• codex-cli: 0.144.2
• Windows 11 x64
• About 180 local threads, approximately 278.8 MB total

Findings:

• All rollout JSONL files were valid; no duplicate thread IDs or corrupted JSON were found.
• Some image-heavy compacted histories contain repeated inline input_image/base64 data.
• A full thread/resume response contained 143 turns and was about 586 KB.
• Using:

excludeTurns: true
initialTurnsPage: { limit: 16, sortDirection: "desc", itemsView: "full" }
reduced the initial response to about 52 KB and improved responsiveness.

• Reducing the page to 8 turns reduced the payload further, but did not materially improve the remaining switch-to-input delay.
• After a desktop restart, an orphaned older app-server process caused repeated HTTP 409 "Remote app server already online" errors and model refresh timeouts. Terminating only the orphaned process stopped those errors.

Suggested improvements:

1. Use paged thread/resume loading by default.
2. Enable the input box before older turns finish hydrating.
3. Avoid duplicating inline base64 image data in compacted replacement history.
4. Use lightweight image references/content hashes and lazy-load older images.
5. Ensure old app-server processes are terminated when the desktop app restarts.

I can provide sanitized statistics, but cannot upload raw rollout files because they contain private conversation history.

A clarification for future readers: I believe “the compaction logic is moved locally using the same endpoint as normal turns” refers to remote compaction v2, introduced in #20773 and enabled by default for eligible providers in #27573.

This does not mean Codex returned to the older prompt-based local compaction path. In v2, Codex assembles the compaction request and rebuilds replacement history client-side, but sends a compaction_trigger through the normal /responses transport. The backend returns an opaque compaction item containing encrypted_content, which Codex carries into later turns. The dedicated /responses/compact request and server-rebuilt history are the parts that were replaced; the source still calls the new path remote_compaction_v2.

So “local” here describes orchestration and history reconstruction, not a plaintext local summarization turn. It also does not by itself establish that the original input_image retention issue was fixed: #23728 explicitly preserved non-text retained content such as images when rebuilding v2 history, and the current implementation still retains and counts those images.