Compaction silently discards all tool outputs and assistant reasoning

Compaction silently discards all tool outputs and assistant reasoning

After reading through compact.rs, I ran a reproducible experiment to quantify the information loss. The results are significant enough to warrant a structural discussion.

How compaction works (code evidence)

collect_user_messages() is the only filter applied to history before rebuilding the compacted context:

pub(crate) fn collect_user_messages(items: &[ResponseItem]) -> Vec<String> {
    items.iter().filter_map(|item| match parse_turn_item(item) {
        Some(TurnItem::UserMessage(user)) => Some(user.message()),
        _ => None,  // ALL other item types dropped here
    }).collect()
}

build_compacted_history_with_limit() then reconstructs history from:

• ≤20,000 tokens of user messages (recency-weighted, truncated)
• One LLM-generated summary (last assistant message from compaction turn)
• Ghost snapshots (preserved separately)

Every other item type is permanently dropped: assistant messages, tool calls, tool results, file contents, bash outputs, patch diffs, test results.

Experiment: information retention across compactions

I simulated a realistic 6-turn coding session (auth bug fix: read files → run tests → apply patch → regression test) and ran it through the same logic as compact.rs. The script is fully reproducible — it mirrors collect_user_messages() and build_compacted_history_with_limit() exactly.

Session composition before any compaction (25 items, 1,670 tokens):

| Item type | Count | Tokens | Share |
|---|---|---|---|
| tool_result (file reads, bash output, patches) | 7 | 1,324 | 79.3% |
| assistant_message (reasoning, analysis, plans) | 6 | 221 | 13.2% |
| user_message | 5 | 71 | 4.3% |
| tool_call | 7 | 54 | 3.2% |

After 1st compaction (6 items, 229 tokens — 13.7% retention):

| Item type | Count | Tokens | Share |
|---|---|---|---|
| user_message | 5 | 71 | 31.0% |
| LLM summary | 1 | 158 | 69.0% |
| tool_result | 0 | 0 | 0% |
| tool_call | 0 | 0 | 0% |

After 2nd compaction (115 tokens — 6.9% of original):

All 5 critical facts checked for presence in surviving context:

| Fact | After 1st | After 2nd |
|---|---|---|
| cache::invalidate fix (the actual bug fix) | ✅ | ❌ |
| test_concurrent_refresh failure (confirmed root cause) | ✅ | ❌ |
| user_42 regression test content | ❌ | ❌ |
| mobile.rs also calls the buggy path | ❌ | ❌ |
| Line 47 in auth.rs (location of the fix) | ❌ | ❌ |

Key findings

1. Tool results are 79% of session tokens but 0% survive compaction. These are the highest-density items in a coding session — file contents, test output, applied diffs.

1. "Telephone game" degradation: 1st compaction retains 13.7%. 2nd compaction of that retains 50% → 6.9% of the original. Each compaction summarizes the previous summary.

1. The system acknowledges this is a problem — the post-compaction warning reads: "Long threads and multiple compactions can cause the model to be less accurate." But the warning fires after the information is already gone.

1. No cross-session persistence: when the session ends, everything is lost regardless of compaction. The next session starts blank.

Root cause: compaction is purely in-context compression

The design treats memory as a context-window management problem: compress when full, discard the rest. This works for short, self-contained tasks. It breaks for:

• Long coding sessions (hours, multiple compactions)
• Tasks where exact intermediate state matters (what file was at line 47, which tests passed)
• Multi-session work on the same codebase

Proposed direction: Memory MCP Server

Rather than compressing everything into a lossy in-context summary, give the model explicit tools to write important facts to durable external storage and read them back selectively:

memory_save(key, value, tags)   — persist a fact, decision, or file snapshot
memory_recall(query)            — semantic search over saved memories
memory_checkpoint(label)        — snapshot current task state  
memory_list(tag_filter)         — enumerate saved keys

This decouples memory from the context window:

• The model actively chooses what to persist (not passive compression)
• Memories survive session end
• Only relevant context is retrieved on demand (not a full dump)
• Users can inspect and edit what the model remembers

A local-first implementation (SQLite + embeddings) could ship with zero cloud dependencies and integrate naturally with the MCP infrastructure Codex already supports. Compaction would still exist for context management, but critical facts would be durably saved before the window fills.

Happy to prototype this if the direction is of interest. Opening here to gauge appetite before writing code.