Compaction-aware context retention for frequently re-read files

Open 💬 4 comments Opened Apr 5, 2026 by r0b0c0ck
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

Problem

When compaction triggers, all tool outputs (file contents, bash outputs, patch diffs) are permanently dropped (cf. #14589). The agent then re-reads the same large files to continue working. This creates a re-read loop:

  1. Agent reads large files → context fills up
  2. Compaction drops all file contents
  3. Agent re-reads the same files → context fills up again
  4. Goto 2

In a controlled test, a single 610 KB file was re-read 53 times in one session because it was needed after every compaction. This multiplies token consumption by 10-20x for projects with large essential context files.

Data from controlled test (gpt-5.3-codex, xhigh, read-only sandbox):

Most re-read files in a single session:

  • OpenAPI spec (610 KB): 53 reads
  • Analysis doc (93 KB): 24 reads
  • Plan doc (58 KB): 17 reads

These three files account for ~90% of all file reads, yet they contain the same content every time.

Idea

Compaction could retain file contents that are re-read across multiple compaction cycles:

  • Track which files are re-read after compaction
  • After 2+ re-reads of the same file across cycles, include its content (or a content-aware summary) in the compaction output instead of dropping it
  • Alternatively, a context_pinned_files config option that keeps specific file contents across compactions

This would break the re-read loop without changing the compaction threshold.

Workaround

I've added instructions to my project docs telling the agent to never read large files fully (use grep/sed to extract relevant sections). This reduces but doesn't eliminate the problem — the agent still re-reads medium-sized docs (50-90 KB) after each compaction.

Related issues

  • #14589 — Compaction discards all tool outputs (root cause of the re-read loop)
  • #16812 — My earlier bug report with full analysis and controlled test data (closed as not-a-bug)
  • #12208 — PreCompact hook request (would enable userland workarounds)

View original on GitHub ↗

4 Comments

github-actions[bot] contributor · 4 months ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #16812

Powered by Codex Action

r0b0c0ck · 4 months ago

Not a duplicate — #16812 was my bug report (closed as not-a-bug). This is the follow-up feature request that came out of that investigation.

pluto2060 · 4 months ago

The re-read loop is the symptom; the root cause is that file importance isn't tracked across compaction boundaries.

The pattern that eliminates it: maintain a file-priority index outside the context window (a small JSON file, not session state). Files above a relevance threshold get re-injected at the start of each fresh context cycle rather than waiting for the agent to rediscover it needs them.

The index is cheap — ours runs LLM-free and stays under 5% token budget regardless of how many files exist. Compaction becomes a non-event because critical files are always guaranteed to be present at cycle start.

We built this as the CTX layer in Entity (a Claude Code outer loop). The re-read loop dropped to zero after this approach.

r0b0c0ck · 4 months ago

Thanks — that's essentially what I'm asking for here, just built into Codex natively rather than as an external layer. Good to see the approach works in practice.