Compaction-aware context retention for frequently re-read files
Problem
When compaction triggers, all tool outputs (file contents, bash outputs, patch diffs) are permanently dropped (cf. #14589). The agent then re-reads the same large files to continue working. This creates a re-read loop:
- Agent reads large files → context fills up
- Compaction drops all file contents
- Agent re-reads the same files → context fills up again
- Goto 2
In a controlled test, a single 610 KB file was re-read 53 times in one session because it was needed after every compaction. This multiplies token consumption by 10-20x for projects with large essential context files.
Data from controlled test (gpt-5.3-codex, xhigh, read-only sandbox):
Most re-read files in a single session:
- OpenAPI spec (610 KB): 53 reads
- Analysis doc (93 KB): 24 reads
- Plan doc (58 KB): 17 reads
These three files account for ~90% of all file reads, yet they contain the same content every time.
Idea
Compaction could retain file contents that are re-read across multiple compaction cycles:
- Track which files are re-read after compaction
- After 2+ re-reads of the same file across cycles, include its content (or a content-aware summary) in the compaction output instead of dropping it
- Alternatively, a
context_pinned_filesconfig option that keeps specific file contents across compactions
This would break the re-read loop without changing the compaction threshold.
Workaround
I've added instructions to my project docs telling the agent to never read large files fully (use grep/sed to extract relevant sections). This reduces but doesn't eliminate the problem — the agent still re-reads medium-sized docs (50-90 KB) after each compaction.
Related issues
- #14589 — Compaction discards all tool outputs (root cause of the re-read loop)
- #16812 — My earlier bug report with full analysis and controlled test data (closed as not-a-bug)
- #12208 — PreCompact hook request (would enable userland workarounds)
4 Comments
Potential duplicates detected. Please review them and close your issue if it is a duplicate.
Powered by Codex Action
Not a duplicate — #16812 was my bug report (closed as not-a-bug). This is the follow-up feature request that came out of that investigation.
The re-read loop is the symptom; the root cause is that file importance isn't tracked across compaction boundaries.
The pattern that eliminates it: maintain a file-priority index outside the context window (a small JSON file, not session state). Files above a relevance threshold get re-injected at the start of each fresh context cycle rather than waiting for the agent to rediscover it needs them.
The index is cheap — ours runs LLM-free and stays under 5% token budget regardless of how many files exist. Compaction becomes a non-event because critical files are always guaranteed to be present at cycle start.
We built this as the CTX layer in Entity (a Claude Code outer loop). The re-read loop dropped to zero after this approach.
Thanks — that's essentially what I'm asking for here, just built into Codex natively rather than as an external layer. Good to see the approach works in practice.