Important Early Context Gets Deleted During Context Window Compaction
What version of Codex is running?
v 0.87.0
What subscription do you have?
None
Which model were you using?
gpt-5-codex
What platform is your computer?
Microsoft Windows NT 10.0.26100.0 x64
What terminal emulator and version are you using (if applicable)?
Powershell
What issue are you seeing?
During the context window compaction process when the context window is exceeded and the system needs to remove history items to fit within model limits, the current implementation uses a simple remove_first_item method that always removes the oldest item regardless of its importance. This can delete crucial user messages containing critical instructions, important assistant responses that establish the context for the conversation, or key decision points that are necessary for understanding the conversation flow. The loss of important early context makes it difficult for the model to maintain continuity and understand the full context of the conversation, leading to degraded performance and potential misunderstandings.
What steps can reproduce the bug?
To reproduce this issue, engage in a long conversation that exceeds the model's context window, ensuring that important user messages or critical instructions appear early in the conversation history. For example, start a conversation with a user message containing important requirements or constraints, then continue with many assistant responses and tool calls. When the context window is exceeded and compaction is triggered, observe that the system removes the oldest items first, which may include the critical user message with important instructions. Continue the conversation and notice that the model's responses may no longer respect the early instructions or may lack context that was established in the deleted messages. The important early context that should guide the entire conversation is lost, making subsequent interactions less effective.
What is the expected behavior?
The system should implement an importance-aware deletion strategy that identifies important items before deletion and prioritizes removing non-important items first. Important items should be defined as user messages (which represent user intent and instructions), messages containing critical keywords like "important", "critical", "must" (in both English and Chinese), and function call outputs that contain error information. When compaction is needed, the system should first identify all important items, then find the oldest non-important item that is not part of a critical call/output pair, remove that item, and only fall back to removing the first item if all items are marked as important. This would ensure that important conversation context is maintained during compaction and that the system makes intelligent decisions about what to preserve, allowing the model to maintain continuity and respect early instructions throughout the conversation.
Additional information
_No response_
8 Comments
There are many tradeoffs involved in compaction. We've explored numerous techniques and variations on those techniques. If you have specific ideas for improvements, let's discuss here.
@etraut-openai
Thanks for taking a look at this issue. I wanted to share my thinking on how to approach this problem.
The main idea I had was to make the deletion process smarter by looking at what each item actually means, rather than just removing things based on when they happened. I realized that some parts of the conversation are way more important than others - like when a user gives specific instructions, those should stick around even if they happened early on, while some assistant responses or tool outputs might be less critical.
So here's how I'm thinking about it: first, I mark all user messages as important automatically, since those represent what the user actually wants and should guide everything. Then I also look for messages that have keywords like "important", "critical", or "must" (I made it work for both English and Chinese), so users can explicitly mark stuff they care about. And finally, I preserve any function call outputs that contain errors, because even if an error happened earlier, you still need to know about it to debug things.
The way the deletion works is pretty straightforward - I do one pass through the history to figure out what's important, then I look for the oldest thing that's NOT important and remove that instead. I also make sure to keep call/output pairs together, so if something is part of a critical tool call sequence, I don't break that up. Only if everything is marked as important do I fall back to the old behavior of just removing the first item, so compaction can still make progress.
I also thought about edge cases - like when you remove a function call, you need to remove its output too to keep things consistent. I'm using the existing normalize mechanism for that, which already handles pairing things up.
Overall, I think this approach should make conversations work better, especially long ones where you give instructions early on and want the model to remember them. It's a balance between keeping the important stuff and still being able to shrink the history when you need to.
I don't think we'd want to hard-code specific natural language terms like "important", "critical", etc. Codex users speak many languages other than English and Chinese, so we shouldn't favor just those two. If we were to adopt a heuristic like this, we'd probably want to leverage the model to classify messages as important or not. This avoids the messiness of a hard-coded prompt.
The heuristics that you're describing sound pretty fragile. I can see how they'd work better in some cases, but I think they'd fail in many others. Your proposed solution also relies on client-side logic, whereas we recently moved much of our compaction logic to the cloud. This allows us to take better advantage of new model intelligence to help with the complex task of compaction.
Like I said above, we've already tried a lot of different techniques related to compaction. It's a complex topic, and there are no easy answers.
ok, got it ,thx
Thanks for the feedback! You're absolutely right - hard-coding keywords is definitely not the right approach, especially given the diverse languages our users speak. And I appreciate you pointing out that compaction logic has moved to the cloud, which I should have considered.
Let me think about this differently. What if we use a more structural approach that doesn't rely on language-specific keywords? For example, we could prioritize based on message types and patterns rather than content:
For the cloud-side implementation, I'm curious if we could leverage the model's intelligence during compaction to evaluate message importance. Maybe the model could assess which messages are critical for maintaining conversation continuity, or we could use embeddings to measure relevance to the current context. This would be much more robust than keyword matching and would naturally handle multilingual scenarios.
I'm also wondering if there's a hybrid approach - keep simple structural heuristics on the client side (like preserving user messages), but let the cloud-side compaction use model intelligence for more nuanced decisions. That way we get the benefits of both approaches.
What do you think? I'd love to hear your thoughts on whether these directions make sense, or if there are other patterns you've found work well for compaction.
We prefer not to use heuristics. They tend to be fragile and fail in ways that are unpredictable and difficult to debug. We'd prefer to leverage model intelligence for problems like this. This allows us to "ride the wave" as model reasoning continues to improve.
Thanks for the explanation! That makes a lot of sense - I can see how heuristics can be brittle and hard to maintain, especially as edge cases pile up. Using model intelligence is definitely a more robust approach, and I love the idea of "riding the wave" as models improve. That's a much better long-term strategy than trying to maintain a bunch of rules.
I really appreciate you taking the time to share your thinking on this. It's been really helpful to understand your approach and the reasoning behind it. I'm definitely learning a lot from this discussion.
I'm curious about how you're leveraging model intelligence for compaction on the cloud side. Are you using the model to evaluate message importance directly, or is there a different approach you've found works well? I'd love to learn more about how this is implemented - it sounds like a really interesting problem to solve.
If there's any way I could help contribute to this, or if you have any pointers on where to look in the codebase to understand the current implementation better, I'd be happy to dive in!
@etraut-openai reading AGENTS.md and current session history (just user requests and codex's answers) from JSON would be enough to drastically cut down user's frustration after compaction at 75%. The issue is not the compaction itself, rather than the spontaneous _forgettiness_ following the context compaction.
Just a guess, but I really doubt that filling the context window with previous user's prompts would be a huge trade-off for that.
Thread ID
019bd843-ad52-7de0-81c0-56b93382b26fLots of irritation from my part there, once it happens, please ignore and don't take it personally, it's just my human emotions there 😉