Removing the need for compactions, and better token usage

What variant of Codex are you using?

App

What feature would you like to see?

This suggestion would require updates to the GPT models also, but if you train the models to be able to do this:

Block-Attention paper: https://arxiv.org/abs/2409.15355

Then:

1. You can dynamically edit context - and remove redundant info, or even bring it back later if it makes sense to - and the same for summaries.
2. Agent instructions become composable, and also easy to switch in and out based on the requirements of the current stage of an agentic workflow (like bring in advanced Python best practices for a specific situation only while that specific thing is being worked on)

I think this also has potential for the API side offering in general, since it would let people manage more complex agentic workflows at lower costs – in exchange for mildly higher complexity in orchestration.

Additional information

The main way I'd be inclined to use this for Codex is alongside a memory management pattern like Claude Code's pattern of having an index of summary information with additional docs for drill down.

That would make it easier to compose context based on current needs.

I think the natural route would be to have a project level broader memory, and also a current thread level memory for recent work.
Plus, that way you could also decide what portions of current context to pass to sub-agents, without having to either share the whole thread context or explicitly define a complete new context before calling the sub-agent.

... Much potential seriously.