Agent sometimes incorporates private chat context into standalone outputs

Resolved 💬 1 comment Opened May 18, 2026 by diegomrsantos Closed May 18, 2026

Summary

Codex sometimes treats private conversation context as content that belongs in the standalone artifact it is drafting.

When the user corrects the agent, the correction should guide the next draft. It should not be copied into the artifact unless that correction is part of the intended audience-facing message.

Concrete Example

I was using Codex to draft an inline comment for an external document/review.

Codex first suggested a comment that said the document should make its EIP-7732 scope clearer. I replied that this was wrong, because the document already says it is about EIP-7732.

Instead of simply removing that concern and drafting a standalone comment about the remaining issue, Codex produced a new draft that included:

The SIP is already scoped to EIP-7732, but this snapshot is older than the current Gloas spec...

That sentence only makes sense inside my private conversation with Codex. The external reader does not know that Codex previously misunderstood the scope, or that I corrected it. In the standalone comment, the phrase "already scoped" reads like a response to an invisible disagreement.

A better output would have been something like:

Could we update this pinned consensus-specs snapshot to the current Gloas revision? Some later details in the SIP should be checked against the latest EIP-7732 flow, including SignedProposerPreferences fields such as dependent_root and target_gas_limit, proposer-dependent-root bid matching, and payload processing via execution_requests_root, parent_execution_requests, and verify_execution_payload_envelope.

The important difference is that the corrected version uses the private correction to improve the draft, but does not mention the private correction itself.

Expected Behavior

When producing a standalone artifact, Codex should distinguish between:

  • context used to understand the task
  • text that belongs in the artifact

User corrections such as "that is not the issue," "do not mention X," or "the document already says Y" should normally update Codex's understanding, not become audience-facing prose.

Actual Behavior

Codex sometimes incorporates the conversational correction into the produced artifact, creating text that references invisible context, prior disagreement, or an internal drafting process.

Impact

This makes Codex unreliable for standalone writing tasks. The user has to manually check whether the result contains chat-local references that will confuse an external reader.

This is not specific to PR comments. It can affect comments, emails, documentation, summaries, release notes, issue reports, or any output intended to stand on its own.

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗