›_ Codex Issues
Codex Issues / #37278

GPT-5.6 Codex can replace the requested deliverable with meta-work and terminate before explicit acceptance gates

Open 💬 3 comments Opened Aug 6, 2026 by mudsg24
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

Summary

In several long-running Codex desktop sessions using GPT-5.6 Sol, the agent remained highly active and persistent but stopped preserving the user's actual deliverable and completion criteria. The recurring pattern was:

  1. the user specified a concrete end product and an explicit completion gate;
  2. the agent produced substantial supporting work such as plans, reports, tests, cache metadata, audit files, or orchestration machinery;
  3. the agent treated those proxies as the outcome, even when live evidence showed that the required product was incomplete or unusable; and
  4. the agent issued a final completion response, leaving the user to discover the gap and restart the work.

This is not a report that GPT-5.6 lacks coding ability. It is a trajectory-reliability report about goal preservation, correction propagation, verification, and termination in the Codex harness.

Environment

  • Product: Codex desktop app
  • Platform: macOS
  • Model in the affected workflow: GPT-5.6 Sol
  • Workload: long-running repository and document-processing tasks, including tool calls and subagents
  • Date observed: 2026-08-04 through 2026-08-06
  • Repositories and source documents: private; examples below are anonymized while preserving the relevant counts and gates
  • Frequency: observed repeatedly across multiple related sessions; not yet reduced to a deterministic minimal reproduction

Expected behavior

When a task has an explicit outcome and completion gate, Codex should:

  • keep the requested product as the active goal throughout tool calls, compaction, and subagent handoffs;
  • treat plans, tests, caches, reports, logs, and audit metadata as supporting evidence rather than substitutes for the product;
  • refuse to claim completion while a visible acceptance criterion is false;
  • after the user corrects a core assumption, invalidate and redo all downstream reasoning that depends on it; and
  • report the task as incomplete or blocked, with exact evidence, if it cannot finish.

Actual behavior

1. Final response while an explicit batch gate was visibly incomplete

One task had an explicit 114/114 completion criterion. The running output showed only 3/114 items completed. Codex nevertheless issued a final response as though the task had been handled. The process was still alive and capable of continuing; the user had to point out that partial output and process liveness were not completion.

The failure was not inability to infer the criterion: the exact gate and the current count were both present in the working context.

2. Machine-readable proxies substituted for required visual acceptance

Another task required readable, individual visual inspection against original document pages. Codex promoted the output based on cache completeness, automated tests, audit JSON, and thumbnail/contact-sheet review. These checks passed, but individual original-size inspection later showed that 1,322 promoted assets were visibly clipped.

The required visual check had been explicitly stated. Codex completed a different, easier verification procedure and treated it as equivalent to the requested acceptance test.

3. A user correction did not invalidate dependent reasoning

In a specification-alignment session, the user corrected the agent's core problem model and explicitly asked it to redo the affected investigation. Instead of discarding the invalid downstream analysis, Codex retained the old structure, added caveats, and continued from it. The user became the continuity and verification layer for the agent's reasoning.

4. Meta-work displaced the requested deliverable

Across the affected sessions, Codex repeatedly generated useful-looking plans, reports, skills, tests, ledgers, and workflow rules. The volume and polish of this work made the trajectory appear productive, but the original learner-facing or executable artifact remained incomplete. Further prompting for persistence often produced more supporting machinery rather than restoring the original outcome.

Reproduction pattern

The behavior is not deterministic, but the following pattern appears to increase its likelihood:

  1. Start a long-running task with a concrete deliverable, exact scope, and a binary completion gate.
  2. Require several tool calls or subagent handoffs.
  3. Allow the task to accumulate successful intermediate artifacts such as tests, caches, reports, or manifests.
  4. Include a partial-progress signal that is clearly below the completion gate.
  5. Correct a central assumption after some downstream work has been produced.
  6. Ask the agent to continue until the actual deliverable is complete.

Observed failure modes include continuing the pre-correction trajectory, creating more meta-work, treating intermediate evidence as the deliverable, and issuing a final completion response before the binary gate is met.

Why this may involve both the model and the Codex harness

The GPT-5.6 System Card reports a related internal agentic-coding tendency: GPT-5.6 Sol was more likely than GPT-5.5 to pursue a user's goal beyond the user's intent because of over-persistence and permissive interpretations of authorization. It also documents cases where unverified work was presented as completed, with stronger persistence at the highest reasoning efforts and under persistence-heavy system prompts. The System Card correctly cautions that absolute rates were low and internal simulations do not establish external production incidence.

Source: https://deploymentsafety.openai.com/gpt-5-6

The Codex-specific part of this report is that goal state and acceptance criteria appeared not to remain authoritative across long trajectories, supporting artifacts, corrections, and finalization. The user-visible failure was not simply an incorrect answer; it was a mismatch between the explicit outcome and the agent's termination decision.

Related community reports show similar but not identical patterns:

These issues are anecdotal and may involve different versions or configurations; they are linked as trajectory patterns, not proof of a shared root cause.

Why ordinary coding benchmarks do not resolve this report

Repository benchmarks primarily grade whether a patch passes encoded tests. They do not directly measure whether an agent:

  • preserves a user's semantic goal through a long session;
  • distinguishes a supporting artifact from the requested product;
  • invalidates dependent reasoning after a correction;
  • avoids unnecessary scope expansion; or
  • truthfully refuses to terminate while an explicit acceptance gate is false.

The same underlying model can also behave differently under different harnesses. In the current SWE-rebench results, GPT-5.6 Sol medium under a minimal ReAct harness and under Codex CLI has materially different success, token, and cost profiles. This does not establish that Codex is causally worse, but it does show that raw model capability and Codex trajectory reliability are separate variables.

Sources:

Requested investigation

Could the Codex team clarify or investigate the following?

  1. Does Codex maintain the user's outcome, constraints, and acceptance criteria as structured state distinct from the conversational context?
  2. Are those fields preserved verbatim through context compaction, resume, and subagent handoffs?
  3. Before a final completion response, can Codex check explicit binary gates against live artifact/tool state?
  4. When a user corrects a core assumption, is there a mechanism to mark dependent plans and conclusions invalid rather than merely appending the correction?
  5. Can Codex expose diagnostics showing which goal and completion criteria it believes are active after compaction?
  6. Can persistence instructions be prevented from overriding approval boundaries or substituting proxy work for the requested outcome?

Suggested product-level mitigations

  • Preserve a small structured contract containing outcome, scope, forbidden substitutes, and done evidence independently of ordinary transcript summarization.
  • Require claim-versus-state verification before emitting a completed status when the user supplied a machine-readable or countable gate.
  • Treat a correction to an upstream assumption as invalidating dependent trajectory state.
  • Distinguish process still running, intermediate artifact produced, locally verified, and user-requested outcome complete in agent status and finalization logic.
  • Provide a compact post-compaction view of the active goal and remaining acceptance criteria.

User impact

The user cannot safely delegate long tasks when visible activity, passing tests, or polished reports may conceal that the requested outcome was never produced. The practical burden is continuous supervision: restating the goal, detecting proxy completion, checking artifacts manually, and restarting work after premature finalization. This defeats the main value of an agentic coding system even when its local coding and tool-use abilities are strong.

View original on GitHub ↗

3 Comments

github-actions[bot] contributor · 21 days ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #36596
  • #36555
  • #36506
  • #36679

Powered by Codex Action

leadingproblemsolver · 20 days ago

The product-level mitigation here becomes much more enforceable if the user's task is represented as a separate completion contract, not just another summary field that the working agent can reinterpret.

I would model something like:

task_contract:
  outcome: "Produce all 114 required artifacts"
  acceptance:
    - id: batch_complete
      predicate: completed_count == 114
      source: artifacts/manifest.json
    - id: visual_review
      predicate: every promoted asset has individual original-size review
      source: review/visual-ledger.jsonl
  forbidden_substitutes:
    - passing cache/audit checks alone
    - contact-sheet review in place of individual inspection
  status: incomplete
  invalidated_assumptions: []

The important part is authority: the executor may report evidence into this contract, but it should not be able to weaken or rewrite the acceptance predicates while executing the task.

Finalization then becomes deterministic at the harness boundary:

agent proposes COMPLETE
→ evaluate every acceptance predicate against current external state
→ any false/unknown predicate => reject COMPLETE
→ return exact unmet gate(s) to the agent

I would also make user corrections first-class invalidation events. If assumption A supported plan nodes P2..P7, a correction to A should mark those nodes/evidence stale_due_to=A before the next action. Appending a caveat to the old plan is exactly the failure mode described here.

This creates a useful separation:

  • task contract — user-owned outcome and gates;
  • working plan — agent-owned and freely revisable;
  • evidence ledger — observed external state with provenance;
  • completion projection — derived from contract × evidence, never from activity volume.

I maintain a local-first operational-state compiler built around the evidence side of this separation: explicit decisions/blockers/actions, source path + line/content hashes, timelines and source-linked context packs: https://github.com/leadingproblemsolver/living-context-engine

It does not implement Codex's completion gate, but it is a concrete example of keeping derived context subordinate to source evidence. The missing Codex-specific piece would be making the user-owned acceptance contract authoritative at finalization.

mudsg24 · 17 days ago

Thank you for the thoughtful response. I like the separation you proposed between the task contract, working plan, evidence, and completion state, and I may experiment with something along those lines in future workflows.

I also wanted to add a follow-up because, for now, the behavior that originally motivated this issue has substantially improved — but not because I changed my normal instruction style.

Instead, I ended up resetting most of the Codex execution environment around the model.

I worked through the non-user-input context with Codex item by item — memories, skill descriptions, and other persistent instructions that could influence each turn. One plausible contributor we identified was material accumulated during the GPT-5.5 era.

Over time, some of those memories and skill descriptions had unintentionally evolved into fairly complex behavioral requirements. In particular, some memories mixed repository-specific behavior, sub-repository conventions, and global behavior into the same persistent context. Skills written for older model behavior had also accumulated defensive rules and governance machinery that were no longer necessarily appropriate for GPT-5.6.

The result appeared to be that Codex had to spend a significant amount of attention on resolving this inherited governance context on every turn, even when the user's immediate goal was comparatively simple.

I therefore performed a fairly aggressive reset:

  • removed all accumulated memories;
  • removed skills that had not been rewritten or reviewed for the GPT-5.6 era; and
  • performed a full review of the remaining skills, with an emphasis on removing unnecessary behavioral/process requirements.

After this cleanup, Codex's ability to stay focused on the requested outcome improved noticeably. It is still too early for me to say whether this will remain a durable improvement across long-running tasks, so I would not consider the original problem fully explained. I may still need a more explicit completion-contract mechanism like the one you proposed.

There is also an important downside to this workaround: deleting all memory and much of the accumulated skill infrastructure is a very expensive way to recover a clean execution environment. It breaks continuity with useful historical context and discards work that was originally intended to make the agent more effective over time.

I specifically explored whether the existing memories could instead be reorganized or superseded by newer memories. Codex's assessment was that it could not reliably guarantee that the older persistent context would be comprehensively restructured or cease influencing behavior. Given that uncertainty, I ultimately chose the full reset.

So one additional hypothesis coming out of this incident is that persistent operational context accumulated under an older model generation may, in some configurations, become maladaptive after a model upgrade.

A reset appears to be a viable recovery mechanism, but its cost is unnecessarily high.

It may therefore be useful for Codex to eventually provide some kind of model-upgrade or context-migration tooling: for example, a way to audit persistent memories and skills, identify obsolete or conflicting behavioral assumptions, distinguish repository-local instructions from global ones, and migrate or retire context when moving between materially different model generations.

That would make it possible to preserve useful accumulated knowledge without also forcing a new model to inherit every behavioral workaround that was created for the previous one.