GPT‑5.6 regression: stronger planning, weaker execution and instruction adherence

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• #35131
• #35130
• #35225
• #36086

Powered by Codex Action

I believe this is largely caused by the harness and the way context compression works. A model like Sol generates a lot of context, and with the relatively small context window available to Codex, that window fills up quickly.

After compression, especially when the model’s reasoning is also carried forward, the original weight of what you initially asked it to do becomes relatively small. The model most likely sees more of its own thoughts about what could be improved than the original task itself, so it gradually drifts away from the actual goal.

#34963

I have also noticed that after context compression, the model can fall into a loop and forget what it was doing. They probably assumed that carrying over the latest reasoning would solve this problem, but after compression it may cause the model’s own thoughts to carry more weight than the user’s original prompt.

[https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/](https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/
)

Although it is impossible to be certain, since they do not disclose the details of what exactly the compressed context contains. So, to some extent, this is still educated guesswork.

I can independently confirm this regression on macOS, using both GPT-5.6 Sol (including Ultra) and Terra in Codex.

Over the last couple of weeks, comparable real-world coding and operational tasks have become significantly slower and less reliable. The recurring pattern is excessive planning, auditing and re-auditing, drift from explicit instructions, and an incomplete or incorrect final result.

The most severe case was with /goal: one task ran for approximately 12 hours and consumed an entire week's allowance, yet the result was still wrong. I had to redo the work with GLM 5.2. On comparable tasks, Claude Opus 5 and GLM 5.2 have reached a correct, tested result in a fraction of the time.

The core regression is not merely token consumption: GPT-5.6 Sol and Terra need to be more concise, follow the latest explicit instruction, stop deliberating when evidence is sufficient, and execute and validate the requested work instead of repeatedly expanding scope.

Independent reproduction: Sol Ultra turned a bounded live-test-prep task into a 2h19m certification loop despite repeated scope corrections

I can independently reproduce the behavior described here with GPT-5.6 Sol Ultra in Codex, and I have a complete transcript of the session.

This was not a vague/autonomous request. I was working on a Project Zomboid mod. After an earlier code review had identified specific remaining blockers, my next instruction was simply:

“Ok get everything ready for live testing”

The reasonable remaining scope was: finish the known gameplay blockers, run the existing validation, back up/install the exact candidate, clear old logs, and produce the test package. MP would remain explicitly unconfirmed until live MP testing.

Instead, Sol silently changed the definition of done from “prepare a live-test candidate” into something much closer to “fully certify the runtime, diagnostics system, analyzer, packaging, documentation, lifecycle evidence, and release gates.”

The run lasted 2h 19m 7s.

What the scope expansion looked like

Sol created parallel Core live candidate, Diagnostics live candidate, and Package live candidate workstreams. Once those were nominally complete, it added an independent integration audit. Each new audit finding then became another prerequisite before I was allowed to test the build.

The sequence expanded into things such as:

• terminal sessionEnd evidence requirements;
• dedicated-server save checkpoints;
• strict candidate/build identity validation;
• analyzer behavior for rotated/truncated logs;
• vehicle-marker lifecycle correlation;
• user-facing vs internal logging-level validation;
• PowerShell 5.1 and PowerShell 7 analyzer runs;
• target-aware thumpable pairing edge cases;
• additional packet-order regressions;
• further analyzer rules to prove those new mechanisms.

Some real gameplay bugs were found, and I explicitly told Codex that genuine bugs were valid reasons to continue. My complaint is not that it found bugs. The problem is that it repeatedly promoted diagnostic/evidence-hardening and newly invented certification requirements into blockers for the original task of getting a build ready for live testing.

Explicit course correction did not reset the execution plan

After it had already been running far longer than expected, I told it:

“don't over-engineer the testing please. If it's needed or adding value, then it's OK. I didn't think we had enough to do to take you 2 hours of continuous work.”

Sol correctly acknowledged the problem and said:

“the scope is frozen now” “No more speculative analyzer features or documentation expansion.”

I then clarified that this was a guardrail, not an order to ignore real bugs:

“Again, if you are finding real bugs, then that's ok to proceed. I'm just stating a reminder, not ordering you to change direction.”

Sol again acknowledged the distinction.

But the execution trajectory did not materially reset. It continued adding analyzer/evidence requirements, regression cases, review gates, and integration work. This is the exact pattern described in #35131: the language layer accepts the correction, but the execution plan keeps following the old trajectory.

Eventually I asked:

“What are you still working on? There is no way there was this much stuff broken or to do.”

At that point Sol said there was no code blocker remaining, that core was 242/242 and the analyzer 62/62, and that it was stopping further review. I then had to ask what those numbers even meant; they were individual test assertions, not 242/62 bugs or meaningful independent work items.

I ultimately issued an explicit hard stop:

“Also stop. Like literally right now. No more commands.”

The original deliverable still was not cleanly finished: the deployment attempt had copied candidate files, then failed on a nonexistent preview.png; post-copy hash verification had not run and no test ZIP had been created.

Codex's own postmortem identified the failure very accurately

When I later demanded an explanation rather than an apology, Sol said:

“I silently changed the definition of done from ‘prepare a test build’ to ‘fully certify the runtime, logging system, analyzer, packaging, and documentation.’”

It then described the loop it had created:

1. change runtime log schema;
2. update analyzer;
3. add fixtures;
4. update validators;
5. update documentation;
6. run another integration audit;
7. find another mismatch and repeat.

It also explicitly acknowledged that the two-plus hours were not caused by the tests themselves, but by iterative scope expansion and repeated agent/review cycles, and concluded:

“I never imposed that stopping rule. That was the central failure.”

That retrospective diagnosis is essentially correct. The concerning part is that Sol was capable of articulating the right stopping policy afterward, while failing to enforce it during execution even after I repeatedly supplied that policy in-context.

Why I think this is a model/agent-control bug rather than just prompting

The failure pattern was:

vaguely bounded task → self-created quality criterion → parallel audits → new finding becomes blocker → implementation creates new review surface → review finds another issue → definition of done moves again

The individual steps often sounded locally reasonable. Globally, they were no longer proportional to the user's requested task.

The particularly important behavior is plan inertia after correction: my corrections changed Sol's verbal framing (“scope frozen”, “guardrail understood”) much more than they changed the active execution graph.

The session did undergo automatic context compaction during the run, so context handling may be relevant, but I cannot establish that as the cause from this single case.

Behavior I would like to see tested/fixed

• A user scope correction should override or invalidate incompatible active/subagent plans, not simply append another instruction to the existing plan.
• Maintain a stable, user-grounded definition of done. Newly discovered nice-to-have validation should not silently become a prerequisite unless explicitly instructed.
• Distinguish proven product/runtime blockers from diagnostics hardening, documentation cleanup, speculative edge cases, and release-certification work.
• Before materially broadening a bounded task, ask the user rather than redefining “ready.”
• Treat explicit stop instructions as cancellation of the active/queued execution tree, not merely a conversational acknowledgement.
• Add evals where the model is deliberately given an attractive rabbit hole or audit finding after a bounded task, then receives a scope correction. Success should be measured by whether actual subsequent tool actions narrow accordingly.

I have the full timestamped transcript and screenshots, including the Worked for 2h 19m 7s result and the repeated scope-correction exchanges, and can provide additional sanitized excerpts if maintainers want them.

A newly released product always has issues. Testers are used as guinea pigs.

5.6 sol is absolutely infuriating to work with in recent weeks. I'm angry all the time.