Repeated regressions, false “done” signals, and time-draining syntax loops
Resolved 💬 2 comments Opened Aug 31, 2025 by carli2 Closed Nov 10, 2025
What version of Codex is running?
codex-cli 0.27.0
Which model were you using?
gpt5-high
What platform is your computer?
arm64 on raspberry pi
What steps can reproduce the bug?
- I’m repeatedly seeing Codex spend hours chasing a simple syntax error, and in >50% of sessions it breaks unit tests while still claiming the task is complete.
Issues Observed
- Endless syntax churn: gets stuck in cycles “fixing” parentheses/quoting, often introducing new syntax errors elsewhere.
- Test regressions: modifies code in ways that break existing tests unrelated to the request.
- False completion: reports “done” while the relevant unit test still fails.
- Fragile planning: changes ripple without guardrails; no rollback when a patch increases failures.
- Poor local validation: patches aren’t gated by a fast syntax check or a focused test run.
- sometimes, codex just deletes the tests instead of fixing the code
Impact
- Lost time (hours to a full day) rescuing builds, untangling trivial syntax/aliasing errors, and re-stabilizing tests.
- Erosion of trust: I must re-verify every “done” because the stated outcome often doesn’t match reality.
Minimal Repro Pattern
- I tested on https://github.com/launix-de/memcp/
- in e18f434c653f36a796784b862515edf418837d75 I added a failing unit test
- try to let codex fix it - it will mess up the whole code base or delete the tests
What Would Help
- Mandatory local checks: run a fast syntax/parse step and the targeted test(s) before stating completion.
- “No-regression” mode: forbid merging patches that increase failing tests relative to baseline.
- Precise scope control: restrict edits to files directly implicated by the task unless explicitly widened.
- Atomic patches: apply, test, and auto-rollback if failures increase; report diff and reason.
- Honest status: if tests can’t run, say so; don’t declare “done” without verification.
- Confidence + proof: include the exact test output (pass/fail counts) when claiming completion.
Context
- This occurred in a real repo with a planner/SQL dialect; the pattern is systemic, not project-specific.
Please prioritize reliability over zeal: pass the test, or clearly state why not, with actionable next steps.
What is the expected behavior?
_No response_
What do you see instead?
_No response_
Additional information
_No response_
This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗