Repeated regressions, false “done” signals, and time-draining syntax loops

Resolved 💬 2 comments Opened Aug 31, 2025 by carli2 Closed Nov 10, 2025

What version of Codex is running?

codex-cli 0.27.0

Which model were you using?

gpt5-high

What platform is your computer?

arm64 on raspberry pi

What steps can reproduce the bug?

  • I’m repeatedly seeing Codex spend hours chasing a simple syntax error, and in >50% of sessions it breaks unit tests while still claiming the task is complete.

Issues Observed

  • Endless syntax churn: gets stuck in cycles “fixing” parentheses/quoting, often introducing new syntax errors elsewhere.
  • Test regressions: modifies code in ways that break existing tests unrelated to the request.
  • False completion: reports “done” while the relevant unit test still fails.
  • Fragile planning: changes ripple without guardrails; no rollback when a patch increases failures.
  • Poor local validation: patches aren’t gated by a fast syntax check or a focused test run.
  • sometimes, codex just deletes the tests instead of fixing the code

Impact

  • Lost time (hours to a full day) rescuing builds, untangling trivial syntax/aliasing errors, and re-stabilizing tests.
  • Erosion of trust: I must re-verify every “done” because the stated outcome often doesn’t match reality.

Minimal Repro Pattern

  • I tested on https://github.com/launix-de/memcp/
  • in e18f434c653f36a796784b862515edf418837d75 I added a failing unit test
  • try to let codex fix it - it will mess up the whole code base or delete the tests

What Would Help

  • Mandatory local checks: run a fast syntax/parse step and the targeted test(s) before stating completion.
  • “No-regression” mode: forbid merging patches that increase failing tests relative to baseline.
  • Precise scope control: restrict edits to files directly implicated by the task unless explicitly widened.
  • Atomic patches: apply, test, and auto-rollback if failures increase; report diff and reason.
  • Honest status: if tests can’t run, say so; don’t declare “done” without verification.
  • Confidence + proof: include the exact test output (pass/fail counts) when claiming completion.

Context

  • This occurred in a real repo with a planner/SQL dialect; the pattern is systemic, not project-specific.

Please prioritize reliability over zeal: pass the test, or clearly state why not, with actionable next steps.

What is the expected behavior?

_No response_

What do you see instead?

_No response_

Additional information

_No response_

View original on GitHub ↗

This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗