Codex over-tests routine changes and ignores release-only UI test scope

Summary

In the macOS Codex desktop app, the agent spent about 98 minutes running a full Xcode UI test plan after routine copy and integration changes, even though the relevant focused unit/integration suites had already passed and the user had asked for a dev build, not a release. The total task expanded to hours and required the user to intervene.

Expected behavior

• Scale verification to the change and the requested workflow.
• Use focused tests for ordinary changes.
• Reserve the full UI suite for an explicit release gate or an explicit user request.
• Stop or reassess when a verification step becomes disproportionately long.
• Avoid repetitive progress updates while an unnecessary long-running command continues.

Actual behavior

The agent started the complete Xcode test plan, which included a very large sequential UI suite, after focused tests had already passed. It continued through many simulator launches for roughly 98 minutes until the user explicitly told it to stop. The agent acknowledged afterward that this was unnecessary.

Impact

This made a small product iteration take hours, delayed installation of the requested dev build, consumed significant compute, and undermined confidence in autonomous execution.

Suggested improvements

• Add a proportional-verification heuristic or execution-time budget.
• Require explicit release intent before starting a full UI/E2E suite when focused tests are available.
• Reassess or ask before crossing a long-runtime threshold.
• Persist user and repository preferences such as UI tests being release-only.
• Prefer meaningful milestone updates instead of repeated wait messages.

Environment

• ChatGPT/Codex macOS desktop app
• Local iOS/Xcode project
• Observed 2026-07-27

No private project logs or user data are attached.