Codex CLI: add persistent self-healing monitor workflows for long-running jobs
What variant of Codex are you using?
Codex CLI / TUI, especially on headless Linux, SSH, and remote development machines.
What feature would you like to see?
Summary
Add a first-class, persistent monitor workflow to Codex CLI that can return to the same thread at a future time or when a background event occurs, inspect progress, and perform a bounded diagnose -> fix -> restart/resume -> verify loop when it detects a failure.
This is more than a timer or status reminder. The goal is to let Codex safely supervise long-running development, deployment, evaluation, and ML training jobs without requiring the user to repeatedly ask for progress updates.
Problem
A typical workflow today looks like this:
- Codex starts a long-running build, experiment, training run, deployment, or development task.
- The user leaves the session and returns 30 minutes later.
- Only after the user asks for status does Codex inspect the logs and discover that the job crashed, stalled, produced bad cases, or hit a bug much earlier.
- Time was lost because the agent had no durable way to wake itself, check progress, and continue remediation.
External cron jobs can inspect a process or log, but they do not naturally preserve the active Codex thread, its context, constraints, prior diagnosis, sandbox policy, and repair history.
Example workflows
Illustrative commands:
/monitor in 30m "Check the training run, summarize progress, and investigate any failures."
/monitor every 30m --until complete --name training-guard \
"Inspect logs and metrics. If the run is healthy, keep waiting. If it crashed, stalled,
regressed, or produced bad cases, diagnose the cause, apply fixes within the approved
scope, restart or resume the run, verify recovery, and continue monitoring."
The exact syntax is not important; the end-to-end behavior is.
Desired lifecycle
start job
-> sleep without model calls
-> wake on a timer or relevant event
-> inspect process, logs, metrics, tests, and artifacts
-> healthy: record status and wait again
-> unhealthy: diagnose -> fix -> restart/resume -> verify
-> completed: stop the monitor and report the final result
-> unsafe, ambiguous, or permission-blocked: ask the user
Requirements
- Time- and event-based wakeups
- One-shot checks such as "in 30 minutes".
- Recurring checks such as "every 30 minutes".
- Optional event triggers such as process exit, matching an error pattern, file changes, CI completion, metric thresholds, or stalled progress.
- Same-thread continuity
- Resume the same Codex thread with its existing context and constraints.
- Do not start an unrelated stateless
codex execrun for every check.
- No token-burning polling while idle
- Waiting should make no model calls.
- Local timers or event watchers should wake the model only when a check is due or a relevant event occurs.
- Bounded autonomous remediation
- On a detected failure, Codex may inspect evidence, make changes within the current sandbox and explicitly approved scope, run targeted validation, and restart or resume the job.
- Preserve normal approval boundaries for privileged, destructive, publishing, or otherwise high-impact actions.
- Support maximum retries, time/cost budgets, and fail-closed stop conditions to prevent runaway self-repair loops.
- Durability and delivery safety
- Ideally persist monitor state across terminal or app-server restarts.
- Do not lose completion or failure events while the thread is busy or temporarily unavailable.
- Do not interrupt an active turn; queue or coalesce the wakeup for the next safe point.
- Inspectable control surface
- List active monitors, next wake time, latest check, health state, and last action.
- Pause, resume, trigger now, edit, and stop a monitor.
- Keep a bounded history of checks, detected anomalies, repairs, validation results, and termination reason.
Why this should be an end-to-end feature
Several existing issues cover important primitives:
- #8317 and #15679 cover delayed or recurring scheduling and
/loop. - #29922, #32188, and #32221 cover event-driven monitoring and waking an idle session without polling.
- #17737 covers a monitoring-oriented TUI and warning/error visibility.
Those primitives are necessary, but this request focuses on the missing user-facing workflow and lifecycle contract: monitor -> detect -> diagnose -> remediate -> verify -> continue or stop, with persistent same-thread context and explicit safety boundaries.
If the underlying implementation is split across the scheduler, monitor, goal, and background-exec components, this issue can serve as the end-to-end CLI behavior and acceptance criteria tying them together.
Additional information
The strongest use cases are long-running training and evaluation jobs, development servers, CI or deployment monitoring, overnight test matrices, and other workflows where discovering a failure only after the user manually asks for progress wastes substantial time.
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗