Code-mode `exec` silently degrades a long-running command into a full-context model polling loop (34.6M tokens burned after the task already completed)
Summary
In code mode, when a script passed to exec outlasts the yield deadline, the CLI parks it as a numbered cell and hands control back to the model. There is no completion push and no timer — the only way to learn that the command finished is for the model to spend a turn calling wait(cell_id). Each such turn resubmits the entire conversation context.
The result is a busy-wait loop where "ask whether the process is done yet" costs exactly the same as "perform a full code analysis". In the session below, a single git worktree remove --force over a large directory tree (~9.5 minutes of real unlink time) produced 90 model turns, 21,610,789 input tokens and 3,837 output tokens — an input:output ratio of 5632:1 — and all of it happened after the agent had already delivered its result and written its report to disk.
This is the mechanism-level report behind the symptoms in #38335, #34115 and #38437.
Mechanism
- Model emits
tools.exec_command({cmd, yield_time_ms, max_output_tokens}). - Script exceeds the yield deadline (default 10000 ms). The CLI returns
Script running with cell ID <n>— not an error, not a handle the runtime will follow up on. - Resumption is pull, not push. The model must emit
wait({cell_id, yield_time_ms, max_tokens})to learn anything. - Every
waitis a full model turn: complete context re-sent, ~240k input tokens in, ~31 tokens out. - The loop is not idle. There is no
schedule_wakeupand no timer event in the rollout — it is a continuous turn chain that also holds a concurrency slot for its whole duration.
Two consequences worth stating explicitly:
exit_codecannot rescue this. Per the CLI's own contract,exit_codeis present only "when the command finished during this call" — so while the process is genuinely still running, it is legitimately absent on every single poll. The model is being told the truth; the truth just costs 240k tokens to obtain.git worktree remove(likerm -rf) prints nothing while unlinking. "Deleting file 80,000 of 134,000" and "hung forever" are byte-identical from the model's point of view, so it has no basis on which to stop polling.
Observed incident
Session 019ffc35-8a14-76b3-bd0b-7e6bfe18b62c, plan prolite, 2026-08-14 17:39:56Z → 18:15:09Z.
Total: 275 turns / 54,701,161 tokens.
Timeline:
| turn/record | event |
|---|---|
| 638 | task result written back (ok: true) — work complete |
| 642 | report artifact written to disk — deliverable complete |
| 648 | exec_command with a 7-statement cleanup compound, yield_time_ms: 30000 |
| 649 | Script running with cell ID 88 / Wall time 11.0 seconds |
| 651–663 | wait{cell_id:"88", yield_time_ms:1000} × 5 |
| 664 | Script completed |
| 666 | write_stdin({session_id: 1095, chars: "", yield_time_ms: 30000, max_output_tokens: 5000}) |
| 667 | Script running with cell ID 89 / Wall time 11.0 seconds |
| 668–931 | 16 byte-identical write_stdin execs + 67 wait calls, cells 89–98 and 103–107, every one at yield_time_ms: 1000, all returning net-empty output |
| 932 | model gives up and sends ^C (Ctrl-C) — 9.5 minutes later |
| 935–938 | task_complete |
Note record 666: an inner yield_time_ms: 30000 nested inside a call whose own outer yield deadline is 10000 ms. The park is structurally guaranteed, every time.
The actual slow statement was the second of seven:
git -C repos/<repo> worktree remove ../../agent_workspace/<id>/worktrees/<name> --force
Filesystem forensics confirmed it was genuinely progressing, not deadlocked: the target tree still held 134,186 files (130,633 of them under node_modules), with node_modules mtime 02:13 and the package store mtime 02:14 — i.e. actively unlinking mid-poll. Statements 3–7 of the compound never ran.
Per-turn bills during the loop (token_count events):
rec trigger input cached output total
662 wait 237,280 236,672 31 237,311
665 wait 237,339 236,672 31 237,370
668 exec 237,400 236,672 45 237,445
...
931 wait 242,794 241,792 31 242,825
937 exec 242,920 241,792 396 243,316
90 turns: input 21,610,789 / output 3,837
mean: 240,119 in / 42 out ratio 5632:1
Input ratchets upward monotonically (237,280 → 242,920) because each poll writes itself into the history that the next poll must re-send. Every turn's entire product is 31 tokens: {"cell_id":"89","yield_time_ms":1000,"max_tokens":5000}.
Inter-call gaps are irregular inference latency, not a schedule — median 3.05 s, range 2.23–15.27 s (wait-preceding, n=72). There are zero schedule_wakeup, user_message, task_started or interrupt records after 648. And with yield_time_ms: 1000, the 67 waits spent a combined 67 seconds actually waiting, across 9.5 minutes of wall clock.
The escape hatch exists, and nobody can use it
I want to be precise here, because the fix is not "add a primitive". The primitive is already shipped:
// @exec: {"yield_time_ms": 120000, "max_output_tokens": 1000}
The CLI's own documentation even demonstrates it: "imagegen needs a few minutes to finish. In code-mode, use the first-line @exec directive to give the initial call 120 seconds and the same yield for any waits that follow." There is no documented upper bound — the only stated constraint is "must be non-negative safe integers".
Across 19,392 exec calls in my fleet's rollouts, the pragma was used 0 times.
That is the actual defect surface:
- The default (10000 ms) is far below the duration of ordinary filesystem and build operations.
- Exceeding it degrades silently into a paid polling loop instead of surfacing a handleable condition.
- Using the escape hatch requires knowing the command's duration before running it — precisely the thing you cannot know for
rm -rf,bun install,git worktree remove, or any user-supplied tool. - Resumption is pull-only, so the cost of not knowing is billed per turn at full context.
This is not a one-off
Same fleet, same rollout corpus:
- 8,752 MCP tool calls; 86 took ≥10 s; 70 of those 86 (81%) were parked into a cell. Each parked call then needed median 4 / mean 4.7 / max 13
waitturns to collect. 329 waits + 70 parked execs ≈ 399 turns of pure polling — on the order of 10^8 tokens at these context sizes. - All 12 slowest calls were a single long-running tool endpoint. Any user who registers a slow MCP tool or shells out to an external CLI reproduces this by default.
- Global
waityield_time_msdistribution:30000→ 2,516 calls,1000→ 979 calls. The model frequently picks a 1-second poll interval against multi-minute work, which is the worst possible ratio. (I'll note honestly that this specific choice is model behavior — but the CLI is what makes a 1 s interval cost 240k tokens instead of being harmlessly cheap.)
Question about quota enforcement
In the same session, rate_limits.primary.used_percent progressed:
17:40:07 21,664 cumulative tokens 92.0%
17:46:01 4,176,704 95.0%
17:53:23 14,271,350 98.0%
17:57:08 20,084,287 100.0% <-- limit reached
The busy-wait loop did not begin until ~18:05 — 8 minutes after used_percent hit 100.0 — and the CLI accepted a further 18 minutes and ~34.6M tokens of requests, with rate_limit_reached_type reported as null on every event.
Is post-100% execution expected behavior? If the limit is a soft accounting boundary rather than an admission gate, that is a reasonable design — but combined with the polling loop above it means a single unattended slow command can spend multiples of a weekly allowance with no backpressure anywhere in the stack.
Expected behavior / asks
- Push cell completion, or at minimum make the default yield adaptive (e.g. exponential backoff up to a bounded ceiling) so that N-minute commands do not cost N/interval full-context turns.
- Raise the 10 s default, or apply the documented pragma value to follow-up
waits automatically. - Include elapsed time and a liveness signal in the
wait/park output so the model can distinguish "progressing" from "hung" and choose a sane interval. Right now both look like empty output. - Cap or warn on repeated identical polls — 16 byte-identical
write_stdincalls in a row is trivially detectable. - Per-session usage telemetry surfaced to the user, so a runaway loop is visible before it consumes a plan rather than after.
- Quota reset for the affected account. The 34.6M tokens were spent entirely after the task's result had been delivered and its artifact written, in a loop the model had no primitive to avoid and no signal to exit.
Environment
- macOS (Darwin 25.3.0), Apple Silicon
- Codex CLI in code mode, ChatGPT-plan auth, plan
prolite - Non-interactive
codex exec, multiple concurrent sessions - Evidence: rollout JSONL
019ffc35-8a14-76b3-bd0b-7e6bfe18b62c(identifiers redacted; happy to share the relevanttoken_count/custom_tool_call/function_callrecords on request)
Related
- #38335 — same class, maintainer-labeled
rate-limits. I posted a condensed version of this trace as a comment there (https://github.com/openai/codex/issues/38335#issuecomment-5289676893); this issue exists so the mechanism is searchable under its own title rather than buried in a thread about quota symptoms. Its Reproduction B (external CLI delegation burning 2–3% of a weekly allowance in ~1 minute) and its two asks — "waiting for an external CLI or background process should not silently generate large amounts of model usage" and "automatic polling, retries or waiting should not repeatedly resubmit large contexts at significant quota cost" — are exactly the mechanism documented above. - #34115 — empty
write_stdinpolling of a live background process; explicitly mentionsrm -rfcleanup routing. - #38437 — 56.4M tokens / 2.59B cached.
- #36827, #38093, #38453, #38367, #38480 — related quota-burn reports.
3 Comments
Potential duplicates detected. Please review them and close your issue if it is a duplicate.
Powered by Codex Action
Verified the shape on
main@ 1f41cc5d92:wait(core/src/tools/code_mode/wait_spec.rs) takes a model-chosenyield_time_ms, there's no completion push, and nothing in the tool result nudges the model toward longer yields — in your incident it chose 1,000 ms polls, and the harness faithfully executed 90 of them.Three fix layers, smallest first:
waitis called on a cell that's still running and produced no new output, the harness can simply block longer than requested (escalating 30s → 2m → 5m) before returning — turning 90 poll turns into ~3 without any protocol change. The model's request is a hint, not a contract worth 240k tokens per second of impatience.Second independent incident with the same mechanism. Two things to add: a data point on your caching question, and a second cost profile for the same bug.
Codex CLI 0.149.0,
codex execdriven by an orchestrator (non-interactive). Our variant iswrite_stdinpolling on aunified_execsession rather thanwait(cell_id)in code mode — same pull-not-push shape, different tool.Session totals (175 turns):
| | |
|---|---|
| input tokens | 23,957,294 |
| cached input | 23,542,272 (98.3%) |
| new input | 415,022 |
| output tokens | 52,694 |
| input:output | 454:1 |
174 tool calls: 86
write_stdin, of which 84 are pure polls (chars: ""), against 80exec_command. Requestedyield_time_msacross all 166 occurrences:{1000: 68, 10000: 40, 25000: 1, 30000: 56, 35000: 1}— note the 35000, where the model asked for 35s andclamp_yield_timesilently capped it to 30s (#22541).Splitting turns by kind is where it gets clear:
| | poll turns | work turns |
|---|---|---|
| turns | 84 | 90 |
| context re-sent | 13,507,179 | 10,450,115 |
| new input | 179,819 | 235,203 |
| output | 6,151 | 46,543 |
| avg context/turn | 160,799 | 116,112 |
| avg output/turn | 73 | 517 |
The polls re-sent more context than all the real work combined, for 12% of the output.
On your point 3 (caching): ours hit cache, and it did not save us. 98.7% on the poll turns alone — 13,327,360 of 13,507,179. Your 21.6M billed input for 90 near-identical polls suggests you did not get those hits.
But per #37299, cached context is still metered against the subscription: that report shows the account usage API returning ~290M tokens for a single day at 97–98% cache, taking a Pro account from 0% to 90% of its weekly limit in ~15.5 hours. Their per-turn profile (~137–141k input, 97.4–98.1% cached) is almost identical to ours.
So a cache hit makes this loop cheaper in dollars but not in quota. Our 13.5M of re-sent poll context counted against the window even though only 179,819 tokens of it were new. Worth stating explicitly in the fix discussion: layers 1 and 2 pay for themselves in quota terms even when prompt caching works perfectly. A cache-only fix would not.
A second cost profile, which I think broadens the report. You noted that
git worktree removeprints nothing while unlinking. We were waiting on CI withgh run watch, which without a TTY cannot overwrite with ANSI, so it reprints the entire job tree on every 10s refresh. That put 179,819 tokens of genuinely new input through the loop on top of the context re-send — 2,140 per poll, actually more per turn than our work turns' 2,613.So the same bug has two shapes: a silent command costs pure context re-send, a chatty one costs that plus real input. Both reduce to "asking whether the process is done costs the same as doing the work".
And one nuance: our wait was entirely legitimate. The agent had finished, committed, pushed, opened a PR, and was waiting on a pipeline that genuinely took 21.6 minutes — jobs running sequentially with multi-minute gaps. It wasn't spinning on a hung process and had no way to know better. So this isn't only about commands that produce no output; it also hits the ordinary "wait for my own CI" pattern, where the correct duration is tens of minutes and the harness caps observation at 30s.
Your fix layer 1 (escalating server-side yield) would have collapsed our 84 polls to roughly 5. Layer 2 (completion push) would have made them zero.
Workaround for anyone hitting this from an orchestrator: instruct the agent not to wait for CI at all — finish the turn, and let the caller poll outside the model loop, where a
gh run watchsubprocess costs nothing. And if an agent must wait on something, prefer a command that stays silent until it's done: the redraw is what turns a cheap loop into an expensive one.---
Cross-linking, since these two threads don't reference each other: #37299 has the billing-side evidence (account usage API, ~290M/day at 97–98% cache) that answers the caching question raised here, and this issue has the mechanism-level explanation that #37299 lacks. The automated duplicate detector flagged them as possible duplicates — they are not; they are two halves of the same problem, and a fix for one does not fix the other.