Background process polling wastes tokens: each write_stdin poll triggers full API turn with complete history

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• #11629

Powered by Codex Action

Please multiple team member sees this issue best workaround I found is use a subagent to not mess up with main agent context.

I completely agree with this issue, but the token waste is only half the problem. The biggest dealbreaker here is that the AI literally "loses patience" and abandons long-running tasks.

While the token burn is awful, the current polling behavior completely breaks down on heavy, time-consuming commands (like a large compilation or a massive test suite that takes 20-30 minutes).

Here is what happens:

• The agent starts the command in the background.
• It polls write_stdin 10 to 15 times over the span of a few minutes.
• Because the process is still running and returning nothing new, the AI agent gets "impatient" (likely hitting a maximum iteration limit or hallucinating that the process is hung).
• It abruptly stops checking, assumes the task failed, or just drops the context entirely.

It is incredibly frustrating to wait for a 30-minute build, only to realize the AI gave up on monitoring it 5 minutes in.

For single, synchronous, or known long-running tasks, the agent should stop abusing the background terminal feature, and simply execute them in the foreground.

@jitlabs-sg can't you already do option B with background_terminal_max_timeout in config.toml?

* It polls write_stdin 10 to 15 times over the span of a few minutes.

You're lucky. It's polling every second (yield_time_ms=1000) by default for my tasks. I can't use it for background tasks that don't produce frequent output. I've even added heartbeat output and the model ignores it and aborts because "it's taking too long to get the final output."

All of the reasons everyone put in this issue are extremely valid and relevant, but I would like to add something to this.

I found a workaround by using MCP servers for long-running commands, but the problem is you have to develop your own sandboxing and agent thread bookkeeping. Basically, the only way to ensure long-running commands run until completion is by routing them through an MCP server that can be abused or avoided, or just create a bunch of friction.

They should at least allow a config to raise the minimum yield time so that we can raise it to whatever we want, such as 2 billion years. That would be better than the way it currently works.

You guys are gonna get a kick out of this.

• ~~New skill: https://github.com/robertmsale/.codex/tree/main/skills/command-execution~~
• ~~New MCP: https://github.com/robertmsale/.codex/tree/main/mcp/command-execution~~

^^^ Outdated

Codex searches your PATH for the shell you're using, and assigns that path to the PTY that the agent will use for the rest of the program. There is no configurable way to change the polling frequency, and in unified_exec=false mode there is no way to raise the command execution cap beyond 10 minutes (also the agent can set arbitrarily low timeouts, which it does fairly often, and runs the command multiple times up to the 10 min cap before giving up or deciding the timeout is a codebase issue and tries to fix non-problems). However, you can trick Codex (the application) into loading a shell that produces a job ID with every command execution that the agent sees, and with the MCP server they can pass that job ID and wait until that command completes. On mac the process looks like this:

1. Create a folder somewhere (e.g. /opt/homebrew/shim on my system)
2. Symbolically link the launch-job script, essentially a passthrough to zsh with recursion-safe job ID wrapper into that folder as zsh
3. Launch codex with env PATH="/path/to/shim:$PATH" codex

The skill instructions tell them that all commands produce a job ID whether they like it or not, and if the command is not finished they use the MCP to wait until the command finishes. Set the tool call timeout to some absurdly high value and that's all she wrote!

Past couple days have been using this with extremely good results. Wasting tokens is not the biggest issue, as others have pointed out, it's the decisions the agent makes when a process runs for what they perceive as "too long", and no amount of prompt engineering seems to prevent them from eventually terminating a good process (besides tediously steering the agent). Also this prevents the spinlock/reawakening so it really does save tokens.

~~Anyways, figured I'd share this little workaround incase OpenAI decides to close this as not planned (lack of votes or otherwise), or it just takes months to land on an agreed solution + implementation.~~ Not practical, preserved for historical reference only

Looks like this in practice:
<img width="674" height="523" alt="Image" src="https://github.com/user-attachments/assets/aa3ad4f9-5b97-4415-aa2a-b39a364de180" />

I pushed on the narrowest tool-side fix locally, and my read is that this issue is real.

One important detail from the current code: background_terminal_max_timeout does not currently do what people in this thread seem to expect for empty polls. In the write_stdin path, empty polls still effectively use the short fixed window because the requested yield is clamped into the 5s .. background_terminal_max_timeout range. So when the model asks for something like 1000ms, the actual empty-poll wait is still 5000ms, not the configured background timeout.

Locally, I patched the unified-exec path so that:

• empty write_stdin polls wait up to background_terminal_max_timeout
• empty polls return as soon as fresh output arrives or the process exits
• non-empty writes keep the shorter interactive clamp

That means a single empty poll can bridge a long silent period without bouncing back to the model every 5 seconds, but it still yields promptly once there is something useful to send back.

I also added a regression that crosses the old 5-second ceiling (sleep 5.5 && echo ...) and verifies that one empty poll captures the delayed output. Targeted codex-core tests passed locally.

I ran a full workspace cargo test too. It was not fully green on this machine, but the new unified-exec regression passed in the full codex_core run, and several non-unified failures from that workspace run passed when rerun in isolation. The only still-reproducible blocker I hit separately was the existing seatbelt test pair.

So my current read is:

• the issue is real
• background_terminal_max_timeout is currently only an upper bound, not the actual empty-poll wait window people expect
• a narrow tool-side fix appears viable without changing the overall follow-up loop

This is fundamentally an architecture problem, not a tuning problem. The core issue is that scheduling decisions (should I wait for this process? how long?) are being routed through the LLM, which means every "should I keep waiting?" check costs a full API round-trip with the entire conversation history re-serialized. Adjusting MIN_EMPTY_YIELD_TIME_MS from 5s to 30s (Option B) just makes the bleeding slower — you still pay for reasoning tokens the model burns "deciding" to keep waiting, which is a foregone conclusion.

We hit this exact class of problem when building our multi-agent orchestrator. The solution was separating the control plane from the inference plane completely. The orchestrator is deterministic Python code — it never calls an LLM. It spawns agents as subprocesses, then checks liveness with proc.poll() (literally a non-blocking syscall that returns the exit code or None). The tick loop runs every ~30 seconds, calls check_alive() on each agent session, and reaps dead ones. Zero tokens consumed for scheduling. Zero API calls for "is my build done yet?"

The key architectural insight: process lifecycle management is not an LLM problem. An LLM cannot wait more efficiently than a select() call or a PID check. When you route "is process X still running?" through the model, you're paying for the model to parse the question, "reason" about it (consuming output tokens for a trivially deterministic answer), and emit a tool call — all while re-transmitting the full conversation context. With 250+ history items at ~10s cadence during a 60-second build, that's ~6 full-context API calls doing nothing.

Option C in the issue description is closest to the right fix. But I'd go further: the tool handler layer should own the entire wait lifecycle. When exec_command spawns a background process, the local runtime should:

1. Register the PID in a local process table
2. Return immediately to the model with "process started, PID N"
3. Watch for exit locally (epoll/kqueue, not polling the API)
4. Only re-enter the model when there's actual new information: process exited (with exit code + final output), or meaningful stdout was produced

This is effectively an event-driven architecture instead of a polling one. The model gets called when there's something to think about, not to ask "anything new?"

Related patterns that help in practice:

Per-task model routing. We assign a complexity tier to each task and route to an appropriate model. A trivial formatting fix doesn't need the same model or token budget as a multi-file refactor. This prevents the secondary waste where an expensive model is burning tokens on wait-poll cycles for work that shouldn't have used that model in the first place.

Token growth monitoring. We sample cumulative token consumption per agent every 30 seconds and detect quadratic growth patterns — when the per-interval token delta doubles across consecutive windows, it signals unbounded context accumulation (exactly what the polling loop produces). Agents that exceed a token threshold with zero file changes get auto-killed. A circuit breaker guards against infinite compaction loops when context windows fill up.

Cost anomaly detection. Per-task cost ceilings relative to the complexity tier median, burn-rate tracking against budget, output/input token ratio alerting, and retry-spiral detection (cumulative retry cost exceeding a multiplier of the first attempt). These catch exactly the scenario described here — a routine build-and-test cycle silently burning 10x the expected tokens.

The proxy billing model (per payload KB) makes this even worse since you're paying quadratically: each poll re-sends the growing history, so poll cost increases over the session lifetime. With N polls and history growing by ~3 items per poll, total payload is roughly proportional to N^2.

For anyone dealing with this now: if you can intercept at the proxy layer, you could short-circuit write_stdin tool calls that return "(no new output)" by caching and replaying the response without forwarding to the API. Ugly, but it stops the bleeding while the proper fix ships.

We've been iterating on these patterns in Bernstein (https://github.com/chernistry/bernstein) — the token monitor and cost anomaly detector specifically exist because LLM-mediated polling is a recurring footgun in agent orchestration.

This is a real problem now with 5.5 being very aggressive with write_stdin polling. I'm observing 5.5 using somewhere around 2-10x more input tokens (largely cached ones) than say 5.4. Codex analyzed its own logs and came to the conclusion that this is the culprit.

The scenario is almost always a (long) running background task that gets multiple polls during its lifetime, with the whole prompt being fully re-sent to the endpoint (hopefully cached) potentially dozens of times.

This will very quickly exhaust limits and/or cost real money in tokens depending on how you are connected to the backend api. Not sure if this is a harness deficiency or an actual model deficiency. To the extent you're seeing reports of 5.5 exhausting limits quickly, this is almost certainly the culprit.

I am sorry to say but i am pausing 5.5 adoption until this gets solved. I don't mind paying for the top tier model doing the stuff better. But this looks like a bug. openai should reimburse somehow for this. Hope somebody is listening...

I think the issue is even more fundamental than “polling wastes tokens.” The real problem is that process-liveness is being delegated to the model loop, so the system pays inference cost for a question the runtime could answer deterministically.

A useful split is:

• control plane: process lifecycle, wait conditions, output availability, exit status
• reasoning plane: what to do after new information arrives

Right now write_stdin empty-poll behavior blends those together. That means every “is the build done yet?” check becomes:

1. rehydrate history
2. call the model
3. let the model decide to wait
4. issue another tool call
5. repeat

That is expensive even if most input tokens are cached, and it gets worse after long sessions because the serialized history keeps growing while the information content of each poll is basically zero.

The most reliable fix I’ve seen in agent systems is to make the runtime event-driven:

• when a background process starts, register it locally
• block locally until one of three events happens: new stdout/stderr, exit, or deadline reached
• only then re-enter the model with a fresh tool result

In other words, needs_follow_up=true should not be the default for an empty poll. An empty poll should usually stay inside the runtime, not bounce back into the model.

This also has second-order benefits beyond token burn:

• fewer compactions triggered by wait loops
• less chance the model “gets impatient” and abandons a healthy long-running task
• less repeated re-reading after context compression

We hit a similar pattern while building knowledge-graph: once durable state and wait-state were moved out of the conversational loop, long sessions became much more predictable because the model stopped paying reasoning tax on deterministic bookkeeping. Same broad lesson here: keep waiting local, use the model only when something changed. Repo in case useful: https://github.com/hilyfux/knowledge-graph

@SproutSeeds

I pushed on the narrowest tool-side fix locally, and my read is that this issue is real.

Do you have a patch pushed on a branch somewhere? Getting burned on this and we already have some local patches in our environment so easy lift to get a short term workaround on our end at least while maintainers decide on an actual fix.

Thanks for the feedback and discussion. I agree this is an architectural gap that we need to fill.

I've been exploring some options, but I'd love to hear additional ideas from the community. Are there solutions that you've explored that you've found to be especially effective? I'd like to land on a solution that generalizes across a wide variety of polling scenarios. One promising approach is to delegate the polling loop to a local script, which can be written by the agent if it's not part of the project.

@etraut-openai

Seems like the ideal fix is a more fundamental split where more of this waiting/polling logic lives in the local deterministic harness, avoiding most empty model callbacks entirely.

As a stopgap, maybe the poll/callback interval could be increased meaningfully, or made user-configurable as a responsiveness/cost tradeoff. I do understand there is useful behavior here: with concurrent background tasks, the model sometimes starts additional work while waiting for earlier tasks, so fully blocking on long-running commands would lose some valuable agent behavior, especially for users who care more about throughput than token cost.

Another possible mitigation would be to throttle/batch callbacks. For example, only re-enter the main model loop when there is enough new terminal output, the process exits, the user interrupts, or some longer timeout is reached. A manual “flush/wake” keypress could also preserve on-demand responsiveness while making long-running autonomous costs more predictable. More generally, this could expose responsiveness as a sliding scale: highly reactive, moderate, or batch-oriented for users who are fine with slower feedback in exchange for lower token burn.

A more elaborate version would be a cheap local/model-side gate that decides whether the new background-process state is actually worth surfacing to the main loop, but I suspect the simpler invariant gets most of the value: no new output + process still running should usually be handled locally rather than as another model-visible turn.

I pushed the narrow stopgap branch here:

• branch: https://github.com/SproutSeeds/codex/tree/issue-13733-empty-poll-timeout
• commit: https://github.com/SproutSeeds/codex/commit/c3956e3885d7c7bd7627b067032141fa82b95655
• diff: https://github.com/openai/codex/compare/main...SproutSeeds:issue-13733-empty-poll-timeout

What it changes:

• empty write_stdin polls use the configured background terminal timeout as the wait window
• empty polls still return as soon as fresh output arrives or the process exits
• non-empty writes keep the shorter interactive clamp
• the write_stdin tool schema now calls out that empty polls may wait longer

Verification:

• cargo test -p codex-core polls -- --nocapture --test-threads=1
• cargo test -p codex-core write_stdin_tool_matches_expected_spec -- --nocapture --test-threads=1
• just fix -p codex-core
• just fmt
• git diff --check

I would treat this as a bounded stopgap, not the full architectural answer. It addresses the current empty-poll wait-window bug while preserving prompt return on output/exit. The broader fix still seems like runtime/control-plane ownership of wait state so long-running silent processes do not keep re-entering the model loop just to ask whether anything changed.

I'd love to hear additional ideas from the community. Are there solutions that you've explored that you've found to be especially effective?

@etraut-openai I have an idea that could not only fix the underlying issue, but also make it the most powerful feature in the toolbox.

Right now there is a Starlark specification for "rules" where you can prefix match commands and decide to automatically run with or without sandbox:

# Prompt before running commands with the prefix `gh pr view` outside the sandbox.
prefix_rule(
    # The prefix to match.
    pattern = ["gh", "pr", "view"],

    # The action to take when Codex requests to run a matching command.
    decision = "prompt",

    # Optional rationale for why this rule exists.
    justification = "Viewing PRs is allowed with approval",
)

I think it makes complete sense to extend the rules system to allow additional configurations. Aside from @jitlabs-sg suggestions about not awakening the agent if commands produce no new outputs (fantastic idea), we should be able to hint to unified_exec that certain commands could hang in perpetuity, and rules would be the easiest place to do it. Take for example:

prefix_rule(
    pattern = ["cargo", ["check", "build"]],
    decision = "allow",
    blocking = true,
)

Adding a parameter like blocking hints that this command can potentially run for a very long time. Let's say 45 mins build time. We do not want this to be interrupted. Whereas:

prefix_rule(
    pattern = ["cargo", "test"],
    decision = "allow",
    blocking = false,
    minimum_poll_time = 60, 
)

This tells unified_exec "high risk for infinite loop, deadlock" and "let agent awaken no less than every 60 seconds to check on it".

Basically let this feature be tunable instead of one-size-fits-all applying a polling standard to all commands. I think the way rules prefix match using tree sitter makes this a really strong place to extend unified exec. There could be more configurations added to cover edge cases, but I think those two parameters would offer a lot of value.

This is the kind of bug where the runtime is charging users to re-decide whether nothing changed. I agree the preferred fix is local waiting plus event-driven wakeup, not another model turn for every empty poll. More generally, background work needs a separate admission rule from foreground reasoning: if there is no new artifact, no new output, and no exit status, the model should not be spending a fresh full-history turn to rediscover that fact.

The main annoyance here really is when you have a context with 150k tokens in it and then this background loop simply resubmits 150k tokens every time there's a poll for a background process state and it slowly grows.

It would be considerably better if there was a "watcher" agent/thread/context that gets initialized with minimal information and its only job is to wake up the main thread. So let's say you keep resubmitting 2-3k tokens as you watch a job. This would already be a massive improvement and would let most of the existing workflows stay effectively unchanged.

I want to clarify some misperceptions in this thread. Several of you have said that when a new turn is started, the entire history is sent to the server. That's not the case. Only the new message is sent to the server. This has been true since we introduced a web socket transport for the responses API several months ago. Prior to that, the harness sent the full history to the responses API each time using the HTTP transport. Even then, most of that history was cached on the server, and cached tokens do not affect your usage.

I wanted to clarify this because the token impact of polling isn't as significant as you might think from reading the comments in this thread. That said, a "watcher" tool is still a good idea!

That seems inaccurate. When I pay for API on a per token basis the cached tokens certainly get charged. In fact this is the vast majority of my spend on a codex session even though the cached tokens are "cheap".

@jusava, that's a good point. I was looking at it through the lens of subscription plan usage.

@etraut-openai, It's a major drag on both paid API users as well as your server loads. Anecdotally, around the time when 5.5 got introduced, my guess a combination of both harness changes as well as model changes took the ratio of input to cache input from 1:3-5 to something like 1:10-20. So while 5.5 is definitely more token efficient vs. 5.4 in output terms, it's a lot more expensive (both server load and token cost perspective) due to this.

As a concrete example, one of my recent codex sessions had the following breakdown in token usage:
Input: 61M = ($305)
Cached Input: 750M ($375)
Output: 2M (reasoning .5M) ($60).

So in this example more than half of the cost came from cached tokens and in this instance the input vs. cached input ratio is actually not that "bad". I can often see 1:20 or even worse depending on the task. And this seems for no good reason, just effectively burning tokens. Not everybody has steipetes unlimited token budget..

I agree the preferred fix is local waiting plus event-driven wakeup, not another model turn for every empty poll.

I think this, but also we need a way to classify certain commands as blocking and for setting up a minimum poll time. Two contrasting examples:

cargo build: Say it's totally uncached, no incremental, has to pull in all deps, takes up to 45 minutes. The worst thing ever is watching an agent go "Still waiting on the build to complete" 540 times. In fact, at the 500th time, the agent says "Hmmm, this build is taking too long. Something must be awry. Terminating the build now." Yikes...

npm run dev: Fires up a dev server for some stack like React/Vite with hot reloading, polls stdin a few times until they get the host:port, then proceeds to make hot edits, take screenshots with Playwright and look at em, polling occasionally to look out for "ts compilation failed", and after edits are done kills the process. This is essentially the perfect usage for unified_exec=true as it currently exists.

It's not just about "no new outputs, no reawaken". Sometimes we want a command to truly be backgrounded. The current solution is run it in tmux and let em tail the pane, but if they run commands in tmux it escapes the sandbox, so that solution is not without its own downsides.

Being able to say "This command will, full stop, take 45 minutes. Have a good nap." Or at least "This command will take 45 mins, I'll let you wake up every 5 mins to check on it." I don't think this is a one-size-fits-all solution, we need to have a configuration layer to make unified_exec not be wasteful.

I think the cleanest generalization is a runtime-owned watcher with explicit wake receipts, not another loop the model has to keep rediscovering.

For the empty-poll case, the runtime already has the decisive facts locally:

• process is still alive
• no new stdout/stderr arrived
• no exit status exists yet

If none of those changed, the default action should stay local. Re-enter the model only when something changed or when a user-configured wake boundary is actually hit.

Where configuration helps is command class, not the default control flow. A config layer for truly blocking commands or minimum wake intervals makes sense, but silent background work with no new artifact should not keep spending model turns by default.

There should be a function similar to Claude Code Monitor, where when a long command is running, a Monitor watches whether the task has completed. If it has finished, then it checks the task.

ref:https://x.com/noahzweben/status/2042332268450963774

Same problem

This needs a control-plane wait path.

If a process is alive and stdout/stderr/exit status have not changed, the runtime can keep waiting locally. Re-enter the model only on a state transition: new output, exit, timeout boundary, user input, or a configured progress interval.

For cost debugging, I would also record wait receipts separately from model turns: command id, wait duration, empty polls suppressed, cached-token churn avoided, and final wake reason. Then users can see whether spend came from real work or background waiting.

---

_Generated with ax._

@etraut-openai on your "delegate the polling loop to a local script" idea, and wanting something that generalizes: I went and built roughly this, and the piece that generalizes it is to not have the agent author a script per case, but to expose the wait itself as a first-class tool where the watcher is just an arbitrary shell command. A local script, tail -F log | grep, inotifywait, a CI status loop, or the build itself then all collapse to one start/stop/list surface. The runtime owns the wait and only re-enters the model on a real state transition: new output, exit, or a timeout/interval boundary. That is the control-plane split most of this thread converged on, turned into a tool the model drives.

It helps to separate the two harms, because they need different framing after your caching clarification. The token cost is the smaller, arguable half now. The undisputed half is reliability: the failure people keep reporting (a 30-minute build, the agent gets impatient around poll 15 and abandons a healthy process) goes away when the session is genuinely idle and woken by the event, because the model is never sitting in a loop deciding whether to give up. While the watch is quiet there are zero turns at all.

Two ideas raised above slot in cleanly. The wait-receipts approach (record wait duration, empty polls suppressed, and wake reason per watcher) makes spend attributable to real work versus background waiting. And the per-command blocking / minimum-poll config is the complementary layer that decides which commands get a long-lived watcher versus the interactive clamp.

I have this running in my own Codex (CLI and Desktop) behind a feature flag, with tests. Working reference, one commit on the v0.142.0 tag: https://github.com/openai/codex/compare/rust-v0.142.0...yaanfpv:codex:monitor-tool, and the full design is in #29922. Happy to turn it into a PR if the direction fits.

This is a bad bad problem , might require a rocket scientist to solve

I have a quantified multi-session reproduction that extends this issue from single-process polling to coordinator/worker orchestration.

Environment:

• codex-cli 0.144.6
• gpt-5.6-sol
• 258,400-token context window
• macOS
• Three sessions: coordinator, completed worker reused as supervisor, and active Unity worker

I parsed the local JSONL logs using:

• response_item as a response packet
• payload.type == reasoning as a model reasoning step
• SHA/manifest-related tool requests by matching shasum, SHA-256, artifact hashes, candidate manifests, and hash manifests

Adjacent-window comparison:

Functional code/test window:

• 510 response packets
• 170 reasoning steps
• 149 tool requests
• 23.15M total_tokens

Five minutes after the worker explicitly froze source changes and Unity runs:

• 203 response packets
• 61 reasoning steps
• 66 tool requests
• 13 SHA/manifest-related tool requests
• 9.02M total_tokens
• 8.82M cached input tokens (97.8%)
• 64.5% of those raw tokens came from the coordinator and already-completed supervisor, rather than the active worker

Important correction: reasoning steps per packet did not increase (33.3% -> 30.0%). Instead:

• response packet rate increased 11.1%
• tool-request rate increased 23.6%
• SHA/manifest tool share increased from 0.67% to 19.70% (29.3x)
• total tokens per reasoning step increased 8.6%

This suggests the primary amplification is repeated full-history/cache replay caused by tool polling and cross-session supervision, not additional useful reasoning.

Expected behavior:

1. Worker completion should notify the coordinator without model-driven polling.
2. Completed workers should become idle and not supervise replacement workers.
3. Status/wait operations should have a no-model or lightweight-context path.
4. Codex should expose per-lane poll, packet, and cached-token metrics.
5. Idle/finished workers should have an automatic lifecycle stop.

Affected local thread IDs can be uploaded with /feedback:

• coordinator: 019f7d0e-3f38-7301-a471-5ea0bd2c7db1
• completed supervisor: 019f7ea2-fd97-7401-b7ec-847e61af721b
• runtime worker: 019f7edd-a5f5-7952-9c44-2c440960900c