macOS Desktop: completed recurring automations retain CUA node_repl workers

Open 💬 2 comments Opened Jul 27, 2026 by sookyeomKim
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

Summary

Completed recurring automations leave
/Applications/ChatGPT.app/Contents/Resources/cua_node/bin/node_repl
workers alive for hours. These are not zombies: they remain sleeping (S) and
are direct children of one codex app-server.

Environment and evidence

  • macOS, ChatGPT Desktop bundle
  • Parent command:

/Applications/ChatGPT.app/Contents/Resources/codex -c features.code_mode_host=true app-server --analytics-default-enabled

  • Observed during investigation: 284 node_repl workers, all S, all direct

children of the same app-server PID

  • Worker elapsed times ranged from about one to eight hours
  • Earlier observation: 268 workers and 806 automation session records that day
  • Worker creation times correlated with recurring-automation executions
  • The executable is a private Desktop resource:

/Applications/ChatGPT.app/Contents/Resources/cua_node/bin/node_repl

Public Codex repository investigation

The public openai/codex checkout has no cua_node or node_repl source
reference. Its generic code-mode host runner can accept an external host path,
but it is designed to reuse one live host connection per ThreadManager and
already has child kill + wait/reap handling when that shared connection is
dropped.

That model does not account for hundreds of direct node_repl children under
one app-server. The concrete CUA executable selection and per-automation
lifecycle appear to be in the Desktop bundle/private CUA runtime.

This is adjacent to #12491, but it should not be assumed to be fixed by #19753:
#19753 explicitly addresses stdio MCP server shutdown, whereas the observed
workers are cua_node/bin/node_repl direct app-server children.

Reproduction

  1. Restart Desktop and record the baseline node_repl count.
  2. Run a short recurring automation repeatedly and wait for each run to report

completion.

  1. After each completion, record:

``sh
ps -axo pid,ppid,state,etime,command |
rg '/Applications/ChatGPT\.app/Contents/Resources/cua_node/bin/node_repl'
``

  1. Wait at least one hour and compare with the baseline.
  2. Expected: no job-owned worker remains after its run reaches a terminal state.

Actual: the count rises and completed-run workers remain sleeping.

Requested owner investigation

Please route to ChatGPT Desktop / CUA runtime owners, with Codex app-server
code-mode maintainers consulted for integration.

Instrument the private CUA spawn site with:

  • automation job/run ID, spawned PID, parent PID, and spawn call site;
  • terminal reason: complete, cancel, error, channel close, or app-server stop;
  • terminate result and wait/reap result;
  • a bounded per-job registry count.

Do not use global pkill. If workers are job-owned, add a regression test that
runs several short jobs and asserts each owned process exits and is reaped on
all terminal paths. If the runtime is intentionally shared, assert a bounded
single-instance count and prove it is reused rather than recreated per run.

Related

  • #12491
  • #19753
  • #24510

View original on GitHub ↗

2 Comments

github-actions[bot] contributor · 1 month ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #35485

Powered by Codex Action

gkze · 1 month ago

Adding a macOS datapoint that confirms the retention behavior, plus a downstream consequence I haven't seen documented in any of the related issues (#26984, #27601, #35485): the leaked pipes exhaust the global XNU pipe-buffer pool, which deadlocks unrelated processes on the machine — software that has nothing to do with Codex — at FD counts nowhere near any per-process limit.

Environment

ChatGPT Desktop 26.721.41059 / build 5848
Bundled CLI: codex-cli 0.146.0-alpha.3.1
Codex Framework 150.0.7871.128
macOS 26.5.2 (25F84), Darwin 25.5.0, arm64

Retention datapoint (matches this issue)

  • 36 node_repl hosts under one codex -c features.code_mode_host=true app-server after ~2.5 h of light use, all sleeping (S), ~0.08 s total CPU each.
  • Spawn times cluster in bursts of 2–3 within seconds (e.g. 11:20:50, 11:20:58, 11:21:05), consistent with recurring-automation runs rather than interactive threads.
  • Only 3 of 36 had live workers (node kernel.js --session-id … + codex app-server --listen stdio://) under them; the rest were childless and hours old.
  • Each retained host pins ~13 pipe ends: ~5 on the app-server side (stdio + control pipes) and ~8 in the host.
  • The node_repl hosts are only the visible tip. Enumerating all children of the app-server shows every configured stdio MCP server and every enabled plugin's local MCP shim is retained the same way — roughly 50 full generations of the entire roster (~600 direct children), one generation per thread/automation run:

``text
52 uv tool uvx pagerduty-mcp
52 uv tool uvx mcp-proxy-for-aws@latest …
52 uv tool uvx markitdown-mcp@0.0.1a4
52 …/1Password.app/Contents/MacOS/onepassword-mcp
51 npm exec dotenv-cli … -- uvx mcp-proxy --transport streamablehttp …
51 node …/bunx-…-mcp-remote@latest/…
48 node …/bunx-…-@vantasdk/vanta-mcp-server/…
45 node ./mcp/server.mjs --stdio (plugin shims)
45 node ./mcp/server.mjs
45 node ./mcp/server.cjs --stdio
20 npm exec <private mcp server>
20 npm exec convex@latest mcp start
18 npm exec xcodebuildmcp@latest mcp
13 …/ChatGPT.app/Contents/Resources/cua_node/bin/node_repl
``

Each generation is ~20 processes (plus uvx/npx grandchildren). The app-server's own pipe FD count (1,701 at the snapshot above) matches ~3 stdio ends × ~570 retained children almost exactly — i.e. the same never-reaped per-thread lifecycle as this issue's node_repl workers, applied to every stdio MCP server. This looks like the same root cause class as #26984 / #30408, observed from the Desktop app-server.

Previously-undocumented consequence: global pipe-pool exhaustion deadlocks third-party processes

XNU allocates pipe buffers from a small fixed global pool. With the app-server holding ~1,600 pipe ends (plus the retained hosts' ends), the pool exhausts, and new pipes are granted almost no buffer. At that point any process on the machine that writes a modest amount into a fresh pipe before a reader exists blocks in write() forever.

The classic victim is a bash heredoc: bash writes the document into a pipe before exec'ing the command, assuming pipe-buffer capacity. Captured from a hung, completely unrelated process (direnv/nix-direnv sourcing an .envrc):

execute_disk_command
  do_redirections
    do_redirection_internal
      write   ← blocked indefinitely; target pipe granted ~0 buffer

Reproducible probe while the app-server holds ~1,600 pipe FDs (sizes in bytes):

heredoc  700B: HANGS  (bash small-doc path: direct write into pipe)
heredoc 4000B: ok     (bash switches to temp-file path, no pipe involved)
heredoc 16000B: ok

Practical impact here: every direnv export on the machine wedged permanently (nix-direnv writes its ~700 B reload script via heredoc), across multiple unrelated repos and terminals. Wedged writers never recover even after the pool frees — the pipe's buffer never grows and no reader ever appears — so each occurrence also requires hunting down and killing stuck processes.

Two properties make this failure mode nasty to attribute:

  1. It breaks processes with no relationship to Codex, with no error — just a hang.
  2. It happens at ~1,600 FDs, far below ulimit -n-style limits, so none of the usual EMFILE symptoms from #26984/#27662 appear first.

Evidence supporting reap-on-completion as the fix

SIGTERM-ing a childless (completed) node_repl host is handled cleanly today: the app-server notices the exit and promptly closes its side's pipe ends (parent FD count drops within seconds, verified repeatedly). So the graceful-teardown path already works — it's just never initiated. A reap when the run reaches a terminal state (or an idle timeout) would fully resolve the retention side.

Interim workaround for anyone else hitting this — cull childless hosts older than a few minutes (completed runs; the live ones keep their kernel.js worker as a child):

for as in $(pgrep -f 'features.code_mode_host=true app-server'); do
  for pid in $(pgrep -P "$as" -f node_repl); do
    [ "$(pgrep -P "$pid" | wc -l)" -eq 0 ] || continue   # has worker => active
    et=$(ps -p "$pid" -o etime= | tr -d ' ')
    case "$et" in
    *-* | *:*:*) ;;                                      # hours/days old
    *) [ "${et%%:*}" -ge 3 ] || continue ;;              # MM:SS, keep < 3 min
    esac
    [ "$(ps -p "$pid" -o ppid= | tr -d ' ')" = "$as" ] && kill "$pid"
  done
done

Note this only slows the bleed: the retained per-thread MCP server generations (the ~600-child table above) dwarf the node_repl share, and there is no safe external heuristic for distinguishing a dead thread's MCP servers from a live one's — so the pool still exhausts eventually and only an app-server restart resets it. A reap-on-thread-termination (or idle timeout) covering all per-thread stdio children, not just node_repl, appears to be the complete fix.