Codex for macOS accumulates Computer Use / MCP helper processes and unreaped zombie children, causing HID lag and WindowServer/TCC stalls

Open 💬 10 comments Opened Jun 2, 2026 by quasa0
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

Codex macOS Resource Leak / Unreaped Child Processes Report

GitHub Issue Draft

Title

Codex for macOS accumulates Computer Use / MCP helper processes and unreaped zombie children, causing HID lag and WindowServer/TCC stalls

Summary

On macOS, a long-running Codex session accumulated a large number of Codex-owned helper processes related to Computer Use / MCP tooling. The machine became visibly laggy. Inspection showed:

  • Dozens of live SkyComputerUseClient mcp processes.
  • Dozens of live node_repl processes.
  • Dozens of live codex app-server --listen stdio:// helper processes.
  • More than 100 zombie child processes whose parent was the main Codex GUI process.
  • macOS spin reports for Codex showing slow HID event handling, with WindowServer blocked in the TCC event-monitor permission path.

Manual cleanup of stale helper processes reduced live Codex-related memory use by roughly 3 GB, but the zombie processes could not be cleared without restarting Codex because the main Codex process did not reap them.

Related Issues

This may share root cause with MCP stdio lifecycle leaks such as #17832, but this report adds a separate Computer Use case with SkyComputerUseClient accumulation, Codex-owned zombie children, and macOS WindowServer/TCC HID stalls. It does not look like the SkyComputerUseClient code-signing crash in #21200 or the simple unexpected-exit case in #18549.

Environment

Sanitized system details:

  • App: Codex for macOS
  • Codex version: 26.527.31326
  • Codex bundle version: 3390
  • Codex bundle ID: com.openai.codex
  • Chromium base: 148.0.7778.179
  • Computer Use app bundle ID: com.openai.sky.CUAService
  • Computer Use app version: 1.0
  • Computer Use build: 799
  • Computer Use client bundle ID: com.openai.sky.CUAService.cli
  • Computer Use client version/build: 1.0 (799)
  • OS: macOS 26.5 build 25F5042g
  • Architecture: Apple Silicon / arm64
  • CPU: Apple M-series Pro class, 14 logical CPUs
  • Memory: 24 GB
  • Thermal state: no thermal or performance warning recorded during sampling

Privacy redactions applied:

  • User home path replaced with <HOME>.
  • Host/device name redacted.
  • Project/workspace paths omitted.
  • Unrelated app/process names omitted unless directly relevant.
  • Raw spin report should not be attached unless separately redacted; it contains a full process inventory.

Observed Behavior

The main Codex process stayed alive for approximately 16 hours. During that period, helper processes accumulated rather than being cleaned up.

Initial process counts observed:

SkyComputerUseClient       count=50   cpu=18.0%   mem=4.7%   rss=1448.1 MB
node_repl                  count=47   cpu=0.0%    mem=0.0%   rss=405.3 MB
codex app-server stdio     count=47   cpu=3.2%    mem=6.3%   rss=1715.4 MB
Codex app/server/renderers count=several, additional RSS/CPU

Aggregate Codex-related footprint before cleanup:

codex_related count=143-144
codex_related RSS ~= 4.8-4.9 GB
codex_related CPU ~= 28-70% depending on sample

Zombie processes:

zombies=126
parent=<codex-main-pid> zombies=126

After killing stale helper processes, the zombie count increased slightly rather than disappearing:

zombies=128
parent=<codex-main-pid> zombies=128

That strongly suggests the main Codex process is not reaping exited child processes.

System Symptoms

The machine was visibly laggy. System-level sampling showed:

Load Avg: 8.12 6.57 6.23
Processes: 980 total
Threads: 6301
CPU usage: 13.70% user, 22.98% sys, 63.30% idle
PhysMem: 22 GB used, 1.3 GB unused
Compressor: ~5.5 GB
Memory pressure: not critical
Thermal warnings: none

The strongest live CPU consumers during the bad state included:

trustd        ~90-100% CPU in multiple samples
syspolicyd    ~20-60% CPU in multiple samples
WindowServer  ~40-50% CPU in multiple samples
Codex GPU     ~8-40% CPU depending on sample
Codex renderer/app server also periodically high

After helper cleanup, trustd and syspolicyd dropped back to low single-digit CPU in later samples. This suggests the large helper-process population may have been amplifying code-signing/trust/TCC work.

macOS Spin Report Evidence

macOS generated at least two Codex-related spin reports during the session.

Recent spin report summary, sanitized:

Date/Time:        2026-05-31 17:51:29 local time
OS Version:       macOS 26.5 (Build 25F5042g)
Reason:           Slow response to HID event
Command:          WindowServer
Affected Process: Codex [<codex-main-pid>]
Event:            spin
Duration:         2.98s
Duration Sampled: 2.77s
Steps:            277 (10ms sampling interval)
Report threshold: 0.5s
Active CPUs:      14
Memory size:      24 GB
Memory pressure:  average 40%, highest 42%
Available memory: average 14.26 GB, lowest 13.75 GB
HID Events:       LeftMouseDown*, LeftMouseUp

The heaviest stack for the target process was in WindowServer waiting on TCC event-monitor access checks:

WindowServer
  SLXServer
  server_loop
  CGXRunOneServicesPass
  post_port_data
  rendezvousHandler
  CGXHandleMessageWithResponseBuffer
  _XGetEventShmem
  WSEventSourceCreateGraphShmemForAuditToken
  WSAuditTokenCanMonitorEvents
  WSAuditTokenHoldsCachedProcessRight
  audit_token_check_tcc_access
  TCCAccessPreflightWithAuditToken
  TCCAccessRequest
  tccd_send_message
  xpc_connection_send_message_with_reply_sync
  dispatch_mach_send_with_result_and_wait_for_reply
  mach_msg2_trap

The slow HID event line:

CGXGetNextEvent  1-274  3000.000ms  WindowServer thread

There was another earlier Codex spin report the same day:

Affected Process: Codex
Event: spin
Duration: 1.28s
WindowServer stack again included kTCCServiceListenEvent / TCC preflight path

Nearby TCC logs also showed repeated privacy preflights for screen/event access by a non-Codex screen/window utility. That may have amplified the symptom, but it does not explain the Codex-owned process leak or the Codex-owned zombies.

Cleanup Experiment

The session was kept alive by not killing:

  • The main Codex GUI process.
  • The main Codex codex app-server --analytics-default-enabled process.

Stale helpers were terminated carefully:

Killed 40 stale SkyComputerUseClient mcp processes.
Killed 80 stale node_repl / codex app-server --listen stdio:// helper processes.
Killed 1 stale SkyComputerUseService process.
Killed several old orphaned Codex crash/keyboard helper processes from previous app launches.

After cleanup:

codex_related count ~= 15-18
codex_related RSS ~= 1.8-1.9 GB
SkyComputerUseClient herd gone
node_repl / stdio helper herd gone
trustd / syspolicyd CPU substantially lower
zombies still present under main Codex process

A single fresh helper triad later respawned:

node_repl
SkyComputerUseClient mcp
codex app-server --listen stdio://

That may be the active/current session helper set. The bug is not that one helper set exists; the bug is that many old helper sets accumulated and were not cleaned up.

Expected Behavior

Codex should:

  • Keep at most the necessary active helper processes for currently active tools/sessions.
  • Tear down stale SkyComputerUseClient, node_repl, and stdio codex app-server helpers after a tool/session completes.
  • Reap exited child processes so zombie children do not accumulate.
  • Avoid triggering system-wide TCC/trust/code-signing churn due to stale helper population.
  • Avoid causing WindowServer HID stalls when Computer Use/event-monitor permissions are involved.

Actual Behavior

Codex accumulated:

  • Approximately 50 Computer Use client processes.
  • Approximately 47 node_repl processes.
  • Approximately 47 stdio codex app-server children.
  • 126+ zombie children owned by the main Codex process.

The system became laggy and macOS generated Codex spin reports involving WindowServer/TCC event-monitor checks.

Repro / Diagnostic Commands

These are safe commands to run locally. Redact output before posting publicly.

Count Codex-related helpers:

ps -axo pid,ppid,pgid,stat,%cpu,%mem,rss,etime,args |
  awk '/Codex|SkyComputerUse|node_repl|codex app-server/ {print}'

Aggregate Codex-related footprint:

ps -axo pid,ppid,pgid,stat,%cpu,%mem,rss,etime,args |
  awk '/Codex|SkyComputerUse|node_repl|codex app-server/ {
    cpu += $5; mem += $6; rss += $7; n++
  }
  END {
    printf "codex_related count=%d cpu=%.1f mem%%=%.1f rssMB=%.1f\n",
      n, cpu, mem, rss/1024
  }'

Count zombie children by parent:

ps -axo stat,ppid |
  awk '$1 ~ /^Z/ { z++; p[$2]++ }
  END {
    print "zombies=" z+0
    for (ppid in p) print "parent", ppid, "zombies", p[ppid]
  }'

Inspect top CPU processes:

ps -axo pid,ppid,%cpu,%mem,rss,etime,args |
  sort -nrk3 |
  head -30

Sanitize paths before sharing:

sed -E 's#/Users/[^/]+#<HOME>#g; s#[[:alnum:]_.-]+_2026-[0-9-]+-[0-9]+_[^./]+\\.spin#<redacted-spin-file>.spin#g'

Workaround

The clean workaround is to quit and reopen Codex. That should clear:

  • Live stale helpers.
  • Zombie processes owned by the old Codex parent.

If keeping the current session alive is required, stale helper processes can be killed manually, but this does not clear zombies because the main Codex process still fails to reap them. Killing children may actually increase the zombie count until Codex is restarted.

Suggested Fix Area

Likely areas to inspect:

  • Lifecycle ownership for Computer Use MCP client processes.
  • Lifecycle ownership for per-session node_repl processes.
  • Lifecycle ownership for stdio codex app-server --listen stdio:// child processes.
  • Child-process reaping in the main Codex macOS process.
  • Cleanup path when a thread/tool session ends, errors, times out, or is superseded.
  • Interaction between Computer Use/event-monitoring permissions and TCC preflight calls.

Severity

High for long-running desktop sessions:

  • Causes visible system lag.
  • Accumulates multiple GB of resident memory.
  • Creates unreaped zombie children.
  • Triggers macOS spin reports.
  • Involves WindowServer/TCC HID event stalls, so user input latency is affected.

Notes For Maintainers

I can provide raw spin reports privately if needed, but they contain a full process inventory and local paths. The issue body above intentionally omits those details while preserving the relevant stack, counts, versions, and cleanup experiment.

View original on GitHub ↗

10 Comments

github-actions[bot] contributor · 1 month ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #25697
  • #24347
  • #25742
  • #25667
  • #25719

Powered by Codex Action

quasa0 · 1 month ago

Reviewed the suggested duplicates. I do not think this is an exact duplicate.

Closest is #25719 because it also involves macOS syspolicyd / trustd pressure around Codex Desktop / Computer Use. This issue adds separate lifecycle evidence: accumulated SkyComputerUseClient processes, duplicate node_repl / codex app-server --listen stdio:// helpers, 126+ Codex-owned zombie children, and WindowServer/TCC HID spin reports.

The Windows/Linux MCP issues (#25697, #24347, #25742) look related at the lifecycle class level, but this report is specifically macOS Codex Desktop + Computer Use + unreaped children. #25667 is also adjacent, but focuses on code_sign_clone temp dirs after quit.

johnsilvavlogs · 1 month ago

Adding current-build evidence from another macOS machine. This still reproduces on the current Sparkle appcast build and looks like the same lifecycle/perf class as this issue, with one extra signal: repeated AppServerConnection initialize/reconnect failures while Computer Use MCP helpers fan out during ordinary tool discovery.

Environment:

  • Codex Desktop: 26.602.40724, bundle build 3593
  • Appcast top item verified as the same build: 26.602.40724 / 3593, published Fri, 05 Jun 2026 17:00:22 +0000
  • Global CLI: codex-cli 0.135.0
  • Computer Use plugin: computer-use@openai-bundled 1.0.809
  • Computer Use helper bundle: com.openai.sky.CUAService, version/build 1.0 (809)
  • macOS: 26.3 build 25D125, arm64
  • Hardware: Apple Silicon MacBook Pro, M1 Max, 64 GB memory

Negative checks:

  • Codex app is current; not a stale install.
  • Computer Use plugin is present and enabled in ~/.codex/config.toml.
  • SKY_CUA_SERVICE_PATH points at <HOME>/.codex/plugins/cache/openai-bundled/computer-use/1.0.809/Codex Computer Use.app.
  • Computer Use helper code signature verifies cleanly.
  • state_5.sqlite and logs_2.sqlite both pass pragma integrity_check.
  • Memory pressure was low: 0 swap used, no pageouts/swapouts in the sampled window.
  • Current system TCC DB has Screen Recording and Accessibility allowed for both com.openai.codex and com.openai.sky.CUAService.

Current process inventory, without restarting/killing Codex:

main codex app-server:
  PID 55044, parent Codex GUI PID 54986, ~36% CPU, ~614 MB RSS, uptime ~1h15m

Codex-owned helper fan-out:
  7 x SkyComputerUseClient mcp
  7 x node_repl
  7 x codex app-server --listen stdio://
  14 x node ./mcp/server.{cjs,mjs}

The oldest Computer Use MCP/helper family was ~71 minutes old; two more were ~35-36 minutes old; newer helper families appeared during this diagnostic even though no Computer Use desktop action was invoked. That suggests the Computer Use MCP process is being started as part of tool listing/session setup, not only as part of actual Computer Use actions.

High-signal log evidence:

~/Library/Logs/com.openai.codex/2026/06/07/codex-desktop-478b4760-97d6-4bdf-b1dc-7dc5f6179de3-54986-t0-i1-004733-0.log

L13129-L13135:
  Initialize handshake still pending
  connectionError={"code":"connection-failed","message":"Codex app-server initialize handshake timed out"}
  reconnectAttempt=151
  app_server_connection.fail cause=initialize_handshake_timeout
  app_server_connection.state_changed cause=transport_closed
  initialize_handshake_result outcome=failure

L13583-L13588:
  same failure again at reconnectAttempt=152

Earlier in the same long-running app session:

~/Library/Logs/com.openai.codex/2026/06/06/codex-desktop-30fe6ed6-63f9-405c-aa0b-80661ad44790-49672-t0-i1-000000-1.log

L6836-L6842:
  screen recording reconcile failed kind=PermissionRequired state=Unsupported
  error_message="No shareable content available: Content unavailable: The user declined TCCs for application, window, display capture"
  followed immediately by reconnectAttempt=101 and initialize_handshake_timeout / transport_closed

L8235-L8240:
  IAB_LIFECYCLE route window is not live
  browser use route window missing for cursor/active state

There is also a Computer Use MCP protocol-compat warning in the SQLite logs:

2026-06-06 15:40:20 codex_mcp::connection_manager
  Failed to list resources for MCP server 'computer-use':
  Mcp error: -32601: Method not found: Unknown method: resources/list

2026-06-06 15:40:20 codex_mcp::connection_manager
  Failed to list resource templates for MCP server 'computer-use':
  Mcp error: -32601: Method not found: Unknown method: resources/templates/list

I think the user-visible symptom is "Computer Use feels flaky", but the local root-cause shape looks broader:

  1. Long-running Codex Desktop accumulates per-turn MCP/helper families.
  2. Computer Use MCP helpers are spawned during ordinary tool discovery, so unrelated work increases Computer Use helper count.
  3. AppServerConnection is stuck in repeated initialize handshake timeout / transport closed loops.
  4. Browser Use route state and screen-capture reconciliation also show lifecycle/TCC-adjacent failures, which makes desktop/browser-control surfaces feel unreliable even when the Computer Use plugin is installed and current.

This does not look like the plugin-missing issues (#18258, #21579, #26429) on this machine. It looks closer to this issue plus the app-server/MCP timeout family.

0xdevalias · 1 month ago

Feedback ID / Bug Report ID: 019eb57d-d318-78f2-8c45-76dea130046d

---

A brief summary of what seems to be related issues that I just noticed due to my system being at its process limit, so failing to even run programs anymore:

Codex 26.527.31326 (3390)
macOS 26.3 (25D125), Darwin 25.3.0 arm64
main Codex GUI PID 19100
~9,940 unreaped zombie children under PID 19100
MYUSERNAME process count 10,581–10,656 / kern.maxprocperuid 10,666
zsh fork failed: resource temporarily unavailable
kill -CHLD did not cause Codex to reap children
manual helper kills did not solve it
updating/restarting to 26.608.12217 cleared zombies to 0

The most important maintainer-facing points (according to Codex, GPT 5.5 (high)) are:

  • nearly 10,000 zombies were direct children of the main Codex GUI process;
  • the leak accumulated across an approximately 11-day app lifetime;
  • it exhausted kern.maxprocperuid and prevented all ordinary process spawning;
  • killing live helpers did not release the zombies;
  • SIGCHLD and SIGCONT+SIGCHLD did not trigger reaping;
  • old-parent exit immediately cleared the table;
  • restart cleanup does not yet prove the new version prevents recurrence.

---

Summary of seemingly related issues by Codex (GPT 5.5, high) while reviewing this:

  • #12491
  • directly relevant historical precedent, but closed
  • #26293
  • is adjacent. Add only a short cross-link if SkyComputerUseClient turn-ended orphans are independently confirmed.
  • #26327
  • tangential lifecycle analogy
  • #26984
  • tangential lifecycle analogy
  • #26904
  • likely unrelated beyond generic process exhaustion

Other potentially useful/relevant threads for maybe more context:

---

Full overview / summary of everything tried/explored as per Codex (GPT 5.5, high):

Codex Desktop zombie-process accumulation and macOS process-table exhaustion

Executive summary

On June 11, 2026, a long-running Codex Desktop process on macOS had accumulated
approximately 9,940 unreaped zombie children. The zombies consumed nearly all
of the per-user process-table allowance:

user processes:       10,581-10,656
kern.maxprocperuid:   10,666
Codex-owned zombies:  ~9,936-9,949

This directly caused ordinary process creation to fail:

zsh: fork failed: resource temporarily unavailable

The evidence establishes that:

  1. The main Codex GUI process was the parent of almost all zombies.
  2. The zombies had accumulated throughout an approximately 11-day app lifetime.
  3. Killing dozens of live Codex helper processes recovered only a small number

of process slots and did not remove the zombies.

  1. Sending SIGCHLD, including after SIGCONT, did not make Codex reap them.
  2. Updating Codex caused the old main process to exit; immediately afterward,

the zombie count was zero.

The evidence does not establish:

  • which executable each zombie had originally been, because macOS displayed

the exited children only as <defunct>;

  • that the newer Codex release fixed the underlying leak, because the zero

count was measured immediately after restart with almost no subsequent use;

  • that the live node_repl, Computer Use, or stdio app-server helpers were

necessarily the zombie source, although their accumulation makes them
plausible lifecycle-related contributors.

This is best treated as a severe additional reproduction of
openai/codex#25744, with a
stronger demonstrated impact: complete per-user process-table exhaustion.

Environment

Public-safe environment details:

Codex Desktop: 26.527.31326 (bundle build 3390)
Updated to:    26.608.12217, released June 9, 2026
macOS:         26.3 (25D125)
Darwin:        25.3.0 arm64
Architecture:  Apple Silicon

Redactions used below:

<USER>             local account name
<HOST>             host/device name
<HOME>             /Users/<USER>
<OLD_CODEX_PID>    19100
<NEW_CODEX_PID>    72552

PIDs are retained where useful because they are ephemeral and essential for
understanding parent-child relationships.

Timeline and observations

1. Initial process-limit diagnosis

The first diagnostic set checked OS details, launchd/kernel limits, process
counts by user, and process counts by executable.

Command:

sw_vers
uname -a

echo "launchctl limits:"
launchctl limit maxproc
launchctl limit maxfiles

echo "kernel process/file limits:"
sysctl kern.maxproc kern.maxprocperuid kern.maxfiles kern.maxfilesperproc

echo "current process counts:"
ps -axo user= | sort | uniq -c | sort -nr | head -20

echo "top process names:"
ps -axo comm= | sort | uniq -c | sort -nr | head -30

Relevant output:

ProductVersion: 26.3
BuildVersion:   25D125
Darwin 25.3.0 ... arm64

maxproc                10666  16000
kern.maxproc:          16000
kern.maxprocperuid:    10666

10656 <USER>

9927 <defunct>
  75 /Applications/Codex.app/Contents/Resources/codex
  73 /Applications/Codex.app/Contents/Resources/node_repl

Interpretation:

  • The account was only about 10 processes below kern.maxprocperuid.
  • Approximately 93% of the account's process-table entries were zombies.
  • This was process-table exhaustion, not primarily a low configured limit.

2. Identification of the zombie parent

Command:

ps -axo ppid= | sort | uniq -c | sort -nr | head -20

Relevant output:

9946 19100

The parent was then identified:

ps -o pid,ppid,user,stat,etime,command -p 19100

Output:

PID    PPID  USER    STAT  ELAPSED       COMMAND
19100  1     <USER>  S/R   10-23:27:xx  /Applications/Codex.app/Contents/MacOS/Codex

Direct-child sampling:

ps -axo pid,ppid,stat,etime,command \
  | awk '$2 == 19100 { print }' \
  | head -50

Representative output:

115  19100  Z       11:38:55     <defunct>
140  19100  Z    02-04:25:15     <defunct>
142  19100  Z       06:15:55     <defunct>
152  19100  Z    10-21:47:02     <defunct>
155  19100  Z    10-21:47:02     <defunct>
...

The oldest sampled zombies were almost as old as the Codex process itself.
This indicates gradual accumulation across the long-running app session rather
than a single recent burst.

At this point even a simple diagnostic pipeline failed:

zsh: fork failed: resource temporarily unavailable
[1] broken pipe

3. Killing live helper processes did not solve the limit

Dozens of live Codex-related backend/helper processes were manually terminated.
The follow-up checks were:

launchctl limit maxproc
sysctl kern.maxprocperuid kern.maxproc

ps -axo user= | sort | uniq -c | sort -nr | head -10
ps -axo comm= | sort | uniq -c | sort -nr | head -20

ps -axo stat= \
  | awk '$1 ~ /^Z/ { n++ } END { print n+0 }'

ps -axo ppid=,stat= \
  | awk '$1 == 19100 && $2 ~ /^Z/ { n++ } END { print n+0 }'

Relevant output:

<USER> processes:         10581
total <defunct>:          9949
zombies under PID 19100: 9936

Live helper counts had dropped substantially:

13 node_repl
13 SkyComputerUseClient
 7 SkyComputerUseService

Interpretation:

  • Terminating live helpers recovered only approximately 75 process slots.
  • The account remained approximately 85 processes below the limit.
  • Zombie counts increased while live helper counts fell.
  • Existing zombies could be released only by the parent calling

wait()/waitpid(), or by the parent exiting.

The top-zombie-parent check made the concentration explicit:

ps -axo pid=,ppid=,stat=,command= \
  | awk '$3 ~ /^Z/ { z[$2]++ } END { for (p in z) print z[p], p }' \
  | sort -nr \
  | head -20 \
  | while read count ppid; do
      printf "%6s zombies  PPID=%s  " "$count" "$ppid"
      ps -o pid=,ppid=,stat=,etime=,command= -p "$ppid" 2>/dev/null
    done

Relevant output:

9936 zombies  PPID=19100  ... /Applications/Codex.app/Contents/MacOS/Codex
   1 zombie   PPID=<node_repl PID> ...
   1 zombie   PPID=<node_repl PID> ...

The few one-zombie node_repl parents were negligible compared with the
approximately 9,936 zombies directly parented by the main GUI process.

4. SIGCHLD did not trigger reaping

Commands:

kill -CHLD 19100

ps -axo ppid=,stat= \
  | awk '$1 == 19100 && $2 ~ /^Z/ { n++ } END { print n+0 }'

Output:

9940

Retry after continuing the process:

kill -CONT 19100
kill -CHLD 19100

ps -axo ppid=,stat= \
  | awk '$1 == 19100 && $2 ~ /^Z/ { n++ } END { print n+0 }'

Output:

9942

Interpretation:

  • Codex did not run a successful catch-up reap in response to SIGCHLD.
  • The count continued to rise slightly during diagnosis.
  • There was no reliable way to clear the existing zombies without making the

main process exit or injecting reaping behavior into it.

5. Final pre-restart snapshot

The saved codex-zombie-process-snapshot.txt was captured later in the same
bad state.

Relevant output:

maxproc:                 10666 soft / 16000 hard
kern.maxprocperuid:      10666
<USER> process count:    10614
total <defunct>:         9962
Codex parent PID:        19100
Codex elapsed time:      10-23:53:52
zombies under Codex:     9949

The snapshot also showed live Codex-related helpers:

13 node_repl
13 SkyComputerUseClient
 8 SkyComputerUseService
13 Codex renderer processes

This is useful supporting context, but the zombie command names had already
collapsed to <defunct>, so it cannot prove that any of these helper classes
were the exited children represented by the zombie entries.

6. Main-process sample

Command:

sample 19100 10 \
  -file ~/Desktop/codex-main-zombie-parent-sample.txt

Sample metadata:

Process:                  Codex [19100]
Version:                  26.527.31326 (3390)
Launch Time:              2026-05-31 16:39:52 +1000
Sample Time:              2026-06-11 16:28:21 +1000
Physical footprint:       10.6 GB
Physical footprint peak:  11.3 GB

The sampled main thread and many workers were predominantly waiting in
event-loop primitives such as mach_msg2_trap, kevent, and uv_run.
No waitpid, wait4, SIGCHLD, child-exit, or explicit reaping stack was
captured.

This sample is not proof of the missing code path: a 10-second statistical
sample cannot prove that a function never runs. It does show that no active
reaping loop was observed while nearly 10,000 zombies were pending.

The 10.6 GB process footprint is independently notable, but zombies themselves
do not account for that memory. Memory retention and child reaping should not
be conflated without more profiling.

7. Saved unified-log exports

Two log exports were saved:

codex-spawn-limit-logs.txt   202,255 bytes
codex-eagain-maxproc-logs.txt 1,124,680 bytes

The commands used were:

log show --last 24h --style compact --predicate '
  process CONTAINS[c] "Codex" OR
  process CONTAINS[c] "codex" OR
  process CONTAINS[c] "node_repl" OR
  eventMessage CONTAINS[c] "posix_spawn" OR
  eventMessage CONTAINS[c] "fork failed" OR
  eventMessage CONTAINS[c] "resource temporarily unavailable" OR
  eventMessage CONTAINS[c] "EAGAIN" OR
  eventMessage CONTAINS[c] "maxproc" OR
  eventMessage CONTAINS[c] "spawn"
' > ~/Desktop/codex-spawn-limit-logs.txt

and:

log show --last 24h --style compact --predicate '
  process CONTAINS[c] "Codex" OR
  eventMessage CONTAINS[c] "EAGAIN" OR
  eventMessage CONTAINS[c] "resource temporarily unavailable" OR
  eventMessage CONTAINS[c] "maxproc"
' > ~/Desktop/codex-eagain-maxproc-logs.txt

Review result:

direct matches for:
  EAGAIN
  resource temporarily unavailable
  posix_spawn
  fork failed
  failed to spawn
  maxproc

result: 0

The exports contain thousands of ordinary Codex/macOS framework messages, but
no direct spawn-failure line matching the predicates above.

This negative result matters:

  • it does not weaken the process-table diagnosis, which was directly proven by

ps, kern.maxprocperuid, and the shell's fork failure;

  • it means these log exports should not be cited as evidence of a specific

Codex spawn exception;

  • the broad export contains unrelated device names, identifiers, local paths,

account-service activity, and other private system context, so it should not
be attached publicly without heavy redaction.

8. Session-preservation note

codex-session-restore.txt was saved before restart to preserve a manual list
of open Codex windows and thread URLs. It contains 12 thread identifiers and
project/task titles.

It explains why preserving the running app initially mattered, but it does not
identify the leak. The raw file should not be posted because thread IDs,
workspace topics, and project names are private.

9. Update/restart recovery

The Codex update changed the main process from PID 19100 to PID 72552.

Commands:

ps -o pid,ppid,user,stat,etime,cputime,command -p 19100

pgrep -fl '/Applications/Codex.app/Contents/MacOS/Codex'

NEWPID=72552
ps -axo ppid=,stat= \
  | awk -v ppid="$NEWPID" \
    '$1 == ppid && $2 ~ /^Z/ { n++ } END { print n+0 }'

Relevant output:

old PID 19100: no process
new Codex PID: 72552
zombies under new PID: 0

The post-restart attachment summarized in the shared chat additionally reported:

<USER> processes: 645
total zombies:    0

Interpretation:

  • Exiting the old main process released the approximately 9,940 zombie entries.
  • The machine recovered approximately 10,000 per-user process slots.
  • This validates the immediate mechanism and workaround.
  • It does not establish that version 26.608.12217 fixed recurrence,

because the replacement process had just started and had barely been used.

Root-cause assessment

Proven

The main Codex GUI process failed to reap thousands of exited direct children.
Those zombie entries accumulated over an approximately 11-day app lifetime and
eventually exhausted kern.maxprocperuid.

At the OS level, the missing invariant is:

Every child spawned or adopted by the long-lived Codex GUI process must
eventually have its exit status collected with wait/waitpid or equivalent.

Strongly suggested, but not proven

The broader Codex helper lifecycle likely contributes to the rate of child
creation:

  • 75 codex resource processes and 73 node_repl processes were live before

manual cleanup.

  • Later snapshots still showed repeated node_repl, Computer Use client, and

Computer Use service processes.

  • Related users report repeated MCP, stdio app-server, Computer Use, and

notification-helper fanout.

However, the original executable identity of each zombie is unavailable.
Therefore the public report should say these helper families are plausible
sources or adjacent lifecycle symptoms, not confirmed identities of the
approximately 9,940 zombies.

Separate adjacent failure modes

The following should remain conceptually separate unless further evidence links
them:

  • live MCP/helper processes that remain running;
  • live helpers orphaned to launchd with PPID=1;
  • exited children left as zombies under a still-running Codex parent;
  • file-descriptor leakage causing EMFILE;
  • live git process spawn storms caused by watcher/retry loops.

They may share ownership and teardown weaknesses, but they consume different OS
resources and require different direct fixes.

Reproduction and monitoring commands

Compact diagnostic snapshot

Run while the problem is present:

{
  echo "== OS =="
  sw_vers
  uname -a

  echo
  echo "== process limits =="
  launchctl limit maxproc
  sysctl kern.maxproc kern.maxprocperuid

  echo
  echo "== process counts by user =="
  ps -axo user= | sort | uniq -c | sort -nr | head -10

  echo
  echo "== total zombie count =="
  ps -axo stat= \
    | awk '$1 ~ /^Z/ { n++ } END { print n+0 }'

  echo
  echo "== top zombie parents =="
  ps -axo pid=,ppid=,stat=,etime=,command= \
    | awk '$3 ~ /^Z/ { z[$2]++ } END { for (p in z) print z[p], p }' \
    | sort -nr \
    | head -20 \
    | while read count ppid; do
        printf "%6s zombies  PPID=%s  " "$count" "$ppid"
        ps -o pid=,ppid=,stat=,etime=,command= -p "$ppid" 2>/dev/null
      done

  echo
  echo "== Codex-related live processes =="
  ps -axo pid=,ppid=,pgid=,stat=,etime=,command= \
    | awk '/Codex|codex|node_repl|SkyComputerUse/ { print }'
}

Redact the username, host name, home paths, workspace paths, thread IDs, prompt
payloads, and unrelated processes before posting.

One-shot current Codex check

CODEX_PID="$(
  pgrep -f '/Applications/Codex.app/Contents/MacOS/Codex' \
    | head -1
)"

echo "Codex PID: $CODEX_PID"

ps -axo ppid=,stat= \
  | awk -v ppid="$CODEX_PID" \
    '$1 == ppid && $2 ~ /^Z/ { n++ } END {
      print "zombies under Codex:", n+0
    }'

ps -axo stat= \
  | awk '$1 ~ /^Z/ { n++ } END {
      print "total zombies:", n+0
    }'

ps -axo user= \
  | awk -v user="$USER" '$1 == user { n++ } END {
      print "user processes:", n+0
    }'

Long-running recurrence monitor

This version refreshes the PID each iteration, so it survives app updates and
restarts:

while true; do
  CODEX_PID="$(
    pgrep -f '/Applications/Codex.app/Contents/MacOS/Codex' \
      | head -1
  )"

  printf '\n== %s ==\n' "$(date)"
  echo "Codex PID: ${CODEX_PID:-not-running}"

  if [[ -n "$CODEX_PID" ]]; then
    ps -axo ppid=,stat= \
      | awk -v ppid="$CODEX_PID" \
        '$1 == ppid && $2 ~ /^Z/ { n++ } END {
          print "zombies under Codex:", n+0
        }'
  fi

  ps -axo stat= \
    | awk '$1 ~ /^Z/ { n++ } END {
      print "total zombies:", n+0
    }'

  ps -axo user= \
    | awk -v user="$USER" '$1 == user { n++ } END {
      print "user processes:", n+0
    }'

  sleep 60
done

Safer emergency recovery near the process limit

When pipelines can no longer fork, use shell builtins:

kill -TERM <CODEX_PID>

If the process does not exit:

kill -KILL <CODEX_PID>

SIGCHLD was tested in this reproduction and did not recover the zombies.
Raising maxproc would only postpone exhaustion and is not a real fix.

Issue relevance assessment

Directly relevant

#25744: macOS Computer Use/MCP helpers and unreaped zombies

URL: https://github.com/openai/codex/issues/25744

Relevance: Direct and strongest match

Why:

  • same macOS Codex Desktop surface;
  • same main-GUI-owned zombie failure;
  • same repeated Computer Use/MCP/node_repl lifecycle context;
  • same behavior where killing live helpers does not release zombies;
  • same restart workaround.

This reproduction adds:

  • approximately 9,940 zombies rather than approximately 100;
  • direct proof of kern.maxprocperuid exhaustion;
  • shell-wide fork failure;
  • an unsuccessful SIGCHLD recovery experiment;
  • before/after restart counts;
  • a saved main-process sample and redaction analysis.

Posting recommendation: Post the comprehensive overview here. This should
be the canonical location.

#12491: GUI MCP children not reaped after task completion

URL: https://github.com/openai/codex/issues/12491

Relevance: Direct historical precedent

Why:

  • same child-lifecycle/reaping class;
  • macOS 26.3 build 25D125 is explicitly represented in the issue;
  • reports both zombies and persistent MCP/helper subtrees;
  • reports system slowdown and recovery after cleanup.

Important difference:

  • the issue was closed on May 13, 2026 with a maintainer statement that the

original reported bug was fixed;

  • later comments mix several potentially distinct causes.

Posting recommendation: Do not add the full report to this closed issue.
Link to it from the #25744 overview as prior evidence that a similar failure
class existed and may have regressed or remained in another lifecycle path.

Tangentially relevant

#26293: SkyComputerUseClient turn-ended helpers remain as PPID=1 orphans

URL: https://github.com/openai/codex/issues/26293

Relevance: Adjacent and potentially related, but different observed state

Similarity:

  • Codex Desktop on macOS;
  • Computer Use helper lifecycle;
  • child processes that outlive the event or owner;
  • accumulation during high-volume turns.

Difference:

  • #26293 concerns live, sleeping SkyComputerUseClient turn-ended processes

orphaned to launchd with PPID=1;

  • this reproduction concerns already-exited <defunct> children still owned

by the live main Codex GUI process;

  • the saved pre-restart snapshot shows Computer Use clients, but not enough

argv detail to prove they were turn-ended helpers or the zombie source.

Posting recommendation: Do not duplicate the full report. Add a short
cross-link only if a fresh process capture independently confirms lingering
SkyComputerUseClient turn-ended ... processes. Otherwise, mention #26293 in
the canonical #25744 report as an adjacent lifecycle path.

#26327: Windows helper-stack accumulation

URL: https://github.com/openai/codex/issues/26327

Relevance: Cross-platform lifecycle analogy

Similarity:

  • repeated node_repl, MCP, and nested stdio app-server stacks;
  • long-running Desktop session;
  • restart restores a smaller baseline.

Difference:

  • Windows rather than macOS;
  • live helper stacks rather than proven zombies;
  • the author retracted and closed the report because the causal split was not

precise enough.

Posting recommendation: Do not post there. A brief mention in #25744 is
enough if cross-platform context is useful.

#26984: MCP stdio pipe-FD and orphan leakage

URL: https://github.com/openai/codex/issues/26984

Relevance: Related ownership/teardown class, different exhausted resource

Similarity:

  • child cleanup and process-group teardown concerns;
  • long-running Codex sessions;
  • eventual inability to open/spawn.

Difference:

  • primary failure is EMFILE / file-descriptor exhaustion;
  • focuses on CLI-owned stdio MCP servers and npm/npx descendants;
  • this reproduction is EAGAIN-style process-table exhaustion caused by

zombies under the Desktop GUI process;

  • no FD-count evidence was collected here.

Posting recommendation: Do not post the overview there. Link it from
#25744 only as a possible shared lifecycle class. Comment on #26984 only if a
future reproduction also captures monotonically increasing Codex-owned pipe
FDs with lsof.

Likely not relevant

#26904: WSL slow-filesystem git spawn storm

URL: https://github.com/openai/codex/issues/26904

Relevance: Likely unrelated root cause

Why:

  • WSL/Linux and /mnt drvfs trigger;
  • live git processes spawned faster than they finish;
  • watcher ENOSPC and retry/backoff behavior;
  • OOM from active process growth;
  • no shared evidence of unreaped main-GUI child zombies.

The only commonality is the eventual exhaustion caused by unbounded process
lifecycle behavior.

See Also

0xdevalias · 1 month ago

I have now identified one concrete source of Codex-owned zombie bursts and filed the implementation-specific report as #27570:

In a prospective process trace, opening/switching older threads caused the main Codex GUI process to spawn exactly 5,042 direct:

git hash-object --no-filters -- <PATH>

children. The zombie count reached exactly 5,042 before the children were eventually reaped.

Inspection of the installed Desktop bundle identified an unbounded fallback in the review-summary Git worker:

  1. Desktop attempts one bulk git hash-object call.
  2. Three untracked nested Git repositories cause it to fail with fatal: Unable to hash (null).
  3. Desktop retries all 5,042 paths concurrently using an unbounded Promise.all.
  4. The fallback does not receive the outer request's abort signal.

This provides a confirmed Git-based source for transient main-GUI-owned zombies. It may explain some or all of the approximately 9,940 long-lived zombies previously documented here, but that historical connection remains unproven because their executable identities were already lost.

Nicolas0315 · 28 days ago

Adding a current verification snapshot from another macOS Codex.app session (2026-06-22 JST).

Environment:

  • macOS 26.5.1 (25F80), Darwin 25.5.0 arm64
  • Codex.app: 26.616.51431 (build 4212), bundle id com.openai.codex
  • App bundled CLI: codex-cli 0.142.0-alpha.6
  • PATH CLI / managed app-server: codex-cli 0.141.0
  • app-server: running at ~/.codex/app-server-control/app-server-control.sock, parent process /Applications/Codex.app/Contents/Resources/codex app-server --analytics-default-enabled
  • MCP state: computer-use, event-stream, and node_repl enabled; onepassword disabled

Current observation, without restarting or killing Codex:

  • SkyComputerUseClient event-stream mcp: 19 processes, oldest sampled process about 16h old
  • SkyComputerUseClient mcp: 19 processes, oldest sampled process about 16h old
  • node_repl: 21 processes, oldest sampled process about 16h old
  • Most sampled helpers are direct or nested descendants of the same long-lived app-server process.
  • A minimal node_repl reset and JS call currently succeeds, so this is not the immediate sandboxPolicy missing-field failure mode.

Health checks:

  • codex doctor --json initially reported the app-server socket as stale/unreachable with a 2s probe timeout.
  • After codex app-server daemon version, a second doctor run reported app_server.status=ok.
  • Doctor still reports overallStatus=fail only because this noninteractive environment has TERM=dumb.
  • Auth, config, MCP, network, WebSocket, and SQLite integrity checks were OK.

Repro / verification commands used:

defaults read /Applications/Codex.app/Contents/Info CFBundleShortVersionString
defaults read /Applications/Codex.app/Contents/Info CFBundleVersion
/Applications/Codex.app/Contents/Resources/codex --version
codex --version
codex mcp list
codex doctor --json
codex app-server daemon version
ps -axo pid=,ppid=,stat=,etime=,command= | rg 'SkyComputerUseClient|node_repl'

Interpretation:
This looks like the same helper lifecycle / fan-out class as this issue, not a new distinct issue. A useful fix invariant would be that old per-session Computer Use / node_repl helper families stay bounded or are reaped while the long-lived app-server remains alive.

dysthymia · 25 days ago

Adding a fresh sanitized macOS Codex.app crash/stackshot data point that looks like the same lifecycle/resource-leak class, but with a more severe end state: WindowServer was terminated by watchdog after a Codex-owned node process storm.

Environment:

  • Codex.app: 26.616.81150, bundle build 4306
  • Platform: Darwin 25.5.0 arm64 arm
  • macOS: 26.5.1 build 25F80
  • Hardware class: Apple Silicon desktop, 16 GB RAM
  • Raw report not attached because it contains local paths, full process inventory, display/peripheral details, and project/workspace identifiers.

What happened:

  • The user-visible symptom looked like a system restart / black screen / session reset.
  • last reboot showed the machine did not actually reboot; uptime was still about 22 days.
  • The generated report was a WindowServer stackshot / bug type 409.
  • Termination was WATCHDOG, indicator monitoring timed out for service.
  • The specific watchdog detail was:
unresponsive work processor(s): WindowServer main thread
51 seconds since last successful checkin

Most relevant stackshot numbers, redacted and aggregated:

Codex jetsam coalition total RSS: 44.7 GiB
Codex jetsam coalition process count: 1386
node processes in that coalition: 1352
node_repl processes in that coalition: 17
Codex-named processes total RSS: 2.7 GiB
WindowServer RSS: 1.19 GiB

Largest processes in the Codex coalition:

1147.9 MiB  Codex (Renderer)
1091.8 MiB  node
 617.3 MiB  codex app-server
 578.7 MiB  Codex (Service)
 259.2 MiB  Codex

This seems important because the biggest signal was not only Codex renderer memory. The stackshot had more than a thousand node children/related processes grouped into the same Codex jetsam coalition. That matches the helper-process accumulation pattern reported above, but the observed impact reached a system UI watchdog termination.

Other context:

  • Other non-Codex memory users were present, but Codex's coalition was by far the largest by total RSS and process count.
  • The post-incident live system no longer had the same 1386-process state, but still had dozens of Codex-related helper/app-server/repl processes.
  • Memory pressure later recovered, so the stackshot is the useful evidence point.

Expected behavior:

  • Codex should not accumulate hundreds or thousands of stale node, node_repl, or app-server helper processes across a long-running desktop session.
  • A long-running Codex session should not be able to push the graphical session into a WindowServer watchdog reset.
  • Stale helper processes should be bounded and cleaned up when their tool/session/thread is no longer active.

Useful local diagnostic commands for future reports:

ps -axo pid,ppid,rss,etime,command |
  awk 'BEGIN{IGNORECASE=1} /Codex|\/codex|cua_node|node_repl/ {print}'

ps -axo command |
  awk 'BEGIN{IGNORECASE=1} /Codex|\/codex|cua_node|node_repl/ {n++} END{print n+0}'

memory_pressure
last reboot | head

I intentionally omitted raw paths, usernames, project/workspace names, third-party project identifiers, display/peripheral names, and the raw stackshot process inventory.

Nicolas0315 · 23 days ago

Adding a current helper-accumulation datapoint from the latest macOS Desktop build I have installed.

Snapshot time: 2026-06-28 11:26 JST

Environment:

  • Codex Desktop: 26.623.42026, build 4514
  • Codex CLI / app-server: 0.142.3
  • Platform: macOS arm64
  • MCP config: 13 configured servers, 4 disabled, 7 stdio, 6 streamable HTTP
  • Active local surfaces include Desktop plus IDE-extension app-server processes.

Current sanitized helper counts:

chrome_devtools_mcp: 59
playwright_mcp: 30
computer_use_mcp: 41
computer_use_event_stream: 22
node_repl: 22
xcodebuildmcp: 40
codex_app_server matched processes: 20

Selected app-server FD counts from the same snapshot:

standalone codex app-server --listen unix://: 258 lsof entries
VS Code extension app-server: 96 lsof entries
Cursor extension app-server samples: 25, 25, 27 lsof entries
launchd maxfiles soft limit: 256

This looks like the same lifecycle class as this issue, with one extra angle: in a multi-surface setup where Codex Desktop, CLI, VS Code, and Cursor are all present, the shared MCP/tool configuration appears to multiply the same helper families across surfaces. I did not kill or restart anything before collecting these numbers.

Omitting raw process args, full local paths, and private project/tool details here.

clairernovotny · 21 days ago

Adding a narrower source-level repro for the app-server/MCP helper retention path. This is from a current local openai/codex checkout, with local paths sanitized.

The live symptom was not just macOS FD pressure. Once the long-running app-server accumulated repeated MCP/helper runtime groups, new Codex unified exec process creation started failing with:

Bad file descriptor (os error 9)
dup of fd 255 failed
failed to openpty

At the same time, an already-live Node MCP runtime could still spawn /bin/pwd, so the failure was not a global OS inability to fork/exec. It pointed at Codex session/tool runtime lifecycle and unified exec descriptor handling under retained helper pressure.

Concrete repro test

I added an ignored regression test in:

codex-rs/core/tests/suite/mcp_refresh_cleanup.rs

Test name:

shutdown_cleans_superseded_mcp_server_with_in_flight_call

The test does this:

  1. Starts a stdio MCP server and records its PID via MCP_TEST_PID_FILE.
  2. Starts a long in-flight MCP sync call that holds the current MCP runtime.
  3. Refreshes MCP servers, creating a replacement MCP server and making the first runtime superseded.
  4. Calls fixture.codex.shutdown_and_wait().await.
  5. Asserts both the replacement MCP server and the superseded MCP server exit.

Run command:

cd ~/dev/codex/codex-rs
RUST_MIN_STACK=16777216 RUST_BACKTRACE=1 cargo test -p codex-core \
  shutdown_cleans_superseded_mcp_server_with_in_flight_call \
  -- --ignored --nocapture

Observed failure:

test suite::mcp_refresh_cleanup::shutdown_cleans_superseded_mcp_server_with_in_flight_call ... FAILED

Error: superseded MCP process <pid> survived session shutdown while an in-flight tool call held the old runtime: timed out waiting for process to exit

Without RUST_MIN_STACK=16777216, this same scenario can abort earlier with a Tokio worker stack overflow during shutdown, so the larger stack is useful to reach the cleaner process-retention assertion.

Repro patch

<details>
<summary>Patch for the ignored repro test</summary>

diff --git i/codex-rs/core/tests/suite/mcp_refresh_cleanup.rs w/codex-rs/core/tests/suite/mcp_refresh_cleanup.rs
index 620b6c734c..7e12d1e3ec 100644
--- i/codex-rs/core/tests/suite/mcp_refresh_cleanup.rs
+++ w/codex-rs/core/tests/suite/mcp_refresh_cleanup.rs
@@ -133,3 +133,131 @@ async fn refresh_keeps_superseded_mcp_server_alive_for_in_flight_calls() -> anyh
     fixture.codex.shutdown_and_wait().await?;
     wait_for_process_exit(&replacement_pid).await
 }
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+#[ignore = "reproduces shutdown leaking a superseded MCP runtime that is held by an in-flight call"]
+async fn shutdown_cleans_superseded_mcp_server_with_in_flight_call() -> anyhow::Result<()> {
+    skip_if_no_network!(Ok(()));
+
+    let server = responses::start_mock_server().await;
+    let temp_dir = tempfile::tempdir()?;
+    let pid_file = temp_dir.path().join("mcp.pid");
+    let pid_file_for_config = pid_file.clone();
+    let command = stdio_server_bin()?;
+    let fixture = test_codex()
+        .with_config(move |config| {
+            let mut servers = config.mcp_servers.get().clone();
+            servers.insert(
+                "refresh_cleanup".to_string(),
+                McpServerConfig {
+                    auth: Default::default(),
+                    transport: McpServerTransportConfig::Stdio {
+                        command,
+                        args: Vec::new(),
+                        env: Some(HashMap::from([(
+                            "MCP_TEST_PID_FILE".to_string(),
+                            pid_file_for_config.to_string_lossy().into_owned(),
+                        )])),
+                        env_vars: Vec::new(),
+                        cwd: None,
+                    },
+                    environment_id: DEFAULT_MCP_SERVER_ENVIRONMENT_ID.to_string(),
+                    enabled: true,
+                    required: false,
+                    supports_parallel_tool_calls: false,
+                    disabled_reason: None,
+                    startup_timeout_sec: Some(Duration::from_secs(10)),
+                    tool_timeout_sec: None,
+                    default_tools_approval_mode: None,
+                    enabled_tools: None,
+                    disabled_tools: None,
+                    scopes: None,
+                    oauth: None,
+                    oauth_resource: None,
+                    tools: HashMap::new(),
+                },
+            );
+            config
+                .mcp_servers
+                .set(servers)
+                .expect("test MCP servers should accept any configuration");
+        })
+        .build(&server)
+        .await?;
+    wait_for_mcp_server(&fixture.codex, "refresh_cleanup").await?;
+
+    let superseded_pid = wait_for_pid_file(&pid_file).await?;
+    assert!(process_is_alive(&superseded_pid)?);
+
+    let barrier = serde_json::json!({
+        "id": "mcp-refresh-shutdown-cleanup",
+        "participants": 2,
+        "timeout_ms": 1_000
+    });
+    let long_call = tokio::spawn({
+        let codex = Arc::clone(&fixture.codex);
+        let barrier = barrier.clone();
+        async move {
+            codex
+                .call_mcp_tool(
+                    "refresh_cleanup",
+                    "sync",
+                    Some(serde_json::json!({
+                        "barrier": barrier,
+                        "sleep_after_ms": 300_000
+                    })),
+                    /*meta*/ None,
+                )
+                .await
+        }
+    });
+    fixture
+        .codex
+        .call_mcp_tool(
+            "refresh_cleanup",
+            "sync",
+            Some(serde_json::json!({ "barrier": barrier })),
+            /*meta*/ None,
+        )
+        .await?;
+    fs::remove_file(&pid_file)?;
+
+    responses::mount_sse_once(
+        &server,
+        responses::sse(vec![
+            responses::ev_response_created("resp-1"),
+            responses::ev_assistant_message("msg-1", "done"),
+            responses::ev_completed("resp-1"),
+        ]),
+    )
+    .await;
+    fixture
+        .codex
+        .set_openai_form_elicitation_support(/*supported*/ true)
+        .await?;
+    fixture.submit_turn("refresh MCP servers").await?;
+
+    let replacement_pid = wait_for_pid_file(&pid_file).await?;
+    assert_ne!(replacement_pid, superseded_pid);
+    assert!(process_is_alive(&superseded_pid)?);
+    assert!(process_is_alive(&replacement_pid)?);
+
+    fixture.codex.shutdown_and_wait().await?;
+
+    let replacement_exit = wait_for_process_exit(&replacement_pid).await;
+    let superseded_exit = wait_for_process_exit(&superseded_pid).await;
+    if superseded_exit.is_err() {
+        long_call.abort();
+        let _ = long_call.await;
+        let _ = wait_for_process_exit(&superseded_pid).await;
+    } else {
+        let _ = tokio::time::timeout(Duration::from_secs(1), long_call).await;
+    }
+
+    replacement_exit?;
+    superseded_exit.map_err(|err| {
+        anyhow::anyhow!(
+            "superseded MCP process {superseded_pid} survived session shutdown while an in-flight tool call held the old runtime: {err}"
+        )
+    })
+}

</details>

Suspected root cause path

shutdown_session_runtime() currently shuts down:

sess.services
    .latest_mcp_runtime()
    .manager_arc()
    .shutdown()
    .await;

This repro shows that a superseded runtime can remain alive when an in-flight tool call still holds it. The latest runtime is shut down, but the old runtime held by the in-flight call is not reliably cancelled/shut down by session shutdown.

That gives a concrete source for stale stdio MCP/helper processes surviving after the owning session/thread is closed or shut down. In a long-running Desktop/app-server process, repeated occurrences can leave old helper groups and their descriptors around until app-server eventually hits the soft FD boundary and unified exec starts failing.

Useful fix invariant

Session shutdown / thread close should cancel and shut down all MCP runtime snapshots associated with that session, not only the latest published runtime. Superseded runtimes held by in-flight tool calls need an owner-session shutdown/cancellation path so their stdio children cannot survive the session that created them.

clairernovotny · 21 days ago

Posting the repro test source directly, since the previous comment had it inside a collapsed <details> diff block.

File:

codex-rs/core/tests/suite/mcp_refresh_cleanup.rs

Source to append after refresh_keeps_superseded_mcp_server_alive_for_in_flight_calls:

#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
#[ignore = "reproduces shutdown leaking a superseded MCP runtime that is held by an in-flight call"]
async fn shutdown_cleans_superseded_mcp_server_with_in_flight_call() -> anyhow::Result<()> {
    skip_if_no_network!(Ok(()));

    let server = responses::start_mock_server().await;
    let temp_dir = tempfile::tempdir()?;
    let pid_file = temp_dir.path().join("mcp.pid");
    let pid_file_for_config = pid_file.clone();
    let command = stdio_server_bin()?;
    let fixture = test_codex()
        .with_config(move |config| {
            let mut servers = config.mcp_servers.get().clone();
            servers.insert(
                "refresh_cleanup".to_string(),
                McpServerConfig {
                    auth: Default::default(),
                    transport: McpServerTransportConfig::Stdio {
                        command,
                        args: Vec::new(),
                        env: Some(HashMap::from([(
                            "MCP_TEST_PID_FILE".to_string(),
                            pid_file_for_config.to_string_lossy().into_owned(),
                        )])),
                        env_vars: Vec::new(),
                        cwd: None,
                    },
                    environment_id: DEFAULT_MCP_SERVER_ENVIRONMENT_ID.to_string(),
                    enabled: true,
                    required: false,
                    supports_parallel_tool_calls: false,
                    disabled_reason: None,
                    startup_timeout_sec: Some(Duration::from_secs(10)),
                    tool_timeout_sec: None,
                    default_tools_approval_mode: None,
                    enabled_tools: None,
                    disabled_tools: None,
                    scopes: None,
                    oauth: None,
                    oauth_resource: None,
                    tools: HashMap::new(),
                },
            );
            config
                .mcp_servers
                .set(servers)
                .expect("test MCP servers should accept any configuration");
        })
        .build(&server)
        .await?;
    wait_for_mcp_server(&fixture.codex, "refresh_cleanup").await?;

    let superseded_pid = wait_for_pid_file(&pid_file).await?;
    assert!(process_is_alive(&superseded_pid)?);

    let barrier = serde_json::json!({
        "id": "mcp-refresh-shutdown-cleanup",
        "participants": 2,
        "timeout_ms": 1_000
    });
    let long_call = tokio::spawn({
        let codex = Arc::clone(&fixture.codex);
        let barrier = barrier.clone();
        async move {
            codex
                .call_mcp_tool(
                    "refresh_cleanup",
                    "sync",
                    Some(serde_json::json!({
                        "barrier": barrier,
                        "sleep_after_ms": 300_000
                    })),
                    /*meta*/ None,
                )
                .await
        }
    });
    fixture
        .codex
        .call_mcp_tool(
            "refresh_cleanup",
            "sync",
            Some(serde_json::json!({ "barrier": barrier })),
            /*meta*/ None,
        )
        .await?;
    fs::remove_file(&pid_file)?;

    responses::mount_sse_once(
        &server,
        responses::sse(vec![
            responses::ev_response_created("resp-1"),
            responses::ev_assistant_message("msg-1", "done"),
            responses::ev_completed("resp-1"),
        ]),
    )
    .await;
    fixture
        .codex
        .set_openai_form_elicitation_support(/*supported*/ true)
        .await?;
    fixture.submit_turn("refresh MCP servers").await?;

    let replacement_pid = wait_for_pid_file(&pid_file).await?;
    assert_ne!(replacement_pid, superseded_pid);
    assert!(process_is_alive(&superseded_pid)?);
    assert!(process_is_alive(&replacement_pid)?);

    fixture.codex.shutdown_and_wait().await?;

    let replacement_exit = wait_for_process_exit(&replacement_pid).await;
    let superseded_exit = wait_for_process_exit(&superseded_pid).await;
    if superseded_exit.is_err() {
        long_call.abort();
        let _ = long_call.await;
        let _ = wait_for_process_exit(&superseded_pid).await;
    } else {
        let _ = tokio::time::timeout(Duration::from_secs(1), long_call).await;
    }

    replacement_exit?;
    superseded_exit.map_err(|err| {
        anyhow::anyhow!(
            "superseded MCP process {superseded_pid} survived session shutdown while an in-flight tool call held the old runtime: {err}"
        )
    })
}

Run command:

cd ~/dev/codex/codex-rs
RUST_MIN_STACK=16777216 RUST_BACKTRACE=1 cargo test -p codex-core \
  shutdown_cleans_superseded_mcp_server_with_in_flight_call \
  -- --ignored --nocapture

Observed failure:

Error: superseded MCP process <pid> survived session shutdown while an in-flight tool call held the old runtime: timed out waiting for process to exit