[Windows Codex app] Local stdio MCP servers are repeatedly spawned and not reaped within a single task

Open 💬 18 comments Opened Aug 15, 2026 by youngraison
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

What version of the Codex App are you using (From “About Codex” dialog)?

Codex app: 26.810.6296.0

What subscription do you have?

ChatGPT Plus

What platform is your computer?

Microsoft Windows NT 10.0.26200.0 x64

What issue are you seeing?

Within one Codex task, each new turn/tool environment starts another full
set of local tool processes without terminating previous instances.

Observed after restart:

  • qq_mail_mcp: 6 logical MCP instances / 12 Python processes
  • node_repl.exe: 6 instances
  • all instances share the same codex.exe app-server parent
  • creation timestamps increase sequentially within the same task

With cua-driver enabled, each leaked mcp --direct instance also creates a
full-screen Cua.AgentCursorOverlay. At 12-14 instances, the Windows cursor
stutters severely. Restarting Codex clears the processes and restores smooth
cursor movement. Disabling cua-driver prevents those overlay processes but
other stdio MCP instances still accumulate.

What steps can reproduce the bug?

  1. On Windows, configure at least one local stdio MCP server in Codex. In my case, I had:
  • qq_mail_mcp
  • cua-driver using mcp --direct
  1. Fully exit and restart the Codex desktop app so that no old MCP child processes remain.
  1. Open a single Codex task. Do not create additional tasks.
  1. Send several consecutive messages in the same task that require tool-enabled turns, for example asking Codex to run a read-only PowerShell command.
  1. After each turn, inspect the child processes of codex.exe in Task Manager or PowerShell.
  1. Observe that every new turn/tool environment starts another instance of each configured stdio MCP server, while the instances created by previous turns remain running.
  1. After six turns in one task, I observed:
  • 6 logical qq_mail_mcp instances (12 Python processes because each instance has a launcher and runtime child)
  • 6 node_repl.exe instances
  • All instances had the same codex.exe app-server parent.
  1. When cua-driver mcp --direct was enabled, every leaked Cua instance also created a visible full-screen Cua.AgentCursorOverlay covering the dual-monitor virtual desktop.
  1. After approximately 12-14 accumulated Cua instances, the Windows mouse cursor started stuttering severely in normal applications.
  1. Fully exiting and restarting Codex removed all accumulated child processes and immediately restored smooth cursor movement. Disabling the Cua MCP prevented its overlay processes from returning, but other stdio MCP instances continued to accumulate.

What is the expected behavior?

Codex should reuse the existing stdio MCP connection within the same task, or terminate the previous MCP child process when its tool/session context is disposed.

Additional information

_No response_

View original on GitHub ↗

18 Comments

github-actions[bot] contributor · 12 days ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #38526
  • #38614
  • #38693
  • #38537
  • #38714

Powered by Codex Action

najy97 · 12 days ago

macOS reproduction, root cause, and tested patch available

I reproduced the same per-turn stdio MCP accumulation on macOS in the ChatGPT desktop app (bundled Codex app-server, Codex CLI 0.148.0-alpha.9).

Reproduction evidence

Using ps -axo pid,ppid,etime,stat,command, I confirmed that the accumulating processes were direct descendants of the long-lived ChatGPT-owned codex app-server, rather than Hermes, a LaunchAgent, or another watcher/service.

A real session accumulated approximately:

  • 20 instances of a local stdio MCP server
  • 16 bundled node_repl processes

After terminating only those verified app-server-owned children, a few additional tool-enabled turns created new stdio MCP / node_repl pairs that remained alive. A separately parented instance of the same MCP server continued running, confirming that process-name-wide cleanup would be unsafe.

This appears to be the same cross-platform lifecycle problem reported here: a single desktop task repeatedly creates stdio MCP generations instead of converging to a bounded baseline.

Root cause found

I traced two cooperating lifecycle gaps:

  1. Reconciliation did not reuse an unchanged MCP connection while its startup was still pending, so repeated refreshes could spawn duplicate stdio servers.
  2. Stdio shutdown used a one-way terminated flag without shared cleanup completion. If the first shutdown caller was cancelled while waiting through the TERM grace period, a second caller could return immediately. If the app-server/Tokio runtime then exited, the later SIGKILL/exit-confirmation work could be lost. Signal failures were also logged as success, preventing retry.

Superseded connection generations additionally need lease-aware retirement: an in-flight tool call must retain its exact connection, while the process should be cleaned up once the last lease is released.

Reference implementation

A tested reference implementation is available for maintainers to inspect or adapt:

The focused patch:

  • tracks only the spawned child/process group or Windows Job Object as the ownership boundary; it does not scan by PID or process name
  • makes duplicate shutdown callers wait for the same shared cleanup result
  • keeps cleanup running when the initiating caller is cancelled
  • performs SIGTERM -> grace period -> SIGKILL -> exit confirmation on only the owned Unix process group
  • preserves failed cleanup as retryable
  • uses runtime-independent local Drop fallbacks for app-server/runtime exit
  • reuses one unchanged pending connection across repeated reconciliation
  • retires superseded connections only after in-flight leases are released
  • avoids polling/starting a dormant lazy MCP merely to shut it down
  • preserves and directly tests the Windows Job Object/process-handle cleanup paths

<details>
<summary>Implementation map and lifecycle invariants</summary>

  • Spawn ownership and shared cleanup: stdio_server_process.rs
  • StdioServerProcessHandle clones share one retryable cleanup state and target only the process group, Windows Job Object/process handle, or executor handle captured at spawn.
  • State transitions are Idle | Failed -> Running -> Complete | Failed; every concurrent caller joins the same running attempt.
  • Caller cancellation does not own the cleanup task, and dropping the runtime activates the local ownership fallback.
  • Pending reuse and lease-aware retirement: connection_manager.rs
  • An unchanged pending connection is reused instead of spawning a new generation.
  • A superseded connection is absent from the successor publication, so additional strong references are active binding leases; shutdown starts after those leases drain.
  • Publication and final shutdown barrier: runtime.rs
  • Weak references keep retired generations reachable for final thread shutdown without extending lease lifetime.
  • Final shutdown joins current and reachable retired generations, while per-server cleanup runs concurrently.
  • Exact Unix signaling: process_group.rs
  • Signal-0 existence checks and the macOS EPERM member fallback stay scoped to the captured process group.
  • Regression map:
  • pending reuse and cancelled cleanup batch: connection_manager_tests.rs
  • Unix TERM/KILL, retry, runtime exit, and unowned-process preservation: process_group_cleanup.rs
  • in-flight binding across refresh: mcp_refresh_cleanup.rs
  • Windows Job ownership and scenario matrix: stdio_message_limits.rs

Suggested adoption order: spawn-owned process cleanup, pending reuse, lease-aware retirement/final barrier, then platform-specific regression coverage.

</details>

2026-08-20 hard-exit and cleanup-barrier update

The reference branch now includes two additional review-sized commits:

  • 20e4d940d2: process-owned stdio cleanup across runtime and hard parent exits
  • 0a8fb85691: explicit connection leases, retryable retirement, and the final session cleanup barrier

Additional lifecycle hardening:

  • all duplicate shutdown callers now await the same cleanup result
  • cleanup failures remain retryable and failed retired generations stay owned until final shutdown retries them
  • explicit leases replace Arc::strong_count polling for in-flight calls, resources, and event streams
  • session shutdown emits completion only after MCP cleanup succeeds or reports a fatal cleanup error
  • Unix local stdio uses a small supervisor so parent SIGKILL still triggers owned process-group SIGTERM, bounded wait, and SIGKILL
  • remote executor disconnect and detached-session expiry terminate only the captured executor handle
  • Windows local stdio is spawned suspended and must enter the captured kill-on-close Job Object before it is resumed

Current validation on macOS:

  • cleanup integration: 7/7, including initialization failure, timeout, cancellation, in-flight shutdown, duplicate shutdown wait, and parent SIGKILL
  • codex-mcp: 183/183
  • codex-utils-pty: 28/28
  • 20/20 hard-parent-kill supervisor soak with zero test-server or supervisor survivors
  • focused core shutdown, refresh/lease retirement, pending reuse, cleanup retry, app-server status, and remote detached-session expiry tests passed
  • full workspace run executed 15,287 tests; all lifecycle-specific tests passed. The broad run was not fully green because 163 unrelated mock-server/deadline tests failed and 19 timed out under heavy parallel contention; the MCP-adjacent failures were rerun in isolation and passed.

2026-08-21 latest-main rebase and native Windows validation

The branch is now rebased onto upstream daa48072f4 with rmcp 3.1.3 and the new hosted MCP event-streaming path preserved alongside explicit connection leases.

Native Windows validation on exact commit 0a8fb85691:

  • Windows 11 Pro build 26200, x64
  • cargo build -p codex-cli --bin codex: pass
  • new strict suspended-spawn/Job-assignment regression: 1/1 pass
  • codex-utils-pty: 23/23 pass
  • three focused stdio lifecycle tests: 3/3 non-skipped pass
  • codex-rmcp-client: 251/251 pass, 7 skipped
  • codex-mcp: 190/190 pass
  • 20/20 repeated ownership/cleanup runs: exit 0 every time, exact test_stdio_server.exe count 0 -> 0 after every run
  • both Legacy and V20260728 protocol modes covered across success, failure, timeout, and cancellation
  • same-executable unowned sentinel survived every owned Job cleanup
  • ChatGPT PID 9036 and app-server PID 23576, executable paths, CODEX_CLI_PATH Process/User/Machine state, and permanent User/Machine PATH hashes remained unchanged
  • final checkout remained detached at the exact SHA and clean; no Cargo, Rust, nextest, or test-server processes remained

Process enumeration was used only as an after-the-fact baseline assertion. Cleanup itself remained scoped to spawn-captured Job/process ownership.

Validation

The patch has been validated with:

  • 20 consecutive pending-connection refreshes reusing the same connection
  • success, initialization failure, tool failure, timeout, and cancellation cleanup scenarios
  • first shutdown caller cancellation followed by a second caller that waits through forced cleanup
  • cleanup failure followed by a successful retry
  • runtime exit immediately after the second shutdown completes
  • in-flight calls continuing on a superseded connection, followed by retirement cleanup
  • unrelated process groups remaining alive
  • patched macOS app-server validation where 20 repeated tool calls stayed at one ChatGPT-owned stdio MCP server and one node_repl, then both reached zero on app-server shutdown while the separately owned service process survived
  • macOS suites: codex-rmcp-client 231/231, codex-mcp 182/182, codex-utils-pty 22/22, plus the focused core integration test
  • Windows native suites on pre-rebase commit 1e3286d51f: codex-utils-pty 21/21, codex-rmcp-client 226/226, and codex-mcp 183/183
  • Windows focused lifecycle coverage for both stdio protocol modes: normal success, tool failure, timeout, cancellation, root-server exit, and Tokio runtime exit
  • 20/20 Windows lifecycle regression runs; after every run the exact test-server executable count returned to the baseline of zero
  • a same-executable but separately owned Windows sentinel remained alive after every owned Job cleanup
  • rebased cleanly onto 3b45c29062 (main at validation time, including rmcp 3.1.2); the lifecycle-scoped run passed 449/450 tests and the focused core integration test passed, with the sole unrelated OAuth redirect failure reproduced identically on the upstream base
  • repository formatting, scoped Clippy fixes, and git diff --check

The Windows verification used spawned-process handles/Job ownership for cleanup. Exact-path process enumeration was used only as an after-the-fact assertion that the baseline returned to zero.

Related reports include #17832 and #12491.

Under the repository's current contribution policy, external code contributions and pull requests are not accepted, so I will not open one. I am leaving the branch and validation evidence available as a technical reference, and I would be happy to provide additional diagnostics or test a maintainer-provided candidate build on macOS and Windows.

Lalita061985 · 11 days ago

Corroborating on macOS Desktop, with a controlled reproduction and a regression window.

Environment: macOS (Apple Silicon, 36 GB). ChatGPT/Codex Desktop 26.810.4104726.810.52044, embedded codex-cli 0.148.0-alpha.9; ~18 configured MCP servers (mostly stdio). Also tested: signed 26.721.81911 (embedded 0.146.0-alpha.3.1).

Generation leak (this issue): With a single visible task, every recorded task_started/task_complete boundary spawned a complete new stdio MCP generation (~14 processes) with the prior generation retained — 9 generations in ~31 min, timestamps matching boundaries to the second. Incident peak: ~107 complete suites / ~1,500 children / ~41 GB RSS under one app-server, ending in a kernel watchdog reset. The leak also reproduces on the older 0.146.0-alpha.3.1 line (5 generations in ~10 min), so it predates the current release line.

New controlled finding — reaping strands the task (confirms the #28704/#35486 cross-refs): We ran an external reaper to contain memory (SIGTERM stale generations, newest preserved). A same-task/new-task A/B with timestamps: after a sweep, the existing task's tool calls fail in ~6 ms with Transport closed permanently, while a newly created task (fresh generation) passes the same calls. This matches the source: stdio transports are excluded from retry/reconnect (codex-rs/rmcp-client/src/streamable_http_retry.rsPendingTransport::Stdio => false), so any workaround that kills leaked children permanently kills those tasks' tools. Operators are stuck: reap and strand tasks, or don't and hit the watchdog.

Regression window that may help bisecting: embedded 0.147.0-alpha.6.5 ran multi-agent sessions against the same 18-server MCP config for days with no melt. 0.148.0-alpha.9 additionally fails all stdio tool calls even after a fresh cold restart (single generation, direct JSON-RPC initialize/tools/list against the same wrappers passes) — that fresh-start transport failure does not reproduce on 0.146.0-alpha.3.1.

Aggravator: per #37548, the app re-enables Sparkle auto-update on every launch; the working 0.147 host bundle was overwritten in place, and there is no official downgrade path — so recovery from this regression required living with the older, still-leaking line.

Asks: (1) consider inviting @najy97's prepared lifecycle-cleanup branch as a PR — it targets exactly the retained-generation mechanism; (2) stdio reconnect/rebind parity with streamable HTTP, or a documented lifecycle contract; (3) a named candidate build — we have a deterministic 10-boundary qualification harness and will report results back.

VVi3ard · 11 days ago

Дополнение: воспроизводится на stateful Java stdio MCP и ломает состояние workspace.

Окружение:

  • Codex Desktop 26.810.52044, Windows
  • локальный MCP: BSL Language Server 1.0.8
  • команда: java -Xmx8g -jar bsl-language-server-1.0.8-exec.jar -c ../.bsl-language-server.json mcp

После чистого перезапуска Codex был один дочерний Java PID 47376. Один вызов list_workspace_folders создал второй PID 42824; оба остались живы. В предшествующем прогоне накопилось 11 Java-процессов, созданных с 22:52:51 по 22:57:36; все имели общим родителем codex.exe.

Фрагмент C:\Users\volos\.codex\logs_2.sqlite:

app_server.request rpc.method="thread/start"
  -> thread_spawn
  -> session_init
  -> mcp_manager_init
  -> mcp.runtime.refresh:new{server_name=bsl_language_server_aspu}
  -> start_server_task

Для трёх последовательных инициализаций в одном Codex process:

2026-08-17 00:13:33  Java PID 41632
2026-08-17 00:14:12  Java PID 40368
2026-08-17 00:14:36  Java PID 26424

BSL получает от Codex:

Client initialize request - Protocol: 2025-06-18,
Capabilities: ClientCapabilities[
  experimental=null, roots=null, sampling=null,
  elicitation=Elicitation[form=Form[], url=Url[]]
],
Info: Implementation[name=codex-mcp-client, title=Codex, version=0.148.0-alpha.9]

Последствие: register_workspace_folder хранит root только в конкретном экземпляре BSL. При следующем создании процесса список workspace пуст, поэтому stateful MCP-инструменты невозможно использовать надёжно. Тот же JAR в Claude Code работает без этой проблемы.

Ducheved · 11 days ago

I can reproduce this on a newer Windows build of the Codex App, and in my case the accumulation eventually causes severe system-wide stuttering/freezing as system commit approaches the limit.

Environment:

  • Codex App: 26.810.7004.0
  • Windows: 10.0.26200.0 x64
  • Physical RAM: 32 GB
  • Pagefile: 28 GB

I captured a live process/memory snapshot while Windows was already starting to stutter.

The Codex process tree had accumulated exactly 17 copies of several configured local MCP/runtime processes.

Direct children of the Codex process roots included:

  • 51 npx.exe
  • 17 node.exe
  • 17 node_repl.exe
  • 17 python.exe

The repeated groups included Playwright MCP, Chrome DevTools MCP, Context7, filesystem MCP, node_repl, and a local Python MCP server.

The repeated count of exactly 17 instances across multiple configured runtimes is particularly suspicious. This does not look like a single leaking Node heap. It looks like complete MCP/runtime generations are being created repeatedly and retained instead of being reused or reaped.

At the time of the snapshot, the whole system had:

  • 764 processes
  • 418,237 handles
  • 46.04 GiB aggregate process private bytes
  • ~54.3 GiB committed out of a ~59.9 GiB commit limit
  • ~3.8 GiB physical memory available

node.exe system-wide accounted for:

  • 162 processes
  • 12.957 GiB private memory
  • 10.034 GiB working set
  • 37,040 handles

I am not claiming that all 162 node.exe processes belonged to Codex, because other development applications were also running. The direct-child attribution above is from a PID/PPID snapshot and is the portion I can attribute directly to the Codex process roots.

Kernel pool usage was also elevated during the incident:

  • Paged pool: ~1.66 GiB
  • Nonpaged pool: ~1.45 GiB
  • Combined kernel pools: ~3.11 GiB

For comparison, a previous clean post-reboot baseline on the same machine was approximately:

  • Paged pool: ~0.68 GiB
  • Nonpaged pool: ~0.69 GiB

So combined kernel pool usage was roughly 2.3x above that previous baseline.

However, this incident looks different from a previous Codex-related failure I captured on the same machine.

In that earlier incident, the dominant symptom was kernel paged-pool growth:

  • Paged pool reached ~20.6 GiB
  • PoolMon Toke alone reached ~12 GiB
  • SeAt, SeTd, SeDt, and SeTl were also elevated
  • a short process-creation trace showed hundreds of git.exe, powershell.exe, conhost.exe, cmd.exe, and node.exe launches associated with Codex

After reboot in that earlier case:

  • Paged pool returned to ~0.68 GiB
  • Toke returned to approximately 7–8 MiB

The current incident is therefore not dominated by the same extreme kernel Toke growth.

This time, the much larger contributor is accumulated user-space process private commit: approximately 46 GiB of aggregate private bytes with system commit at approximately 54.3 / 59.9 GiB.

The machine becomes progressively less responsive as this state accumulates. By the time of the snapshot, system commit was above 90% and Windows was visibly stuttering/freezing.

This occurred during a long-running Codex workload rather than immediately after startup. The MCP/runtime process population appears to accumulate over time instead of returning to a bounded baseline when previous tool/session contexts are no longer needed.

The process snapshot strongly suggests repeated MCP/runtime generations are involved because multiple independent configured runtimes were present in exactly 17 copies at the same time.

I have preserved sanitized diagnostics from the degraded state, including:

  • PID/PPID process data
  • direct Codex child-process data
  • aggregate process counts, private bytes, working sets, and handles
  • system commit and available-memory counters
  • pagefile statistics
  • PoolMon measurements
  • Codex App version
  • Windows build information

I can provide the sanitized snapshots if they are useful for debugging.

---

ALSO!

As a live workaround, I terminated several old MCP generations using taskkill /PID <pid> /T /F while leaving the main Codex app-server and active Long Horizon task running.

This reduced the system from:

  • 764 processes
  • 418,237 handles
  • 46.04 GiB aggregate private bytes
  • 162 node.exe
  • ~3.8 GiB available RAM

to:

  • 453 processes
  • 358,552 handles
  • 32.19 GiB aggregate private bytes
  • 30 node.exe
  • ~10.6 GiB available RAM

The active Codex Long Horizon task continued running.

This suggests that old MCP generations and their descendant process trees account for a substantial portion of the accumulated memory/process pressure, and that they can be removed independently of the active app-server/task.

tapperwijn89 · 10 days ago

Request for maintainer triage / canonical fix path

This issue now appears to be one of the clearest central reports for the Desktop stdio MCP lifecycle failure also seen in #38825, #38765 and #38877.

The external patch already linked above is unusually actionable: it targets pending-connection reuse, shared shutdown completion, cancellation-safe cleanup and lease-aware retirement, and it has focused validation showing repeated tool calls stay bounded at one MCP server + one node_repl.

Could a Codex maintainer please confirm whether this is aligned with the intended fix direction and, if so, invite the PR or point to the internal/public PR that supersedes it?

For affected Windows users this is now a production-blocking reliability issue rather than a cosmetic leak: related reports include system commit exhaustion, severe stutter/freezes and an OS bugcheck/reboot. A named candidate build would also be useful; I can run a controlled Windows qualification against it and report process-count/commit behavior.

terrydwisely · 10 days ago

Additional current-version reproduction: Python STDIO transports continuously multiply

Reporter: @terrydwisely

This is a same-machine, same-app-server reproduction on Codex Desktop 26.810.7004.0. It adds exact process ancestry, timestamps, memory impact, post-restart behavior, successful live integration tests, stale transport routing, Windows launcher behavior, an absolute-path mitigation, and independent Claude Code verification.

The Outlook and Google Workspace MCP servers must remain enabled. The user is living with the leak until OpenAI ships an application-level lifecycle fix.

---

Windows Codex MCP process leak and duplicate-spawn report

Title

[Windows][Desktop] STDIO MCP servers multiply within one session, survive app-server restart, and bare commands can launch through cmd.exe

Environment

  • Date reproduced: 2026-08-17
  • Computer: DESKTOP-3VKEC62
  • OS: Windows 11
  • Desktop package: OpenAI.Codex_26.810.7004.0_x64
  • Bundled app-server/CLI: 0.148.0-alpha.9
  • Desktop parent: ChatGPT.exe
  • App-server command: codex.exe -c features.code_mode_host=true app-server --analytics-default-enabled
  • Automations: inbox-watchman and refresh-terry-s-morning-briefing both paused

Configured STDIO servers

[mcp_servers.outlook]
command = "C:\\Users\\twisely\\AppData\\Local\\Python\\pythoncore-3.14-64\\python.exe"
args = ["C:\\Users\\twisely\\outlook-mcp-server\\outlook_mcp_server.py"]

[mcp_servers.workspace-mcp]
command = "C:\\Users\\twisely\\AppData\\Local\\Python\\pythoncore-3.14-64\\python.exe"
args = ["-m", "uv", "tool", "run", "workspace-mcp", "--single-user", "--transport", "stdio"]

The commands originally used bare python. They were changed to the verified Python 3.14.2 executable above. The original config was backed up before editing.

Actual behavior

There are separate duplicate-spawn, cleanup, transport-routing, and Windows-launch defects.

1. Duplicate servers inside one app-server lifetime

The current desktop app-server, PID 53644, started at 1:49:59 PM. In its first 25 minutes it produced eight MCP server launch events instead of one Outlook launch and one workspace-mcp launch. Three Outlook launches occurred within four seconds: 1:59:40 PM, 1:59:41 PM, and 1:59:44 PM.

The duplicate behavior reproduced again while this report was being prepared:

  • Existing live pair: Outlook/workspace-mcp started at 1:59:44 PM.
  • Same app-server PID 53644 remained running.
  • A follow-up message in the same task caused another live Outlook/workspace-mcp pair to start at 2:13:37 PM.
  • The 1:59 pair remained alive. The new pair was not a replacement.

This proves the application is creating a fresh server manager/process set on a later turn or steer event instead of reusing or disposing the earlier set.

Post-restart verification at 2:33 PM

The desktop MCP restart did not correct the duplicate-spawn defect:

  • One new app-server, PID 30216, started at 2:33:16 PM.
  • The first Outlook/workspace-mcp pair started at 2:33:18-2:33:19 PM.
  • A second pair started under the same app-server at 2:33:42 PM, only about 24 seconds later.
  • A read-only Outlook folder test and workspace-mcp runtime test both succeeded.
  • Invoking those integrations caused a third pair to start at 2:35:00 PM.
  • The resulting count was three Outlook server instances and three workspace-mcp entrypoints under one app-server.
  • All three launches used the corrected absolute Python path. There were zero target cmd.exe or WindowsApps Python alias launchers.

This is a clean post-restart reproduction: normal application startup, an MCP restart, and ordinary integration use each retained a separate server set instead of converging on one instance per configured server.

Independent verification by Claude Code at 2:39 PM

Claude Code independently inspected the same machine and confirmed:

  • codex.exe PID 30216 started at 2:33:16 PM.
  • By 2:39 PM, it had spawned four Outlook/workspace-mcp transport pairs at 2:33:18 PM, 2:33:42 PM, 2:35:00 PM, and 2:37:52 PM.
  • Those four launches accounted for 24 live processes and approximately 1.17 GB of working-set memory in six minutes.
  • The fourth pair appeared after Codex's own verification pass, proving that the defect reproduces continuously rather than only during application startup or a manual MCP restart.
  • The absolute-path configuration remained effective: zero cmd.exe wrappers and zero WindowsApps Python alias launchers were present.

Current mitigation: living with it until OpenAI fixes it

Both MCP servers must remain enabled because Outlook and Google Workspace are required integrations. The absolute-path configuration is being retained because it removes the alias chain and target console-window launcher. There is no available configuration setting that prevents Codex Desktop from continuously spawning and retaining duplicate transport pairs, so the process leak remains active pending an application fix from OpenAI.

Continued growth while filing the issue

A Codex-side process snapshot at 2:43:39 PM found the same app-server PID 30216 still running with seven Outlook server instances and six workspace-mcp entrypoints. Additional Outlook launches appeared at 2:41:40 PM, 2:43:01 PM, and 2:43:36 PM; additional workspace-mcp launches appeared at 2:41:40 PM and 2:43:03 PM. The newest Outlook process had appeared only three seconds before the snapshot and its corresponding workspace-mcp entrypoint had not yet appeared, showing that startup is neither deduplicated nor serialized as one atomic pair. The bad-launcher count remained zero.

2. Children survive an app-server restart

An earlier app-server PID 11916 started an Outlook and workspace-mcp set at 1:23:30-1:23:35 PM. The desktop restarted its app-server at 1:49:59 PM and replaced it with PID 53644, but all 12 processes in the 1:23 set remained alive with the now-nonexistent PID 11916 recorded as the parent of the two cmd.exe roots.

Relevant desktop log excerpt:

2026-08-17T18:49:59.871Z [AppServerConnection] Restart requested hostId=local intent=restart killCodexProcess=false transportKind=stdio
2026-08-17T18:49:59.871Z [AppServerConnection] Stopping app-server transport connectionId=2 transport=stdio
2026-08-17T18:49:59.873Z [AppServerConnection] Starting app-server connection hostId=local transport=stdio
2026-08-17T18:49:59.897Z [StdioConnection] stdio_transport_spawned ... pid=53644

The stale 1:23 tree was later terminated explicitly: 12 processes, 146.8 MB working set. The stale 1:59 tree was also terminated after the 2:13 duplicate appeared: 8 processes, 332.3 MB working set. No server files or configuration entries were removed.

3. Tool routing remains bound to the stale copies

After the stale 1:59 pair was terminated, the new 2:13 pair remained live and used the corrected absolute commands. However, Outlook and workspace-mcp test calls in the active task both returned:

Transport closed

An automatic retry did not rebind to the already-running 2:13 servers. The application spawned replacement copies but kept the active tool router attached to the older transports.

4. Bare commands use an unnecessarily deep Windows launch chain

With command = "python", the observed workspace-mcp process tree was:

cmd.exe
  WindowsApps Python alias
    real Python 3.14
      uv.exe
        workspace-mcp.exe
          uv environment python.exe
            real Python 3.14

The Outlook server similarly used cmd.exe -> WindowsApps Python alias -> real Python.

5. Console windows can flash

The 1:23 launches used commands such as:

cmd.exe /d /s /c "python ^"C:\Users\twisely\outlook-mcp-server\outlook_mcp_server.py^""

Each cmd.exe had a conhost.exe child. With Windows Terminal configured as the console host, this produced a visible terminal flash.

Configuration-fix verification

An isolated app-server 0.148.0-alpha.9 probe loaded the edited config, started an ephemeral thread, and queried MCP status.

  • Outlook reached ready, server version 1.26.0.
  • workspace-mcp reached ready, server version 3.4.7.
  • Exactly one real Outlook Python process ran.
  • Exactly one workspace-mcp.exe entrypoint ran.
  • Zero cmd.exe processes were used by either server.
  • Zero WindowsApps Python alias processes were used.
  • Zero Windows Terminal or OpenConsole processes were created.
  • Closing the isolated app-server's stdin terminated all six target MCP processes; none remained.

This verifies the absolute-path config workaround. It does not fix the desktop application's duplicate-spawn or stale-routing logic.

Expected behavior

  • Maintain one live instance per configured STDIO MCP server for an app-server/session and reuse it across turns, or deterministically terminate the complete old process tree before starting a replacement.
  • Serialize and deduplicate concurrent MCP initialization so thread/start, turn start, steer, status refresh, and config reload cannot race into duplicate launches.
  • Atomically switch the tool router to a replacement transport before disposing the old transport.
  • On Windows, launch .exe commands directly with redirected stdio and CREATE_NO_WINDOW; do not use cmd.exe for executable commands.
  • Put each server process tree in a Windows Job Object with kill-on-close semantics, or perform equivalent recursive cleanup, so wrappers and grandchildren cannot escape.
  • Resolve/canonicalize bare executable names before spawning, or document that an absolute command is required on Windows.

User impact

  • 174 live MCP-related processes accumulated in 1 hour 47 minutes.
  • Working set reached approximately 6.3 GB.
  • Terminal windows flashed during server startup.
  • Outlook and Google Workspace integrations become unreliable after duplicate cleanup because the tool router remains attached to stale transports.

Suggested attachments

  • This report.
  • A screenshot or CSV from Process Explorer showing PID, parent PID, creation time, executable path, and command line.
  • The relevant app log after reviewing it for sensitive data:
C:\Users\twisely\AppData\Local\Packages\OpenAI.Codex_2p2nqsd0c76g0\LocalCache\Local\Codex\Logs\2026\08\17\codex-desktop-87d9c22d-aa53-48af-9995-9f00c4dc7ac3-39524-t0-i1-143321-0.log
  • The current task/session ID returned by the in-app Feedback flow.

Reporting route

  1. Type / in the Codex composer and choose Feedback. Include this task/session with the report.
  2. Search existing issues at https://github.com/openai/codex/issues.
  3. If no match exists, open a new issue at https://github.com/openai/codex/issues/new/choose and attach the report plus reviewed evidence.

---

Related active reports found before submitting this evidence: #30408, #33946, #35485, #38526, #38614, #38925, and #38981.

twiddlingbits · 9 days ago

Corroborating on Windows Desktop Codex app 26.814.5517.0.

  • Before restart, I found 13 identical Node-based local stdio MCP registrar processes, all direct children of one Codex app-server; the oldest had been alive for about 140 minutes.
  • Fully restarting Codex cleared all Codex-owned copies.
  • In one task after restart, the app-server spawned three identical registrar processes within 44 seconds during tool-enabled turns; previous instances remained alive.
  • Controlled server-side teardown check using the exact registrar bundle: when the client closed stdin, the process exited with code 0 within about 205 ms. This indicates the server honors stdio transport closure and the retention is host-side.
  • Separately parented copies owned by another client were left untouched, confirming cleanup must be parent/transport-scoped rather than process-name-wide.

I originally suspected a project build-directory assemble/teardown race, but that race removes build artifacts during parallel tests and produces missing-file/worker failures; it does not spawn or retain these processes.

CTravkin · 8 days ago

Independent current-build Windows reproduction: 659 descendants / 30.94 GiB working set

Corroborating this on a current Windows build with a continuous live capture.

Environment

  • Codex Desktop package: OpenAI.Codex 26.814.5517.0 x64
  • Bundled Codex CLI/app-server: 0.148.0-alpha.15
  • Windows 11 Pro x64, build 10.0.26200
  • 31.9 GiB physical RAM
  • The user config does not explicitly enable features.code_mode; the tool-enabled environment is initiated by Desktop/runtime.

User-visible impact

After several hours of ordinary Codex work, Windows developed severe system-wide lag. Moving normal desktop windows became difficult and input/UI latency was very high. Codex could not be restarted immediately because two project tasks were still running, which allowed the same app-server lifetime to be measured continuously.

No Codex-owned processes were terminated during this capture.

Continuous growth under one app-server

The same app-server PID remained alive throughout, started at 2026-08-19 16:43:56 local time.

| Local time | Codex descendants | Working set | Private memory | node_repl | node | cmd | python | Handles | Threads |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| 19:27 | 352-358 | 16.5-16.7 GiB | 12.2-12.3 GiB | 15 | 165 | 114 | 30 | 67,383 | 3,318 |
| 19:38 | approximately 510 | 24.0 GiB | 17.31 GiB | 22 | 238 | 167 | 44 | not sampled | not sampled |
| 19:46 | 659 | 30.94 GiB | 22.04 GiB | 30 | 303 | 219 | 58 | 128,480 | 5,701 |

Host memory pressure increased over the same interval:

  • 19:27: 7.91 GiB available; 35.43/44.91 GiB committed (78.9%)
  • 19:46: 2.31 GiB available; 45.64/47.93 GiB committed (95.2%)

Windows appears to have expanded the commit limit during the capture. The application was approaching the host commit limit rather than merely retaining a harmless number of idle processes.

Repeated stdio generations

At 19:46:

  • all 30 node_repl.exe instances were direct children of the same app-server
  • their creation times ranged from 16:44:30 through 19:46:02, across 17 distinct start minutes
  • 29 serena.exe entrypoints and 58 associated Python processes remained live
  • 29 Playwright MCP Node processes remained live
  • shadcn, Mantine, and Tabler each had 58 matching Node processes, consistent with retained launcher/server pairs across approximately 29 generations

This is not a process-name attribution from the whole machine. Counts were taken only from the recursively enumerated descendant tree of the verified Codex app-server.

Shutdown evidence

The active local app-server log contains successful terminal tool outcomes followed by logical session shutdown records, including:

  • dispatch_tool_call_with_terminal_outcome
  • op.dispatch.shutdown
  • Shutting down Codex instance

In a bounded recent log window there were 80 explicit Shutting down Codex instance records, while the corresponding stdio generations remained in the live app-server process tree.

This independently supports the lifecycle diagnosis in this issue: logical session/executor shutdown is occurring, but the owned stdio MCP process generations are not converging to a bounded pool and are not being reaped.

Saturation checks

A separate 15-second sample during the same reproduction showed:

  • aggregate Codex CPU: 5.21% average, 15.37% maximum
  • system CPU: 25% average, 64% maximum (the diagnostic PowerShell process was the top sampled CPU consumer)
  • DPC CPU: 0.33% average, 1% maximum
  • interrupt CPU: 0.39% average, 2% maximum
  • Codex GPU: 0%
  • DWM GPU: 2.06% average, 6% maximum
  • disk queue: 0 in the point-in-time snapshots

The immediate degradation therefore aligns much more strongly with retained process/memory growth than with sustained CPU, DPC, GPU, or disk saturation.

Expected behavior

Completed tool/session contexts should either reuse an unchanged MCP connection or deterministically reap their owned Windows process tree after the final lease is released. The process count and committed memory must converge to a bounded baseline during a long-running Desktop session.

This also provides a concrete mechanism for the Windows-wide lag previously reported in #29187 and is consistent with the retained MCP batches reported in #32797.

Raw task transcripts, complete databases, environment values, and private paths are intentionally omitted. Sanitized aggregate CSV samples and a bounded log summary can be provided if maintainers request them.

naipi11 · 8 days ago

A temporary containment method on Windows is:

  1. Finish or pause active Codex turns, then disable any local stdio MCP servers that are not needed for the current task.
  2. Fully exit the ChatGPT/Codex desktop app (including the tray process), rather than only closing its window.
  3. Confirm the Codex-owned Node/Python/Java child processes have exited, then relaunch the app.
  4. Enable only one required stdio server at a time and watch its child-process count after switching chats. If duplicate generations keep accumulating, fully exit and relaunch before memory/commit pressure becomes high.

Limitation: this interrupts active tasks and only contains the leak. Avoid killing processes globally by executable name, because the same MCP runtime may belong to another application.

szguicheng · 7 days ago

macOS reproduction with turn-count, process-tree, swap, and restart A/B evidence

I reproduced the same unbounded per-turn stdio MCP generation growth on macOS, including system-wide swap exhaustion and an OOM incident.

Environment

  • macOS 26.5.2 (25F84), Apple Silicon
  • ChatGPT/Codex Desktop 26.814.41407 (build 6720)
  • Bundled codex-cli 0.148.0-alpha.15
  • Configured local stdio MCPs: Context7 and mcp-fetch
  • Plugin-provided stdio MCP: claude-mem

Process evidence before a full Desktop restart

The long-lived Desktop app-server owned 526 direct child processes. Exact live instance counts included:

  • claude-mem: 106 launcher wrappers / 104 mcp-server.cjs processes
  • Context7: 105 npm exec parents / 105 server children
  • mcp-fetch: 105 npm exec parents / 105 server children
  • total system process count: 1,589

These were live sleeping processes retained under the same app-server lifetime, not Unix zombies.

The corresponding Desktop log window contained:

  • turn/start: 99
  • thread/start: 7
  • thread/resume: 25
  • unique conversation IDs across start/resume: 12
  • MCP/process teardown markers (shutdown, close, exit, SIGTERM): 0

The near 1:1 match between 99 turn/start calls and approximately 105 retained Context7/fetch generations supports the per-turn lifecycle correlation reported here; this was not 105 concurrently active user tasks.

System impact

At peak:

  • swap: 27,018 MiB used / 27,648 MiB total
  • macOS memory pressure level: 2
  • over a 10-second sample: 67.6 MiB swap-in and 320.1 MiB swap-out
  • the machine had already experienced an OOM event

The claude-mem Bun worker itself was idle at approximately 100 MiB physical footprint with queue depth 0, so the current pressure was not a Bun work queue or model-request backlog.

Restart A/B

The only cleanup intervention was fully quitting and reopening the Desktop app.

Immediately after restart:

  • the old app-server and approximately 100 retained MCP generations disappeared
  • the new app-server started only 3–4 current MCP generations
  • swap allocation shrank from 27,648 MiB total to 9,216 MiB total
  • used swap fell to approximately 8,145 MiB
  • memory pressure returned from level 2 to level 1

This is a current-build macOS reproduction affecting both user-configured stdio MCP servers and a plugin-provided MCP server. It also shows that the retained generations remain owned by the app-server and are released when that owner exits.

I can provide sanitized process-count commands and log-correlation commands if useful for a maintainer-provided candidate build.

KimDani87 · 7 days ago

macOS Apple Silicon corroboration on current builds:

  • ChatGPT bundled Codex: 0.149.0-alpha.4
  • Standalone daemon/CLI: 0.149.0
  • During repeated task/thread activity, one ChatGPT code-mode host plus one standalone remote-control host accumulated 182 MCP-related processes, about 8.5 GiB aggregate RSS.
  • Under the long-lived standalone parent, direct stdio wrapper generations reached roughly 31, 31, 27, and 5 for four configured server types. The ChatGPT host also retained multiple generations.
  • The stdio proxies exit normally when stdin is closed in a direct harness, so the retained transports appear host-side.
  • Terminating only stale child PIDs made the active task return Transport closed; the task did not respawn or rebind the transport.
  • A full host restart cleared the accumulated children. The standalone daemon restart itself timed out because the managed app-server remained defunct under the pid-update-loop; terminating that exact supervisor PID and running daemon start recovered it.
  • After recovery, two repeated calls to the same two MCP servers did not increase counts within the same active turn. However, the fresh ChatGPT host already had 2 to 3 direct generations per configured stdio server, consistent with per-thread or per-task stacks remaining alive.

Local mitigation only reduced blast radius: disabled a stale duplicate registration, scoped an unrelated server out of the project, moved a credential to a mode-600 file-backed source, and performed exact-PID cleanup. It does not fix lifecycle ownership in Codex.

I can provide a sanitized process-tree capture or test a candidate build if useful.

yotamleo · 6 days ago

Same defect reproduced on the CLI (not the desktop app), with a per-subagent trigger and a controlled census. Filing the detail here rather than keeping a separate issue open — full write-up was #39982.

Environment: Codex CLI 0.149.0, standalone install, windows-x86_64, Windows 11 26200. 4 configured stdio MCP servers (1 disabled). Auth ChatGPT, model gpt-5.6-sol.

What is different from the report above: your accumulation is per turn within one task. Ours is per subagent, and the server involved starts lazily (only on first tool use), so this is not eager startup — the server is spawned fresh for a subagent that actually calls it, as a new direct child of the same live codex.exe, instead of attaching to the instance already running under that supervisor.

Controlled run (two subagents, one lazily-started stdio server):

  • each of the two subagents caused a new direct child of the same live supervisor;
  • both children were still running after both subagents had ended — the supervisor stayed live, so nothing closed the connections;
  • the duplicate count under that supervisor went from 10 to 12 and stayed there.

It is not one misbehaving server. After that server was removed from the config entirely, the same live supervisor still held duplicate fleets of four unrelated stdio servers (a Node REPL runtime, uvx mcp-obsidian, and two bun-launched plugin-cache servers). An earlier snapshot on the same machine had one live supervisor holding 5 concurrent instances of a single server and a second holding 3 more. Any stdio server reachable from more than one subagent accumulates the same way.

Relationship to the existing fix: #19753 ("Terminate stdio MCP servers on shutdown") is merged, and it does address shutdown. It does not cover this case, because the supervisor never shuts down — it stays live for the whole session while duplicates pile up underneath it. #38925 makes the same observation.

Why this cannot be cleaned up from outside the client: the supervisor is live the whole time, so nothing in the process tree distinguishes a redundant duplicate from a fleet in active use. There is no lease, no connection id, and no terminal state to read, so terminating a duplicate is a guess that can tear down a server another subagent is mid-call on. Our own cleanup tooling reclaims only fleets whose supervisor is gone; for duplicates under a live supervisor it is deliberately report-only — it counts them and refuses to act.

Ask (either one closes the class): reuse one fleet per active client/project so a second subagent attaches to the running server, or close stdin/the child when the owning client/subagent ends so the server sees EOF. Independently useful: expose a stable terminal state for a connection or lease, so external tooling can act on facts instead of heuristics.

Closely related, same root cause from different surfaces: #38693 (subagent-scoped trees, Desktop), #38925 (accumulation under a live app-server), #37870 (CLI, completed subagents), #38353 (proposes the pooling/teardown design).

yotamleo · 6 days ago

Source-level root cause, condensed here since our thread was folded into this one. Full write-up with the complete chain: https://github.com/openai/codex/issues/39982#issuecomment-5373515471 — read against main @ 51ebf5b1842d44a8e2c955e8b5cd2a589d41e71e.

Short version: this isn't a missing teardown. Teardown is correct where it runs — it just never runs for a subagent that merely finishes.

Per-thread fleets. Every thread, root or subagent, owns its own Session.services.mcp_runtime (codex-rs/core/src/session/mcp_runtime.rs) → its own McpConnectionManager → its own stdio children. Subagents do not share the parent's.

Only one reap path. Children are terminated only by mcp_runtime.shutdown() in shutdown_session_runtime (codex-rs/core/src/session/handlers.rs:397, call at :417), reachable only via Op::Shutdown. For a subagent that arrives from exactly two places:

  • explicit close_agentshutdown_live_agent (codex-rs/core/src/agent/control/legacy.rs:8, :48)
  • residency eviction (codex-rs/core/src/agent/control/residency.rs:117)

A subagent that simply reaches Completed hits neither. It stays resident on purpose so the parent can wait / resume / send_input, and its whole MCP fleet stays alive with it.

Why the count climbs. Eviction is spawn-triggered only (residency.rs:49) — nothing proactively reaps a completed resident. Capacity is max_concurrent_threads_per_session - 1, default 4 → 3 (codex-rs/core/src/config/mod.rs:224, effective_agent_max_threads at :1507). Steady state ≈ (live root threads) × (1 + up to 3) fleets. Our controlled two-subagent pilot took a process census 10 → 12 with nothing reaped on completion, and removing one server left other duplicate fleets standing — the duplication is per-thread, not per-server.

Second-order. impl Drop for McpServerConnection (codex-rs/codex-mcp/src/connection_manager.rs:148) only calls cancel_token.cancel(); it never awaits shutdown(). Any drop not preceded by an explicit shutdown is therefore not a guaranteed reap — relevant on Windows, where an orphaned child isn't cleaned up by process-group semantics.

Suggested fix (smallest defensible). Split "resident so history/resume work" from "holds live OS processes": when a subagent reaches a terminal state — the predicate already exists as is_unloadable (residency.rs:233: Completed/Errored/Interrupted, no active turn, no pending mailbox items) — call mcp_runtime.shutdown() and mark the runtime dormant, re-publishing on the next turn via the existing publish_mcp_runtime / request_mcp_runtime_refresh path. Resume, history and transcript reads never touch the MCP runtime, so nothing user-visible regresses, and ensure_v2_agent_loaded already rebuilds fully-evicted threads — a dormant-runtime resume is strictly smaller than what resume handles today.

Workaround for anyone hitting this now: set agents.max_concurrent_threads_per_session = 1 to shrink the resident cache to zero (costs subagent concurrency), or have the parent call close_agent on each subagent when done — that path does reap immediately.

yotamleo · 4 days ago

Following up on my earlier root-cause comment above with a concrete patch. docs/contributing.md says external PRs are not accepted, so this is a comment rather than a pull request — the branch is linked at the bottom if it is easier to read there.

Read against main @ ad32eba832.

Recap of the cause

Completed subagent threads are deliberately kept resident, and each resident thread owns a full MCP fleet. shutdown_session_runtime reaps stdio children correctly, but it is only reached via Op::Shutdown — sent from close_agent and from residency eviction, neither of which runs when a subagent simply finishes. So the steady state per root thread is up to max_concurrent_threads_per_session - 1 completed-but-resident subagents, each holding a duplicate fleet.

The two properties conflated are "the thread stays resident so history/resume work" and "the thread holds live OS processes". The patch below separates them.

The change

Three files, ~74 lines.

1. codex-rs/codex-mcp/src/runtime.rs — a way to retire the published generation without shutting it down. Factored PublishedMcpRuntime::empty out of McpRuntime::empty and added:

    /// Retires the published generation without disturbing calls already using it.
    ///
    /// New callers observe an empty runtime, while a call that already captured the
    /// previous [`McpConnectionSet`] keeps it until the call returns — the same
    /// lifetime a refresh grants a superseded generation. Its stdio children exit
    /// once that last reference drops. Prefer this over [`Self::shutdown`] when the
    /// owning thread outlives its servers, so a live thread stops holding processes
    /// it no longer needs.
    pub fn retire(&self) {
        let prefix_mcp_tool_names = self
            .current
            .load()
            .config
            .as_ref()
            .is_some_and(|config| config.prefix_mcp_tool_names);
        self.current
            .store(Arc::new(PublishedMcpRuntime::empty(prefix_mcp_tool_names)));
        self.hosted_event_server_removals.send_replace(());
    }

Retiring rather than calling shutdown() matters: an out-of-band call_mcp_tool can be running when a turn ends, and it holds the Arc<McpConnectionSet> it captured. Retiring gives it exactly the lifetime replace() already gives a superseded generation — which is what refresh_keeps_superseded_mcp_server_alive_for_in_flight_calls pins down. The stdio children then exit when that last reference drops, through the kill_on_drop(true) and the Windows job object already set up in stdio_server_launcher.rs, so descendants go too.

2. codex-rs/core/src/session/mcp.rs — the policy, next to mark_mcp_runtime_dirty:

    pub(crate) async fn release_idle_subagent_mcp_runtime(self: &Arc<Self>) {
        if !matches!(
            self.state.lock().await.session_configuration.session_source,
            SessionSource::SubAgent(_)
        ) {
            return;
        }
        // Hold the refresh gate so this cannot race a publish already in flight.
        let Ok(_refresh) = self.mcp_refresh.acquire().await else {
            return;
        };
        // Re-check under the gate: a queued turn may have started while we waited.
        if self.active_turn.lock().await.is_some()
            || self.input_queue.has_pending_mailbox_items().await
        {
            return;
        }
        self.services.mcp_runtime.retire();
        self.mark_mcp_runtime_dirty();
    }

The guard mirrors the is_unloadable predicate already in residency.rs. mark_mcp_runtime_dirty (not request_mcp_runtime_refresh) is deliberate: it must not schedule a prewarm, or the fleet respawns immediately. The next refresh_mcp_if_dirty — which every tool-call path and turn start already goes through — republishes.

3. codex-rs/core/src/tasks/mod.rs — one call site, at the end of on_task_finished:

        if cleared_active_turn {
            self.maybe_start_turn_for_pending_work().await;
            // Runs after the line above so a queued turn keeps its fleet.
            self.release_idle_subagent_mcp_runtime().await;
        }

Ordering is load-bearing: if queued work started a new turn, active_turn is set again and the release is skipped.

Root threads are untouched, McpStartupPolicy::LazyWhenCached still applies to subagents, and disabled servers are unaffected because nothing about the projection changes — the next publish is the same publish that would have happened anyway.

Tests

Four cases added to codex-rs/core/tests/suite/mcp_refresh_cleanup.rs, in the style of the existing test there (real stdio server, MCP_TEST_PID_FILE, process-liveness assertions):

  • a completed subagent's stdio server exits while the thread stays loaded;
  • that same thread republishes a working fleet on its next use (new pid), so resume is unaffected;
  • an in-flight call_mcp_tool still holds its connection when the turn completes, and the process exits only after that call is released;
  • a root thread's fleet survives its own turn completing.

What I ran

cargo clippy -p codex-core -p codex-mcp --tests -- -D warnings is clean. The suite is #[cfg(unix)], so the lifecycle tests ran on Linux even though the original report was Windows 11 / CLI 0.149.0:

running 4 tests
test suite::mcp_refresh_cleanup::completed_root_turn_keeps_its_stdio_mcp_servers ... ok
test suite::mcp_refresh_cleanup::completed_subagent_keeps_in_flight_mcp_calls_alive ... ok
test suite::mcp_refresh_cleanup::completed_subagent_releases_its_stdio_mcp_servers ... ok
test suite::mcp_refresh_cleanup::refresh_keeps_superseded_mcp_server_alive_for_in_flight_calls ... ok

test result: ok. 4 passed; 0 failed

The last one is your existing test, included as a control that the retire path does not change superseded-generation lifetime.

Branch, if a diff is easier to read than a comment: https://github.com/yotamleo/codex/tree/fix/mcp-runtime-release-on-terminal-subagent — no PR opened, per the contributing policy. Happy to re-measure the process census against a build if that would help.

One thing I did not change

impl Drop for McpServerConnection still only calls cancel_token.cancel() and does not await shutdown(). That is fine for this path because the transport is kill_on_drop, but it does mean any other drop-without-shutdown path depends on that rather than on an explicit reap.

grtninja · 3 days ago

Windows corroboration: process/thread/handle exhaustion reaches Win32 1816

I have a separate Windows 11 long-running Codex reproduction that appears to extend this same lifecycle family from duplicate stdio/MCP children into eventual process-creation failure.

Sanitized live snapshot before cleanup

  • 498 total processes
  • 8,304 total threads
  • 239,618 total open handles
  • large duplicate populations of node, node_repl, Python bridge/MCP helpers, and Codex-owned/helper descendants
  • subsequent child/process creation repeatedly failed with Win32 error 1816: Not enough quota is available to process this command

Those aggregate counts are system-wide; I am not claiming every process/handle is Codex-owned. The current cleanup pass is deliberately tracing PID, PPID, executable, creation time, command fingerprint, and owning task/lease before terminating anything.

The user-visible pattern is consistent with this issue: long-running Codex work creates helper generations faster than terminal task/turn cleanup returns them to baseline. Restart/cleanup temporarily restores process creation.

Current-main code audit

I also checked current public openai/codex source rather than relying only on the observed process tree.

codex-rs/rmcp-client/src/stdio_server_launcher.rs now has useful Windows Job Object/process-handle containment, but StdioServerProcessHandle still uses a single terminated: AtomicBool: the first local caller marks termination complete, signals the Job/process, and returns without that method waiting for a verified tree exit/reap; later callers return immediately once the flag is set. That still leaves a cancellation/concurrent-shutdown correctness gap of the kind described in the reference implementation already posted above.

Separately, codex-rs/codex-mcp/src/runtime.rs::replace_fresh() publishes with previous: None, and the current session MCP refresh path calls replace_fresh(). That means hard refreshes intentionally forgo connection reuse, making correct retirement/reaping of old generations especially important.

Additional acceptance criterion suggested by the Windows failure

Please consider a Windows regression that repeatedly performs tool-enabled turns/subagents and asserts all of the following return to a bounded baseline after their owners reach terminal state:

  • exact spawn-owned child/Job processes
  • stdio pipe/process handles
  • node_repl/MCP generations
  • listeners owned by those helpers
  • ability to perform a fresh CreateProcess without Win32 1816

Cleanup should remain spawn/lease-owned; process-name-wide cleanup would be unsafe because unrelated same-name Node/Python processes can be active at the same time.

I am preserving the detailed local process census privately and can add further sanitized before/after measurements once the ownership-bound cleanup run completes.

grtninja · 3 days ago

Follow-up after current-main history review: this reproduces after the existing Windows process-tree fixes

I checked the relevant landed history so this report does not ask for work Codex already has:

  • 9daa491f7c27a5513fec554473a7122d88fca367 / #37366 (2026-08-07): Harden local MCP server process tree cleanup — Windows non-breakaway Job Objects, process-handle fallback, descendant cleanup tests.
  • 8751fd3fcb8031d42b62670a6131872074635c9b / #29608: explicitly shut down superseded MCP managers on refresh.
  • 4e0cf945b7f43f1f9c1d09faadac6def76bdefbf / #19753: explicit stdio MCP shutdown/draining on session shutdown/refresh.
  • 82c981cafc57dfed8383e72c6bbf6082622a3b4a / #10710: original process-group cleanup.

So the Aug 24 Windows exhaustion reproduction should be treated as a remaining lifecycle-generation/retirement/postcondition defect, not simply “Codex lacks process-tree cleanup.”

The remaining current-main concerns are:

  1. StdioServerProcessHandleInner still collapses shutdown state into one terminated: AtomicBool; the first caller signals termination and returns, while later callers can treat terminated=true as terminal without sharing/waiting for a verified cleanup result.
  2. Fresh MCP generations are still created by McpRuntime::replace_fresh() with previous: None on the current session refresh path, so any missing retirement edge has multiplicative impact.
  3. The new Windows Job Object fixes cover owned descendant termination, but the product-level acceptance gate still needs to prove all superseded/task-owned connection generations are retired and reaped before their owner is considered terminal, including subagents and repeated turns.
  4. The separate Windows resource reports (#33356 sandbox/lsass handles, #33776 cleanup taskkill/conhost storms) can compound the same eventual CreateProcess/Win32-1816 symptom even if stdio MCP tree containment itself is working.

The useful regression therefore is not merely “grandchild dies when handle drops.” It is a long-lived app-server task/subagent sequence that measures active MCP generations + process/handle/listener counts and requires them to return to a bounded baseline after each owner reaches terminal state, then verifies a fresh CreateProcess still succeeds.

I’ll add exact before/after ownership-bound Windows cleanup measurements once the local census finishes.

grtninja · 3 days ago

Additional finding: Win32 1816 should be treated as an operation failure requiring attribution, not a machine-capacity diagnosis

A recent Windows reproduction initially surfaced Win32 1816 during a Codex-managed process launch. Further investigation showed that the machine remained capable of launching substantial workloads; the failure was specific to the Codex execution path and required tracing the actual failing operation.

Recommended reliability behavior:

  • Do not convert Win32 1816 directly into a global "machine cannot continue" state.
  • Capture the exact failed operation:
  • executable
  • parent/task identity
  • owner/generation state
  • creation path
  • retry result
  • cleanup result
  • Distinguish:
  • true system resource exhaustion
  • stale Codex-owned lifecycle state
  • helper/runtime compatibility failures
  • transient launch failures

A concrete local recovery path required fixing the failing component rather than relying on small targeted cleanup events. Cleanup should be considered successful only when the original capability is restored or ownership state is proven healthy.

Suggested acceptance criterion:

process launch failure -> ownership attribution -> targeted repair -> successful retry

rather than:

process launch failure -> Win32 error -> assume environment blocker.