The codex vscode plugin fails to start ace-tool mcp, causing an process/thread storm and eventually causing the physical machine to freeze

Open 💬 14 comments Opened Feb 4, 2026 by NormanMises

💡 Likely answer: A maintainer (etraut-openai, contributor) responded on this thread — see the highlighted reply below.

What version of the IDE extension are you using?

0.4.69

What subscription do you have?

api key

Which IDE are you using?

VS Code

What platform is your computer?

Windows connect to remote ssh docker container Linux 5.15.0-60-generic x86_64 x86_64

What issue are you seeing?

When I configure ace-tool or ace-tool-rs or auggie mcp in codex, when I open the vscode codex plugin, it gets stuck, and then I see that the number of process threads in the physical machine is growing wildly, and I know that the physical machine is stuck. Looking at the process, I found that there are many npx ace-tool processes

cat /sys/fs/cgroup/pids.current shows

What steps can reproduce the bug?

Configure ace-tool or ace-tool-rs or auggie mcp in codex
Open the vscode codex plugin

What is the expected behavior?

Hopefully MCP works fine and doesn't cause a process/thread storm

Additional information

some log

2026-01-30T02:03:44.211037Z  INFO codex_core::codex::handlers: Shutting down Codex instance
2026-01-30T02:03:44.223477Z  INFO codex_core::shell_snapshot: Shell snapshot successfully created: /root/.codex/shell_snapshots/019bff50-ba39-7191-a81e-6f7b45ab289b.sh
2026-01-30T02:05:57.936375Z  INFO codex_core::stream_events_utils: ToolCall: write_stdin {"session_id":17607,"chars":"\^C","yield_time_ms":1000,"max_output_tokens":2000}
2026-01-30T02:06:07.445399Z  INFO codex_core::stream_events_utils: ToolCall: exec_command {"cmd":"cd vlfm && echo 'time:' && date && echo 'loadavg:' && cat /proc/loadavg && echo 'cgroup pids.current/max:' && cat /sys/fs/cgroup/pids.current 2>/dev/null && cat /sys/fs/cgroup/pids.max 2>/dev/null && echo 'npx count:' && (pgrep -x npx | wc -l) && echo 'ace-tool-rs count:' && (pgrep -f 'ace-tool-rs' | wc -l) && echo 'process count:' && (ps -e --no-headers | wc -l) && echo 'top 10 by threads (process-level):' && ps -eo pid,nlwp,comm --sort=-nlwp | head -n 11","yield_time_ms":100000,"max_output_tokens":4000}
2026-01-30T02:06:45.618165Z  INFO codex_core::stream_events_utils: ToolCall: exec_command {"cmd":"cd vlfm && echo 'sample npx:' && ps -C npx -o pid,ppid,stat,etime,args --no-headers | head -n 30 | sed -E 's/(--token) [^ ]+/\\1 <REDACTED>/g'","yield_time_ms":100000,"max_output_tokens":4000}
2026-01-30T02:07:17.562698Z  INFO codex_core::stream_events_utils: ToolCall: exec_command {"cmd":"cd vlfm && p=943; echo \"pid=$p\"; ps -p $p -o pid,ppid,stat,etime,cmd --no-headers | sed -E 's/(--token) [^ ]+/\\1 <REDACTED>/g'; pp=$(ps -p $p -o ppid= | tr -d ' '); echo \"ppid=$pp\"; ps -p $pp -o pid,ppid,stat,etime,cmd --no-headers 2>/dev/null || echo 'ppid not found';","yield_time_ms":100000,"max_output_tokens":2000}
2026-01-30T02:07:40.392375Z  WARN codex_core::codex: stream disconnected - retrying sampling request (1/5 in 203ms)...
2026-01-30T02:07:56.065453Z  INFO codex_core::stream_events_utils: ToolCall: exec_command {"cmd":"cd vlfm && echo 'top npx PPIDs:' && ps -C npx -o ppid= --no-headers | sort | uniq -c | sort -nr | head -n 20","yield_time_ms":100000,"max_output_tokens":2000}
2026-01-30T02:08:25.418404Z  INFO codex_core::stream_events_utils: ToolCall: exec_command {"cmd":"cd vlfm && set -euo pipefail\n\nredact(){ sed -E 's/(--token) [^ ]+/\\1 <REDACTED>/g'; }\n\nsnapshot(){\n  echo '---'\n  date\n  echo -n 'pids.current/max: '\n  cat /sys/fs/cgroup/pids.current 2>/dev/null || true\n  echo -n '/ '\n  cat /sys/fs/cgroup/pids.max 2>/dev/null || true\n  echo -n 'npx total: ' && (pgrep -x npx | wc -l || true)\n  echo -n 'npx ace-tool: ' && (pgrep -af 'npx ace-tool' | redact | wc -l || true)\n  echo -n 'ace-tool-rs: ' && (pgrep -af 'ace-tool-rs' | redact | wc -l || true)\n}\n\necho '[before]'\nsnapshot\n\necho '\\n[kill TERM: only ace-tool-related]'\n# 仅杀 ace-tool 相关，避免误伤其他 npx\npkill -TERM -f 'npx ace-tool' || true\npkill -TERM -f 'ace-tool-rs' || true\nsleep 3\n\necho '\\n[mid]'\nsnapshot\n\necho '\\n[kill KILL: leftovers]'\nif pgrep -f 'npx ace-tool' >/dev/null 2>&1 || pgrep -f 'ace-tool-rs' >/dev/null 2>&1; then\n  pkill -KILL -f 'npx ace-tool' || true\n  pkill -KILL -f 'ace-tool-rs' || true\nfi\nsleep 2\n\necho '\\n[after]'\nsnapshot\n","yield_time_ms":100000,"max_output_tokens":4000}
2026-01-30T02:09:42.297560Z  INFO codex_core::stream_events_utils: ToolCall: write_stdin {"session_id":55350,"yield_time_ms":100000,"max_output_tokens":4000}

View original on GitHub ↗

14 Comments

NormanMises · 5 months ago

the issue in ace-tool was https://github.com/eastxiaodong/ace-tool/issues/9

etraut-openai contributor · 5 months ago

If you (or other Codex community members who see this bug report) are so inclined, you might try reproducing this with the Codex CLI. The extension is built on the CLI, and the CLI code is open-sourced, so you (with the help of Codex) can debug and analyze the problem to see if it's in Codex or the MCP server.

NormanMises · 5 months ago

If you (or other Codex community members who see this bug report) are so inclined, you might try reproducing this with the Codex CLI. The extension is built on the CLI, and the CLI code is open-sourced, so you (with the help of Codex) can debug and analyze the problem to see if it's in Codex or the MCP server.

This should be a problem with the vscode plugin, I use the cli and I can start and use this mcp normally

etraut-openai contributor · 5 months ago

The VS Code Extension uses the CLI under the covers, so you should see the same behavior in both cases. One potential difference is in the environment variables that are set when the CLI is launched by the VS Code Extension versus when you launch it yourself from the terminal. Sometimes env variables will change the behavior of MCP servers.

Ashutosh0x contributor · 5 months ago

Hey, I've put up a fix for this. It looks like mcp-client wasn't cleaning up the entire process group on Unix when the transport dropped. I've added a ProcessGroupGuard to ensure we send a SIGKILL to the whole group. That should stop the zombie processes.

NormanMises · 5 months ago

Hey, I've put up a fix for this. It looks like mcp-client wasn't cleaning up the entire process group on Unix when the transport dropped. I've added a ProcessGroupGuard to ensure we send a SIGKILL to the whole group. That should stop the zombie processes.

Has the modification been merged? I really need it

Ashutosh0x contributor · 5 months ago

@NormanMises Not yet, the fix is implemented but still pending merge. I'll update here once it's in.

etraut-openai contributor · 5 months ago

@Ashutosh0x, in case you didn't see, we've announced that we are not accepting unsolicited code contributions from the community. If you'd like to help us improve Codex, we welcome contributions in the form of bug reports, repro steps, and root cause analyses.

Ashutosh0x contributor · 5 months ago

in case you didn't see, we've announced that we are not accepting unsolicited code contributions from the community. If you'd like to help us improve Codex, we welcome contributions in the form of bug reports, repro steps, and root cause analyses.

@etraut-openai Got it, thanks for the update! Totally understand and I’m happy to help with bug reports, repro steps, and root cause analysis instead.

Ashutosh0x contributor · 5 months ago

Analysis of Issue #10581: MCP Process Storm

Bug Description

The Codex VS Code plugin fails to start ace-tool MCP correctly, leading to a process/thread storm that eventually freezes the physical machine. Logs indicate a proliferation of npx ace-tool processes.

Reproduction Steps

Configure ace-tool, ace-tool-rs, or auggie MCP in Codex.
Open the VS Code Codex plugin.
Observe process count increasing on the host/remote machine until freeze.

Root Cause Analysis

The issue stems from incomplete process cleanup in the codex-rmcp-client.

Process Creation: The client spawns the MCP server (e.g., npx ace-tool) using tokio::process::Command.
Termination Behavior: When the client disconnects or reloads (dropping the transport), the default behavior of tokio::process::Command (with kill_on_drop(true)) is to send a termination signal to the direct child process.
Process Groups: Tools like npx spawn subprocesses. When the parent npx process is killed, its children (the actual tool logic) are not automatically killed. They become orphaned and continue running, leading to the observed "storm" as Codex retries connection and spawns new instances.

Proposed Resolution

To ensure complete cleanup, the client must manage the process group:

New Process Group: Start the child process in its own process group (using setpgid or equivalent Unix mechanisms).
Group Kill: Upon termination (Drop), send SIGKILL to the entire process group (negative PID) rather than just the single process ID.

Implementation Details

The fix involves introducing a ProcessGroupGuard in rmcp-client that:

Captures the process group ID (PGID).
Implements Drop to call libc::kill(-pgid, libc::SIGKILL) on Unix systems.
Configures the command with process_group(0) during spawn.

This logic ensures that even if npx acts as a wrapper, the underlying tool processes are terminated along with the wrapper.

etraut-openai contributor · 5 months ago

Thanks @Ashutosh0x, I agree with your analysis.

NormanMises · 5 months ago

Thank you all very much

Ashutosh0x contributor · 5 months ago

Root-Cause Analysis: MCP Process Storm (Issue #10581)

Problem

When using certain MCP servers (notably those invoked via wrappers like npx, e.g., npx ace-tool), the Codex client can trigger a "process storm." This occurs when the client attempts to reconnect or reload, leading to dozens of orphaned subprocesses that eventually consume all system resources and freeze the host machine.

Technical Root Cause

The issue lies in how subprocess termination is handled in the codex-rmcp-client.

Process Wrapping: Tools started via npx or shell scripts create a process tree where the direct child of the Codex client is the wrapper (e.g., node/npx), which then spawns the actual tool logic as a grandchild.
Insufficient Cleanup: The default cleanup mechanism in many async runners (including tokio::process::Command with kill_on_drop(true)) sends a termination signal only to the direct child PID.
Orphaned Subprocesses: On Unix-like systems, killing the parent wrapper does not automatically terminate the children unless they are part of a managed process group that is signaling collectively. The children become orphaned, keeping the MCP transport artifacts (like sockets or pipes) in a zombie or active state, and preventing clean restarts.

As Codex retries the connection, it spawns new wrappers which in turn spawn new tool processes, leading to the process storm.

Proposed Resolution

To ensure reliable cleanup of the entire subprocess tree, the client should manage the Process Group:

Process Group Initialization: The command should be configured to start in its own process group (e.g., using setpgid(0, 0) on Unix).
Scoped Termination (ProcessGroupGuard): Implement a RAII guard (ProcessGroupGuard) that, upon Drop, sends a signal (e.g., SIGKILL) to the entire process group (using the negative PID/PGID) rather than just the individual process.

Implementation Strategy

codex-rs/rmcp-client/src/rmcp_client.rs: Introduce a ProcessGroupGuard and update RmcpClient::new_stdio_client to utilize command.process_group(0) on supported platforms.
Platform Coverage: While the primary issue is observed on Unix due to npx child-parent signaling defaults, the guard provides a structural way to ensure cleanup across the session lifecycle.

Verification

Verified that killing the process group successfully terminates both the npx wrapper and the underlying tool process (e.g., ace-tool).
Integration tests for rmcp_client confirm that transport drops now leave zero lingering subprocesses in the process table.

NormanMises · 5 months ago

It appears this bug still exists. Specifically, the mcp configured with npx works fine in the CLI, but fails to launch in VS Code Codex.
My environment is Ubuntu 2004, and I installed Node.js and CodeX using Volta.
Even though I enabled network permissions in the sandbox and started maximum privilege mode...