Codex search/grep tool processes run indefinitely with no timeout — orphaned rg floods network FS (Lustre/NFS)

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• #36827

Powered by Codex Action

Seconded. And this issue causes the codex to run increasingly slower as the number of orphaned tool calls accumulates. I had an instance where a task ran for 2h+ instead of the typical 20min as a result of this.

Your PID chain (rg → codex app-server → sh) places this in the code-mode cell lifecycle, and it connects to two adjacent reports worth triaging together. In code mode, exec that outlives its yield deadline is parked as a cell by design — the process keeps running and the model is expected to wait on it later (mechanism detailed in #38495). What's missing is any ownership tie-in: nothing kills a cell when the turn that created it completes, the session goes away, or no wait ever comes back — so an expensive rg over a network FS simply runs to natural completion, 87+ minutes in your case. (#38093 is the third face of the same lifecycle gap: termination/completion status vs actual child state.)

Fix shapes, compatible with the cell model:

1. Ownership-scoped reaping: kill surviving cells at turn end (or session teardown) unless explicitly detached — matches your "reap children when the originating request is gone".
2. Hard wall-clock cap per cell (config, generous default e.g. 10–30 min) as the backstop for the abandoned-cell case.
3. For search workloads specifically: rg supports bounded output; the shell/code-mode prompt guidance could steer toward --max-count/path scoping on network filesystems, but 1–2 are the structural fix.

Note plain (non-code-mode) exec already kills at timeout_ms — the gap is specifically cells, which is why this only bites app-server/code_mode_host deployments like your cluster.

Additional reproduction: native Windows Codex Desktop on a local SSD

I reproduced the same underlying failure mode in the native Windows Codex Desktop app, without WSL and without a network filesystem. This case also shows that multiple overlapping code-mode search cells can saturate a local workstation.

Environment

• Codex Desktop package: 26.818.5345.0
• OS: Windows 10 Enterprise 22H2, build 19045.6456, x64
• CPU: Intel Core i7-12700KF, 20 logical processors
• Memory: 64 GB
• Bundled ripgrep: 15.1.0
• Workspace: a large aggregate/multi-project directory on a local SSD
• Parent app-server command included -c features.code_mode_host=true app-server

Private usernames, query terms, and workspace paths are redacted below.

Observed process tree

ChatGPT.exe
└─ codex.exe -c features.code_mode_host=true app-server
   ├─ pwsh.exe
   │  └─ rg.exe
   └─ pwsh.exe
      └─ rg.exe

Codex launched two overlapping searches with this general shape:

rg -n --hidden `
  --glob "!**/Library/**" `
  --glob "!**/Temp/**" `
  --glob "!**/obj/**" `
  --glob "!**/bin/**" `
  "<redacted-pattern>" D:\workspace |
  Select-Object -First 500

One invocation searched the complete aggregate workspace. The other searched two large subtrees inside it. Each rg.exe had 13 threads.

The downstream Select-Object -First 500 only bounds emitted results. When fewer than 500 matches exist, it does not bound the filesystem scan, so rg still traverses the complete requested scope.

Resource impact

The two searches ran concurrently for more than 20 minutes.

An instantaneous Task Manager sample showed:

| Process | CPU | Memory | Disk read |
|---|---:|---:|---:|
| rg.exe A | 32.7% | 2,744 MB | 1,675.5 MB/s |
| rg.exe B | 32.0% | 2,474 MB | 1,784.2 MB/s |

Additional sampling showed:

• Total system CPU remained at approximately 96–98%, reaching 100% in Task Manager.
• Later aggregate disk throughput remained around 843–982 MB/s.
• Each search consumed approximately 2.5–2.9 GB of private memory.
• Together, the two searches used roughly 10–13 logical CPU cores.
• The workstation became noticeably sluggish.
• One search eventually exited naturally; the other was still running at a later diagnostic check.

The aggregate workspace root did not have a single ignore file covering every subtree. It contained multiple full project trees, dependency/SDK directories, generated content, archives, and hidden directories. The commands used --hidden and excluded only Library, Temp, obj, and bin.

Expected behavior

Codex should:

1. Apply a hard wall-clock timeout and CPU/I/O/resource budget to code-mode search cells.
2. Limit concurrent full-workspace searches and avoid overlapping roots.
3. Cancel superseded searches.
4. Reap the complete child-process tree when the owning turn/session ends or no follow-up wait occurs.
5. Prefer scoped candidate directories and file types before searching an aggregate workspace.
6. Avoid broad --hidden searches by default and apply safe exclusions for VCS, dependencies, generated output, archives, caches, and SDKs.
7. Surface long-running background searches in the UI with a user-visible cancel control.

Actual behavior

No effective timeout, throttling, overlap detection, or resource budget was applied. Two model-issued searches were allowed to consume most of the machine's CPU and SSD bandwidth for an extended period.

This does not appear to be a ripgrep defect: rg was executing the broad searches it was given. The suspected defect is in Codex search orchestration and code-mode cell lifecycle management.

Related Windows reports: #7572 and #34264.

Additional Linux reproduction on codex-cli 0.149.1

I reproduced the same code-mode cell lifecycle failure on Linux with local ext4/LVM storage (not Lustre/NFS). This also appears to interact with the nested-result visibility problem described in #32411.

Environment

• Codex CLI: 0.149.1 (npm)
• OS: Linux 5.4.x, x86_64
• Storage: local ext4 on LVM
• Command shape (private paths and patterns redacted):

rg -uuu -l '<pattern-a>|<pattern-b>' /large/workspace /large/build-cache /tmp 2>/dev/null | head -100

The command was launched from a code-mode exec cell through a nested exec_command with yield_time_ms: 30000. The script only emitted result.output, so after 30.2 seconds the outer cell rendered Script completed with empty output while the returned session_id was not surfaced:

const result = await tools.exec_command({
  cmd: "<redacted rg pipeline>",
  yield_time_ms: 30000,
});
text(result.output);

Observed result

• The rg and downstream head survived after their shell/owning cell was gone and were reparented to PID 1.
• rg remained alive for about 89 minutes until manually terminated.
• /proc/<pid>/io showed approximately 1.19 TB in read_bytes.
• The backing device reached about 99.9% utilization with roughly 306 MB/s reads.
• System CPU I/O wait reached about 45.5%; editor, Git, and language-server processes blocked on disk I/O.
• Sending SIGTERM to the exact validated rg PID terminated it.

This confirms the issue still reproduces on 0.149.1 and can saturate local block storage, not only network filesystems.

Expected behavior

• If a nested command yields a session_id, the owning cell should retain/surface that handle and keep lifecycle ownership.
• If the owning cell/turn ends without retaining the handle, Codex should terminate the complete process group, or enforce a hard wall-clock/resource cap.
• A yielded nested command should not appear indistinguishable from a completed command when only result.output is emitted.

No raw session logs, usernames, hostnames, private paths, query terms, or project identifiers are included.