[Windows][Desktop 26.707.12708.0] ChatGPT.exe spawns hundreds of taskkill.exe/conhost.exe processes, causing WMI storms and DWM degradation

Open 💬 34 comments Opened Jul 17, 2026 by AnitaHailey0306
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

[Windows][Desktop 26.707.12708.0] ChatGPT.exe spawns hundreds of taskkill.exe and conhost.exe processes, causing WMI failure storms and DWM degradation

Summary

Codex Desktop on Windows repeatedly leaves taskkill.exe and conhost.exe processes running. During an affected session, 287 live taskkill.exe processes were direct children of the Codex Desktop main process (ChatGPT.exe), with 302 associated conhost.exe processes.

At the same time:

  • Windows Management Instrumentation recorded 987 failed queries in two hours.
  • The Desktop Window Manager handle count increased from 1,205 to 4,458 without a DWM restart.
  • Closing ordinary application windows left a white frame visible for about two seconds.
  • Windows UI responsiveness degraded, and DWM CPU usage remained elevated.

The problem returned within about one hour after a clean Windows restart.

Environment

  • Windows 10 Pro 22H2, build 19045.6466, x64
  • Codex Desktop MSIX package: OpenAI.Codex_26.707.12708.0_x64__2p2nqsd0c76g0
  • Bundled Chromium / ChatGPT.exe: 150.0.7871.115
  • GPU: NVIDIA GeForce RTX 2080 Ti
  • NVIDIA driver: 572.16
  • Display environment also included remote-display and virtual-display software. These may amplify DWM load, but the abnormal taskkill.exe parent chain points directly to ChatGPT.exe.

User-visible symptoms

  1. DWM CPU usage rises and remains above its normal idle level.
  2. Closing a window leaves a white rectangular frame for about two seconds.
  3. Typing and application rendering become unreliable when the condition worsens.
  4. Fully restarting Windows clears the condition temporarily.
  5. The problem returns while Codex Desktop remains in use.

Process-tree evidence

At 2026-07-17 13:21 local time, a Toolhelp process-tree snapshot showed:

| Process | Live count | Parent |
| --- | ---: | --- |
| taskkill.exe | 287 | All 287 were direct children of ChatGPT.exe PID 7516 |
| conhost.exe | 302 | The newly created instances were children of the taskkill.exe processes |
| ChatGPT.exe | 8 | Main process PID 7516 |
| codex.exe | 1 | Child of ChatGPT.exe PID 7516 |
| codex-code-mode-host.exe | 1 | Child of codex.exe |

The process storm was still active: conhost.exe increased from 296 to 302 in five seconds, while the total process count increased from 897 to 907.

DWM and Codex resource growth

DWM was not restarted during this comparison; it remained PID 2108.

| Measurement | Healthy baseline after restart | Affected state about one hour later |
| --- | ---: | ---: |
| DWM handles | 1,205 | 4,458 |
| DWM private working set | about 29 MiB | about 56 MiB |
| DWM processor-time counter | about 9-21% | about 42-85% |
| codex.exe handles | about 542 after a fresh launch | about 3,565 |

The processor-time values above use the Windows per-process performance-counter scale, not Task Manager's whole-system CPU percentage.

GPU counter samples during the affected state showed approximately 177 MiB of local GPU memory attributed to DWM and 100-116 MiB attributed to a ChatGPT.exe GPU process. No corresponding GPU-process crash, DXGI device-loss, or out-of-memory message was found in the Codex logs.

WMI failure storm

The Microsoft-Windows-WMI-Activity/Operational log contained 989 events over two hours. Of these, 987 were Event ID 5858 failures.

A representative failure contained:

Operation = Start IWbemServices::ExecQuery - root\cimv2 :
SELECT * FROM Win32_Process
WHERE ProcessId = ... OR ProcessId = ... [hundreds of process IDs]

ResultCode = 0x800706BA
PossibleCause = Could not send status to client

The recorded ClientProcessId belonged to a short-lived process that had already exited when inspected. The queries repeatedly enumerated a very large set of process IDs, including the accumulating console and cleanup processes.

This explains why the Windows Management Instrumentation service intermittently reached about 20% CPU even though a later 10-second sample returned to 0%: the load occurs in bursts during the failed query loop.

Codex renderer and Windows event evidence

The affected Codex session logged 88 instances of:

ResizeObserver loop completed with undelivered notifications.

They appeared in bursts while the primary renderer window was focused and visible.

No new NVIDIA display-reset event, DWM crash, DXGI device loss, or GPU out-of-memory event was recorded during the clean recurrence window.

Earlier affected sessions also produced Windows Error Reporting records for ChatGPT.exe, including MoAppHang, Top level window is idle, exception 0xc06d007f, and signature PCH_84. Those records may be related but are not required to reproduce the process and WMI growth described above.

Steps to reproduce

  1. Restart Windows to clear all residual processes.
  2. Open Codex Desktop and work in a local task that invokes shell commands.
  3. Continue using the task for approximately one hour. Commands that time out, are cancelled, run concurrently, or produce large output may increase the reproduction rate, but the exact trigger has not been isolated.
  4. Inspect the process tree under the main ChatGPT.exe process.
  5. Observe accumulating taskkill.exe processes and their conhost.exe children.
  6. Inspect Microsoft-Windows-WMI-Activity/Operational for repeated Event ID 5858 failures containing large Win32_Process queries.
  7. Observe DWM handle growth and close an ordinary application window. The closed window may leave a white frame for about two seconds.

Expected behavior

  • Codex should terminate or reap its Windows cleanup subprocesses.
  • Process-tree cleanup should not leave hundreds of taskkill.exe or conhost.exe instances.
  • Cleanup should not repeatedly query hundreds of stale process IDs through WMI.
  • DWM resource usage and general Windows UI responsiveness should remain stable during long Codex sessions.

Working diagnosis

The evidence is consistent with a Windows process-cleanup or cancellation loop in Codex Desktop. The main ChatGPT.exe process repeatedly starts taskkill.exe, while WMI is used to inspect a growing process tree. Short-lived query clients exit before WMI can return status, producing repeated Event ID 5858 failures. The accumulated console processes and handles then increase DWM and system UI load.

This is an inference from the live parent chain, counters, and event timing. Source-level confirmation would require inspection by the Codex maintainers.

Current workaround

  • Fully exit Codex Desktop instead of only closing its window.
  • Restart Windows if residual taskkill.exe or conhost.exe processes remain.
  • Avoid concurrent shell calls, repeated command cancellation, and very large command output until the cleanup path is fixed.
  • Do not restart or terminate DWM or the WMI service as a workaround.

Related reports

Privacy note

This report intentionally excludes the Windows username, full local paths, conversation contents, prompts, repository names, raw Codex databases, and complete log files. Sanitized counter output and selected WMI/DWM event records can be provided if maintainers request them.

View original on GitHub ↗

34 Comments

github-actions[bot] contributor · 1 month ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #33192

Powered by Codex Action

mweichse · 1 month ago

Reproduced on another system: Windows 11 Pro (build 26200), 32-thread CPU, 64 GB RAM, and NVMe storage. The Desktop app updated twice during the session via the in-app prompt (26.707.9981 -> 26.715.2305 -> 26.715.3651; command-runner 0.145.0-alpha.18). The behavior was unchanged across all three versions.

Observed process-creation rate while a single agent turn was active (PID-diffing Get-Process every 400 ms; therefore a lower bound because shorter-lived processes can be missed):

  • 315-1,022 newly observed PIDs per 8-15 second window, or approximately 40-70 observed process creations/second sustained
  • One 15-second window contained 467 conhost, 339 taskkill /pid <pid> /t /f, and 169 git processes, plus PowerShell/CMD helpers
  • Parent sampling via Win32_Process.ParentProcessId showed that the observed taskkill and git children were overwhelmingly parented by ChatGPT.exe

System impact during affected periods:

  • WmiPrvSE.exe sustained 20-26% whole-machine CPU on this 32-thread system (approximately 6.5-8 logical cores) while the process storm and repeated Win32_Process tree queries were occurring. It also reached approximately 11,000 page faults/second.
  • System-wide input latency reached multiple seconds on a wired keyboard, and window switching became slow even though total CPU was 37-73%, disk queue was approximately zero, and about half the RAM remained free. This looked like process/console-window churn rather than conventional CPU, disk, or memory exhaustion.
  • Microsoft Defender added further load during the storm; MsMpEng consumed approximately another 1-3 logical cores. This is consistent with the Defender behavior reported in #30527 and #33875.
  • Control experiment: terminating ChatGPT.exe reduced observed spawning from approximately 45/second to approximately 0.4/second and WmiPrvSE CPU from approximately 25% to 0.3% immediately.

The parent-attribution sampler itself used Win32_Process, so it could contribute some WMI work. I therefore treat the simultaneous WMI load and process storm as correlation rather than attributing every query to taskkill. The immediate control-experiment drop nevertheless strongly associates the overall load with the active Desktop session.

Amplifier: workspace tracking of untracked build caches (consistent with #32113):

The workspace tracker invoked:

git -c core.hooksPath=NUL -c core.fsmonitor= hash-object -- <file>

as a separate git process for each untracked file. A repository containing untracked .gocache/ directories with 2,274 and 944 files was rehashed file by file on repeated sweeps. Adding .gocache/ to .git/info/exclude roughly halved the observed git spawn rate. Large untracked trees such as build caches, node_modules variants, or Unity Library directories may therefore amplify the problem substantially.

Per-command overhead: %USERPROFILE%\.codex\.sandbox\sandbox.*.log showed a setup refresh (including codex-windows-sandbox-setup.exe), a read-ACL pass, and a command-runner helper copy for each command. Even a trivial one-line read consequently involved approximately 10 observed processes including console hosts and the subsequent taskkill tree cleanup.

Possible mitigation directions, in likely impact order:

  1. Consider native process-lifetime management such as Windows Job Objects instead of invoking taskkill /t /f for each cleanup, if compatible with the sandbox architecture.
  2. Reuse a persistent worker/console rather than starting a fresh PowerShell, console host, and sandbox refresh for every command.
  3. Have workspace tracking apply Git ignore/exclusion rules before hashing and batch paths through hash-object (for example, using --stdin-paths).
  4. Throttle, scope, or remove the approximately once-per-second full-process WMI poll reported in #25453.
BIOcanse · 1 month ago

Source-level root cause and the 26.707 -> 26.715 regression boundary

I performed a read-only inspection of the installed Windows Desktop bundle and correlated it with local Codex logs and live process snapshots. No prompt content, repository names, private paths, or credentials are included below.

Inspected build: MSIX 26.715.3651.0, internal Electron app 26.715.31251 (build 5538), bundled app-server 0.145.0-alpha.18.

This is not merely "too many Git processes." The current Desktop build contains a closed feedback loop with four components.

1. Git cancellation invokes an unbounded process-tree kill

The installed app bundle's Windows cleanup path is equivalent to:

~~~js
await execFile(
path.join(systemRoot, "System32", "taskkill.exe"),
["/pid", String(pid), "/t", "/f"],
{ env, windowsHide: true }
);
~~~

There is no timeout, no single-flight guard, no creation-time/process-identity check, and no upper bound on concurrent cleanup helpers. The host awaits this cleanup promise.

The local Git host enables tree cleanup for Git commands, whose normal default timeout is 60 seconds. Therefore every timeout or cancellation can create another unbounded taskkill.exe /PID ... /T /F process.

2. The live review query creates and cancels per-file Git subprocesses

The Desktop worker keeps live review-summary/status-summary queries subscribed to working-tree, index, HEAD, and config invalidations. The review summary includes untracked files by default.

For untracked metadata, the current worker processes up to 8 files concurrently and runs a separate command for each ordinary file:

~~~text
git hash-object -- <path>
~~~

It can also run a per-file no-index diff. Rapid invalidation or cancellation therefore fans out Git children, and each cancellation enters the taskkill path above. Local logs show review-summary as the request kind and thousands of aborted Git commands/hash-object calls.

This is the primary producer.

3. PID-only /T cleanup becomes a self-sustaining cross-kill loop

In a live sample of 120 taskkill processes created over about 34 seconds:

  • 105 target PIDs had already exited.
  • 7 target PIDs had already been reused by taskkill.exe.
  • 7 had been reused by conhost.exe.
  • 1 belonged to git.exe.

This is direct evidence that stale PID-only cleanup races with PID reuse. The /T helpers begin terminating cleanup helpers and console hosts spawned by other cleanup attempts. That converts ordinary cancellation churn into a feedback loop.

An independent bounded Toolhelp/.NET cleanup, which did not invoke taskkill, removed 325 taskkill processes, 164 conhosts, and 10 Git processes in one pass and brought taskkill to zero. Re-entering an active affected Desktop task had previously recreated hundreds of helpers, confirming that the Desktop process lifecycle is the active producer rather than Windows spontaneously generating them.

4. Full-machine CIM sampling amplifies the loop into an OS freeze

The child-process snapshot worker first runs a full:

~~~powershell
Get-CimInstance Win32_Process
~~~

It then queries Win32_PerfFormattedData_PerfProc_Process plus PID-filtered Win32_Process data in batches. A heartbeat runs every 30 seconds, while renderer/event-triggered requests can run with only a 5-second minimum interval. A new worker is created for each snapshot.

Once process count explodes, every sample scans a rapidly changing machine-wide PID set. Timed-out clients and quota failures then add CPU, process, thread, handle, and WMI pressure. This explains the observed "out of memory" condition while about 16.8 GB of physical RAM remained free: the exhausted resources were process/thread/handle/WMI quotas, not RAM capacity.

5. The process registry is not crash-safe

chat_processes.json is loaded with JSON.parse and malformed/all-NUL content is not recovered (other than ENOENT). Records may persist with osPid:null. Updates are written directly with writeFile(JSON.stringify(...)); there is no temporary file, fsync, atomic rename, or last-known-good recovery.

That matches the all-NUL corruption already reported in #29593 and can carry bad lifecycle state across sessions.

Local version boundary

These are aggregate markers from local Desktop logs; session lengths differ, so the table is a regression boundary, not a normalized benchmark:

| Desktop version | Sessions | Snapshot failures | WMI quota markers | hash-object calls |
|---|---:|---:|---:|---:|
| 26.707.9981 | 9 | 69 | 0 | 0 |
| 26.707.12708 | 2 | 57 | 0 | 0 |
| 26.715.2236 | 1 | 54 | 164 | 22 |
| 26.715.2305 | 6 | 3,831 | 8,632 | 2,697 |
| 26.715.3651 | 6 | 614 | 1,604 | 156 |

The unsafe process-lifecycle design predates 26.715, as this issue and older reports demonstrate. However, the catastrophic WMI-quota/freeze regime begins sharply at the 26.707 -> 26.715 boundary on this machine. The recent update is therefore the severity amplifier/regression boundary, even if it did not invent the original bug.

The public CLI changed from app-server 0.144.5 to 0.145.0-alpha.18 in the same boundary. Public commit 6ad0e943cc72 (#32106) raises ancestor-probe concurrency from 8 to 256, which may add startup pressure, but it is not the direct taskkill source. The Electron/Desktop host source and its update diff are absent from the public repository, so identifying the exact private Desktop commit requires OpenAI to inspect its internal 26.707 -> 26.715 diff. The installed current bundle is sufficient to prove the faulty call chain above.

Required remediation

  1. Immediate mitigation: stop or roll back the affected Windows rollout; disable live per-file review fanout and machine-wide CIM sampling until bounded implementations ship.
  2. Replace taskkill /T with a Windows Job Object or bounded Toolhelp traversal. Bind cleanup to PID plus creation time/process handle, not PID alone.
  3. Add a hard timeout, concurrency limit, deduplication, and single-flight semantics to cleanup. Never await a cleanup helper indefinitely.
  4. Coalesce working-tree invalidations and batch Git paths. Do not spawn one Git process per untracked file.
  5. Replace full-machine WMI polling with Job Object accounting, Toolhelp, ETW, or direct process handles scoped to owned children.
  6. Write the process registry atomically with temp-file + flush + rename, validate it on load, and recover from a last-known-good copy.
  7. Add a Windows soak/release-gate test that asserts bounded process creation, cleanup latency, handles, threads, and WMI activity under rapid Git invalidation and cancellation.

Release and ownership concern

This is not an acceptable application failure mode. A coding tool must not be able to hard-freeze the host OS and force repeated power cycles while free memory remains. The architectural warning signs were reported separately for months in #17229, #25453, #26812, #29593, #32113, and #32562. The canonical P0 candidate and multiple precursor issues currently have no public assignee or milestone.

Shipping 26.715 without an end-to-end Windows process-churn soak test turned known defects into a system-wide feedback loop. That is a failure of OpenAI's Codex Desktop maintenance and release process, not an individual contributor problem. OpenAI also does not publish the Desktop host source and its CONTRIBUTING policy rejects unsolicited implementation PRs, so OpenAI cannot reasonably shift responsibility for the fix to affected users.

Please assign an owner, publish a mitigation/rollback, pause the affected Windows rollout, and treat this as a release blocker/P0 candidate. I am available to validate a patched build immediately.

BIOcanse · 1 month ago

Exact regression boundary: build 5484, Git review metadata worker

I narrowed the recent severity regression to a specific production build and shipped bundle file. This refines my previous comment: the unsafe taskkill /T /F, full CIM sampler, and non-atomic chat_processes.json design already existed, but build 5484 added the high-rate Git subprocess producer that can drive those old defects into a system-wide process/WMI storm.

Reproducible four-build comparison

Source: OpenAI's official production appcast: https://persistent.oaistatic.com/codex-app-prod/appcast.xml

I used HTTP Range requests to extract only resources/app.asar from the official ZIPs, parsed each ASAR header, and hashed/searched the relevant bundled files. The build 5488 ASAR SHA-256 independently matches the published distribution hash, and the relevant build 5538 entry hashes exactly match my currently installed Windows package.

| Build | Internal version | Windows package mapping | .vite/build/worker.js SHA-256 | New per-file review metadata path |
|---|---|---|---|---|
| 5440 | 26.707.91948 | prior 26.707 package | cae41298ee5d729ff00c5a72ae9edb6c767fb6e71139c0d6d62371a5900af387 | Absent |
| 5484 | 26.715.21316 | 26.715.2236.0 | 6fccaf96a129f3b8299793a90ddea15442d8af8a54826faa384ee525da4dae27 | First present |
| 5488 | 26.715.21425 | 26.715.2305.0 | 6fccaf96a129f3b8299793a90ddea15442d8af8a54826faa384ee525da4dae27 | Present; byte-identical to 5484 |
| 5538 | 26.715.31251 | 26.715.3651.0 | 8b4855f500ea4f37df95b41122594c4c3e92bee83c968b49b75419a635bdb540 | Still present |

The Windows package/build mappings above come from Codex's own local startup logs (reported internal release version and Sparkle build number).

What changed in build 5484

Build 5440 already contains the review-summary feature, taskkill.exe, Get-CimInstance Win32_Process, and chat_processes.json. However, its worker contains none of these strings:

  • review-working-tree-file-metadata
  • Failed to hash working-tree file
  • Failed to read untracked Git metadata
  • hash-object

All four first appear in build 5484 inside .vite/build/worker.js. The new implementation walks untracked and zero-OID working-tree entries with a concurrency limit of 8 and, per file, launches operations equivalent to:

git hash-object -- <path>
git diff --no-index --raw --numstat -z -- /dev/null <path>

In build 5484/5488 these are located around minified lines 184, 188, and 190. The review-summary signal count in this worker also rises from 9 occurrences in build 5440 to 18 in build 5484.

That new per-file subprocess fan-out feeds an existing unsafe cancellation path. The local Git host uses killProcessTree: true; cancellation reaches:

taskkill.exe /pid <PID> /t /f

The cleanup call has no timeout, no single-flight/coalescing guard, and no process-identity check before killing a potentially reused PID. At the same time, the child-process snapshot worker performs full CIM process/performance queries. Under rapid repo-watch invalidation, the combination is capable of producing exactly the observed feedback loop: many short-lived Git children, many tree-kill cleanups, and WMI query quota exhaustion.

Exact current shipped files (build 5538)
  • app.asar/.vite/build/worker.js

SHA-256 8b4855f500ea4f37df95b41122594c4c3e92bee83c968b49b75419a635bdb540
hash-object: line 184 col 693 and line 190 col 62
review-working-tree-file-metadata: line 188 col 7078

  • app.asar/.vite/build/src-tv6ZWK8o.js

SHA-256 8a07ef9ad8a2643b9812341c0ba7c770803664bd1c473f8603b837084f86d72c
taskkill.exe: line 771 col 5661

  • app.asar/.vite/build/child-process-snapshot-worker.js

SHA-256 5fb99e4f2f053613676958cecebacb24401f37cee5bbc294708ec280793cb47c
full CIM/perf query block: line 96

The production ASAR omits the source maps, so the original TypeScript path is not directly recoverable from the official artifact. An unaffiliated reverse-engineered older bundle maps this logic with high confidence to:

  • main/workers/git-worker-review/metadata.ts
  • main/workers/git-worker-request-dispatcher.ts
  • main/workers/git-worker-repo-watch.ts
  • main/workers/worker-execution-host-client.ts
  • main/workers/child-process-snapshot-worker.ts

Reference only (not an official source tree): https://github.com/JimLiu/decode-codex/tree/6fd43d66ccad32c9c1ab83b9704e0bbbbf2d4c7b/restored/main/workers

Requested maintainer action

Please diff internal build 5440 -> 5484 for main/workers/git-worker-review/metadata.ts first. Revert or batch the new per-file subprocess path, coalesce repo-watch invalidations, and bound the queue globally. Independently harden process cleanup with single-flight, timeout, PID identity verification, and a strict cap; make the process registry atomic and corruption-tolerant.

This is a release-safety failure across interacting subsystems, not evidence against any individual developer. The key actionable boundary is now specific: build 5484, Git review metadata worker.

cnxobo · 1 month ago

Additional clean-reboot reproduction on 26.715.3651.0

I reproduced the same failure on another Windows system after a full reboot. This confirms that the process storm is recreated by a new Codex Desktop session rather than being leftover state from the previous boot.

Environment
  • Windows build 26200.8875 (25H2), x64
  • AMD Ryzen 9 7945HX, 32 logical processors
  • 64 GB RAM
  • Codex Desktop MSIX: OpenAI.Codex_26.715.3651.0_x64__2p2nqsd0c76g0
  • Bundled ChatGPT.exe / Chromium version: 150.0.7871.124
Clean-reboot reproduction
  1. Fully reboot Windows.
  2. Launch Codex Desktop.
  3. Open one local thread and run read-only shell diagnostics. Some WMI/CIM-based commands timed out.
  4. Inspect the process tree approximately 7.5 minutes after Codex launched (system uptime: 12 minutes 29 seconds).

A six-second snapshot produced:

| Metric | First sample | Six seconds later |
| --- | ---: | ---: |
| Total processes | 815 | 809 |
| taskkill.exe | 223 | 220 |
| conhost.exe | 230 | 228 |
| ChatGPT.exe | 8 | 8 |
| Total handles | 250,518 | 249,684 |
| Total threads | 11,821 | 11,810 |

The overwhelming majority of the live taskkill.exe instances were direct children of the Codex Desktop main process, ChatGPT.exe PID 33116. Of the live console hosts, 214 had taskkill.exe as their direct parent.

The process groups accounted for approximately:

  • taskkill.exe: 219 instances, 2.97 GiB aggregate working set, 36,684 handles
  • conhost.exe: 227 instances, 1.72 GiB aggregate working set, 22,445 handles

Creation timestamps showed continuous accumulation from about one minute after Codex startup through the next several minutes.

Desktop log evidence

The previous affected run's Codex Desktop log repeatedly recorded:

[electron-sampler] Failed to collect child process snapshot
Command failed: powershell.exe -NoProfile -NonInteractive -Command
$ErrorActionPreference = 'Stop';
Get-CimInstance Win32_Process |
  Select-Object ProcessId,ParentProcessId |
  ConvertTo-Json -Depth 2

The same log also contains failures from BundledPluginsMarketplace while executing a heavier process/performance snapshot based on Win32_PerfFormattedData_PerfProc_Process and Win32_Process.

This evidence is consistent with the reported feedback loop: process-tree sampling fails or stalls, cleanup launches taskkill.exe /T /F, the cleanup processes and their console hosts accumulate, and subsequent full-machine process enumeration becomes increasingly expensive. The direct parent chain establishes that the cleanup processes originate from Codex; source-level confirmation of the exact retry path still requires maintainer inspection.

A previous manual cleanup reduced the machine from approximately 971 processes to 445, but after a clean reboot and relaunch the same condition returned within minutes.

No raw logs, usernames, repository names, conversation contents, or local paths are included here. I can provide additional sanitized counters if maintainers need a specific capture.

Northern-Summer · 1 month ago

Additional reproduction on Windows 10.0.26200 x64 with Codex Desktop
OpenAI.Codex 26.715.2305.0.

Native 50 ms process sampling reproduced:

  • ChatGPT.exe -> powershell.exe -> conhost.exe
  • approximately 100 process births/minute
  • median lifetime approximately 1.43 seconds

A severe cleanup recurrence reached:

  • 215 taskkill.exe processes directly owned by ChatGPT.exe
  • 224 conhost.exe processes
  • approximately 4.3 GiB combined working set
  • 998 WMI Event 5858 failures in 15 minutes

The packaged app.asar child-process snapshot worker executes the matching
Get-CimInstance Win32_Process / Win32_PerfFormattedData_PerfProc_Process
PowerShell probes. Cockpit Tools was separately stopped and remained absent;
the ChatGPT-owned taskkill/WMI storm still reproduced, so this is not caused
by the local Wiki scripts or Cockpit.

A bounded local mitigation used native Toolhelp snapshots, PID+creation-time
identity, a 10-second age gate and two-snapshot parent-lineage verification.
A one-shot watcher was falsified because births continued after its first
containment. The corrected multi-round watcher made 17 containment attempts,
stopped 284 lineage-verified taskkill processes, never directly stopped
conhost/PowerShell/cmd/ChatGPT, and reached 10 consecutive zero-taskkill
samples. This only mitigates accumulation; it does not prevent upstream births.

Requested upstream fixes:

  1. Replace per-sample PowerShell/WMI spawning with a persistent or native helper.
  2. Coalesce sampling and add rate limiting/backoff/circuit breaking.
  3. Correctly reap taskkill/conhost cleanup children and stop retry amplification.
  4. Provide a supported diagnostic/telemetry sampling disable switch.
BIOcanse · 1 month ago

Correction: exact direct-parent boundary, two independent defects, and maintainer source anchors

I downloaded and compared the complete worker.js files for build 4559 and its direct child build 5059. This corrects two statements in my earlier comments:

  1. Build 5484 is not the first introduction of the core producer/cleanup chain. It adds the later per-file metadata amplifier, but the complete unsafe chain first appears in the publicly recoverable direct-parent boundary 4559 -> 5059.
  2. Saying there was "no single-flight guard" was too broad. There is per-original-child single flight (cleanupPromise ??=). What is absent is a global cleanup limit/coalescer, timeout, circuit breaker, and PID identity validation.

Direct-parent comparison: https://github.com/JimLiu/decode-codex/compare/54c373f781d0ee51f0c16f58ec349ae97a16f941...fa0ef1d8be41adc202d1f92e80cf1937f59ae45a

That repository is an unaffiliated snapshot, not an official source tree. The direct-parent bundle diff is the evidence; OpenAI should verify the corresponding private-source commit internally.

Defect A: the producer continuously creates cleanup work

At build 4559, worker.js:61466 is:

child = await host.spawn({ args, cwd, env });

At build 5059, worker.js:61718 becomes:

child = await host.spawn({
  args,
  cwd,
  env,
  killProcessTree: true,
});

Build 5059 also introduces this review-snapshot cancellation path at lines 64904-65069:

class GitReviewSnapshot {
  abortController = new AbortController();
  signal = this.abortController.signal;

  retire() {
    if (!this.signal.aborted) {
      this.abortController.abort();
      this.removeRetiredStoreWhenIdle();
    }
  }

  commandOptions(options, objectStoreEnv) {
    return {
      ...options,
      env: { ...objectStoreEnv, ...options.env },
      signal:
        options.signal == null
          ? this.signal
          : AbortSignal.any([this.signal, options.signal]),
    };
  }
}

invalidateGitReadCachesForRepoChange(changeType, changedPaths) {
  this.gitReadGenerationValue += 1;
  this.advanceReviewSnapshot();
  invalidateQueriesForRepoChange(/* ... */);
}

advanceReviewSnapshot() {
  const previous = this.reviewSnapshotValue;
  this.reviewSnapshotValue = new GitReviewSnapshot(this, previous.generation + 1);
  previous.retire();
}

The live-query loop at lines 70578-70640 keeps a dirty flag while a request is in flight and schedules another refresh in finally. review-summary reacts to working-tree changes at lines 70669-70677. The repo-event handler at lines 72343-72348 both advances/aborts the review snapshot and calls liveQueryRegistry.handleRepoChanged(...).

Therefore repository churn repeatedly aborts active Git children; each distinct child has opted into process-tree cleanup. Build 5484 later increases the rate with per-file hash-object / no-index metadata work, but it is an amplifier, not the origin.

Defect B: PID-only /T cleanup does not converge

Build 4559 lines 64163-64198 simply call the existing child kill method. Build 5059 lines 64529-64715 add the following code (identifiers expanded for readability, control flow unchanged):

const execFileAsync = promisify(childProcess.execFile);

function wrapChild(child, stdout, stderr, timeout, killTree = async () => child.kill()) {
  let cleanupPromise = null;

  const kill = () => {
    if (timeout != null) clearTimeout(timeout);
    cleanupPromise ??= killTree().catch((error) => {
      try {
        child.kill("SIGKILL");
      } catch {}
      throw error;
    });
    cleanupPromise.catch(() => {});
  };

  const waitPromise = child.wait().then(async (result) => {
    await cleanupPromise;
    // close streams and return result
    return result;
  });

  return { kill, wait: () => waitPromise /* ... */ };
}

async function killProcessTree(child) {
  const pid = child.getPid();
  if (pid == null) {
    child.kill("SIGKILL");
    return;
  }

  if (process.platform === "win32") {
    const systemRoot = process.env.SystemRoot ?? process.env.windir ?? "C:\\Windows";
    await execFileAsync(
      path.win32.join(systemRoot, "System32", "taskkill.exe"),
      ["/pid", String(pid), "/t", "/f"],
      { env, windowsHide: true },
    );
  }
}

wrapped = wrapChild(
  child,
  stdout,
  stderr,
  timeout,
  options.killProcessTree ? () => killProcessTree(child) : undefined,
);

The call stores only a numeric PID. It has no process handle/creation-time identity check, no timeout or abort signal, and no global concurrency bound. The host awaits that helper indefinitely.

This second defect must not be conflated with the producer. taskkill.exe does not itself spawn another taskkill.exe, and this is not literal JavaScript recursion. The recursion/cross-kill occurs in Windows /T tree traversal after stale PIDs are reused by peer taskkill.exe or conhost.exe processes. Defect A supplies new helpers; Defect B collapses completion throughput. While A remains active, live helpers = births - completions grows without bound.

Maintainer source-search anchors

The production bundles reference worker.js.map, but the source maps are omitted. The following unique strings and restored-module boundaries should take an internal maintainer directly to the private TypeScript:

  • main/workers/worker-execution-host-client.ts: search taskkill.exe, Local execution host processes do not support resizing, and killProcessTree.
  • main/workers/git-worker-commands.ts: search git process aborted and the host.spawn(... killProcessTree: true) call.
  • Review repository/snapshot owner: search GitReviewSnapshotStaleError, Git snapshot became stale, advanceReviewSnapshot, and invalidateGitReadCachesForRepoChange.
  • main/workers/git-worker-repo-watch.ts: search scheduleRepoChanged and Git repo changed.
  • main/workers/git-worker-request-dispatcher.ts / live-query registry: search subscribe-live-query, git-live-query-updated, handleRepoChanged, and review-summary.
  • main/workers/git-worker-review/metadata.ts (5484+ amplifier): search review-working-tree-file-metadata, Failed to hash working-tree file, hash-object, and Failed to read untracked Git metadata.

Current official build 5538 still contains the same path:

  • .vite/build/src-tv6ZWK8o.js line 771, column 5661: taskkill.exe;
  • .vite/build/src-tv6ZWK8o.js line 750, column 41972: Git killProcessTree:true;
  • .vite/build/worker.js line 181, column 62314: GitReviewSnapshotStaleError;
  • .vite/build/worker.js line 184, column 693 and line 190, column 62: per-file hash-object;
  • .vite/build/worker.js line 188, column 7078: review-working-tree-file-metadata.

Fresh reproduction separating the two defects

While the affected Desktop task remained active during this investigation, live taskkill.exe count jumped from 0 to 278. A bounded direct, non-tree termination pass against only taskkill.exe could not settle: after 80 samples it still oscillated at 2-3 because new helpers were being created. A second bounded pass that stopped active Git children and existing taskkill helpers (without /T) reached 15 consecutive zero/zero Git/taskkill samples; a delayed five-second check remained zero/zero.

This directly separates the active Git producer from the non-convergent cleanup backlog.

Separate but related release-quality failure: large-conversation UI loading

The same release process is repeatedly shipping an avoidable full-history hydration/rendering problem:

  • #32722 contains production-bundle evidence that a newly connected window iterates every entry in streamingConversations and broadcasts the complete conversationState, including loaded turn history. It serializes the entire message before applying a 256 MiB limit, so even dropped payloads incur construction, serialization, allocation, and copying cost.
  • #25390 reproduces a nearly frozen Desktop UI with a 242.5 MB / 50,589-line local thread.
  • #18693 provides an A/B profile test where restoring only a few giant histories (roughly 455 MB to 1.8 GB each) makes typing, scrolling, switching, and the thread list slow again.
  • #29590 shows eager resume/hydration stalls at much smaller history sizes.
  • #33822 shows the current 26.715 renderer can hang on only about 200 KB of streamed tool output while emitting a high-rate ResizeObserver loop.

These reports are all open. The remedy is conventional application engineering, not an unsolved research problem: send metadata plus a bounded recent tail, page older turns, virtualize rendered rows, keep large image/base64/tool payloads off the critical path, apply byte/count gates before serialization, and add backpressure/cancellation when switching threads.

It is not publicly knowable which internal OpenAI department is supposed to supervise this. What is publicly observable is that this P0 candidate still has no assignee, the unsafe process code has shipped from build 5059 through current build 5538, and multiple source-backed large-history reports remain open while the same unbounded paths continue shipping.

That is a release-governance and cross-subsystem ownership failure. A Windows Desktop release gate should cover both:

  1. rapid Git invalidation/cancellation with assertions on process births, cleanup convergence, PID identity, handles, threads, and WMI activity;
  2. opening/switching multi-hundred-megabyte histories with bounded serialized bytes, renderer nodes, memory, CPU, and time-to-first-visible-turn.

Please assign explicit owners for process lifecycle and conversation hydration, publish the affected/fixed build boundary, and treat failure of either soak test as a release blocker.

tsukimori-dev · 1 month ago

Controlled A/B isolation: the Desktop child-process snapshot sampler triggers the WMI CPU storm

This is a follow-up to openai/codex#33776 with a controlled isolation result from the currently shipped Windows build.

Environment

  • Windows x64, build 26200.8875 (25H2)
  • 32 logical processors
  • Codex Desktop MSIX: OpenAI.Codex_26.715.3651.0_x64__2p2nqsd0c76g0
  • Observation date: 2026-07-18 (UTC+08:00)

Confirmed observations before the intervention

  • The updated 26.715.3651.0 build still reproduced the failure. Within roughly 20 seconds of a fresh Codex session, 22 taskkill.exe processes accumulated and WmiPrvSE.exe reached a raw per-process processor-time reading of about 1527% (about 47.7% of a 32-logical-processor machine). A separate Task Manager capture reached 71.2% whole-machine CPU for one WMI Provider Host.
  • Microsoft-Windows-WMI-Activity/Operational showed repeated Win32_Process and Win32_PerfFormattedData_PerfProc_Process work. Representative failures were 0x800706BA with Could not send status to client, consistent with the short-lived PowerShell client exiting while WMI work was still active.
  • Process-creation auditing attributed the WMI client PowerShell processes to the main Codex Desktop ChatGPT.exe process.
  • Read-only inspection of the shipped bundle found .vite/build/child-process-snapshot-worker.js and its sampler manager. On Windows, the worker runs a PowerShell sequence equivalent to:
  1. Get-CimInstance Win32_Process to collect PID/parent relationships;
  2. Get-CimInstance Win32_PerfFormattedData_PerfProc_Process to collect per-process CPU data;
  3. a large PID-filtered Get-CimInstance Win32_Process -Filter "ProcessId = ... OR ..." query for details.
  • The PowerShell request has a 5-second timeout. The sampler manager also has a 30-second focused heartbeat and allows non-heartbeat triggers at a 5-second minimum interval. The shipped build exposes no supported switch that disables this sampler.

Controlled intervention and result

As a diagnostic isolation test, the legacy Windows PerfProc provider was disabled with:

lodctr /d:PerfProc

Stale WmiPrvSE.exe and already accumulated taskkill.exe processes were stopped once after the provider change.

The same probe used by the sampler then failed fast with Invalid class in approximately 748 ms:

$ErrorActionPreference = 'Stop'
Get-CimInstance Win32_PerfFormattedData_PerfProc_Process

Because the shipped worker sets $ErrorActionPreference = 'Stop' and issues the PerfProc query before the large PID-filtered detail query, the failure aborts the snapshot command before the expensive follow-up query runs.

After one complete 39-second observation window:

  • system-wide taskkill.exe count: 0;
  • maximum sampled WMI-host CPU: 0.05% of the whole machine;
  • no renewed WMI runaway was observed.

A later live check on the same active Codex session still showed:

  • taskkill.exe: 0 system-wide;
  • PerfProc: disabled;
  • no WMI CPU recurrence.

This is a strong A/B result that isolates the child-process snapshot path as the trigger for the WMI CPU component. It does not prove that every taskkill.exe cleanup defect has the same internal cause; the cleanup backlog and the WMI sampler should remain separately fixable defects.

Important limitation

Disabling PerfProc is a machine-wide, reversible diagnostic workaround, not a product-level fix and not a suitable general recommendation. It makes legacy per-process performance-counter/WMI classes unavailable. It can be restored with:

lodctr /e:PerfProc

The installed Codex package was not modified. The upstream sampler still runs and still needs a real fix.

Requested upstream changes

  1. Add a supported emergency switch to disable child-process snapshot sampling on Windows.
  2. Replace repeated PowerShell/WMI full-machine polling with a persistent native helper, Toolhelp/process handles, or Windows Job Objects scoped to Codex-owned descendants.
  3. On timeout, cancel and await the underlying WMI work rather than only terminating the client process.
  4. Enforce single-flight/backpressure, a hard concurrency cap, deduplication, and exponential backoff.
  5. Bound the PID set and avoid giant OR filters against Win32_Process.
  6. Add a Windows release-gate soak test that asserts bounded WMI CPU, process births, cleanup latency, and helper counts under focus changes, long tasks, cancellation, and large descendant trees.

Claim boundary

  • Confirmed: Codex Desktop starts the matching PowerShell/WMI sampler; the current build still contains it; the failure reproduced on the current build; blocking its first expensive PerfProc query eliminated the observed WMI runaway and taskkill accumulation during the validation windows.
  • Strong inference: the 5-second client timeout without effective cancellation/backpressure allows WMI work to overlap after clients exit, producing the 0x800706BA client-gone failures and runaway CPU.
  • Not claimed: disabling PerfProc is a root fix, or that all Codex child processes were zombies. Most of the earlier 216 descendants were live Codex/MCP/Node/Python components; the clearly abnormal residue was the accumulating cleanup helpers and overlapping WMI workers.
BIOcanse · 1 month ago

Temporary mitigation for affected Windows users: Codex Process Guard

For developers who already updated Codex Desktop and are hitting the taskkill.exe / conhost.exe process storm, I published a small independent emergency guard:

https://github.com/BIOcanse/CodexProcessGuard

It uses native Toolhelp snapshots plus PID creation-time and ancestry validation to contain only verified Codex-owned cleanup helpers and directly owned Git processes. It does not patch Codex or eliminate the upstream producer bug; it limits the runaway resource storm so affected developers can keep using Codex while OpenAI prepares a proper fix.

The current release supports Windows 10/11 x64, requires no administrator rights or separate .NET installation, and fails closed if the expected Codex package ancestry cannot be verified. Please read the README and verify the published SHA-256 before running the unsigned binary.

Latest release: https://github.com/BIOcanse/CodexProcessGuard/releases/latest

ZoerZhu · 1 month ago

Diagnostic update: the Git child-process timeout precedes the taskkill / WMI storm

I need to correct an earlier working hypothesis: clearing the sidebar/workspace history does not fix this problem. I rolled back that isolation change and restored the original state. The state database passes quick_check, and the failure still reproduces even when startup work is reduced to only one or two repositories.

Testing again with Codex Desktop 26.715.4045.0 narrows the failure boundary further:

  1. The Desktop UI itself becomes visible in roughly 2–3 seconds.
  2. The background sidebar_workspace_task_groups_task_dirs / Git-origin discovery work starts.
  3. Git commands launched through the Desktop process environment hang until a fixed 60,000 ms timeout.
  4. The same commands, against the same repositories, finish in 31–152 ms when run directly from PowerShell. A Node execFile control test also completes normally.
  5. After the timeout, Desktop invokes:

``text
taskkill.exe /pid <pid> /t /f
``

  1. Concurrent timeouts, cancellations, retries, and process-tree cleanup then produce the taskkill → WMI process enumeration → further cancellation storm described in the original report. Eventually internal RPCs and the main window stop responding.

Additional exclusion tests:

  • Reducing sidebar/history data lowers the number of Git scans, but the remaining one or two scans still hang for the full 60 seconds. History size is an amplifier, not the initial cause.
  • The repositories are healthy; the same Git commands complete normally outside Desktop.
  • Direct Win32_Process / performance-process WMI queries complete normally outside Desktop (roughly 0.7–1.0 seconds). WMI amplifies the cleanup storm but is not the initial trigger.
  • This is not caused by auto-review or approval configuration.
  • There is no documented configuration switch that fully disables the startup Git-origin scan.
  • Other app-server requests such as model/config/plugin status reads also become delayed by approximately 60 seconds once the failure begins, consistent with a blocked process-management/RPC path.

Revised working diagnosis

The evidence now points to a Windows-specific defect in the Desktop wrapper's Git child-process lifecycle path: spawn, stdout/stderr completion, cancellation, or reaping. A single Git subprocess that does not complete correctly is amplified by concurrent, unbounded taskkill /T cleanup into the visible process/WMI storm and application hang.

This is the narrowest boundary that can be established without OpenAI's source, private symbols, or an internal trace. The exact lock or failing code path still requires maintainer-side instrumentation.

Requested fix / instrumentation

Please investigate the Windows Desktop child-process wrapper used by Git-origin discovery, especially:

  • why a Git process that exits in milliseconds outside Desktop is observed as running until the 60-second timeout inside Desktop;
  • whether stdout/stderr or process-exit completion can be lost;
  • bounded concurrency and deduplication for repository discovery;
  • cancellation/reaping without launching an independent taskkill.exe tree for every timed-out command;
  • use of Windows Job Objects or another bounded process-tree cleanup mechanism;
  • telemetry around spawn time, child PID, pipe closure, exit notification, cancellation, and cleanup completion.

A full Windows restart temporarily clears the accumulated process/IPC state, so Desktop works for roughly 10–20 minutes before the cycle starts again. This explains why rebooting appears to help but does not resolve the defect.

I can provide sanitized timestamps and selected logs privately if maintainers need them.

luziyi123448-gif · 1 month ago

I implemented and isolated-tested a bounded cleanup change in the public Rust process paths, but I want to keep the claim narrower than the full Desktop/WMI report here.

Public comparison:

https://github.com/openai/codex/compare/main...luziyi123448-gif:codex/loop-death-fix?expand=1

The Windows portion of the branch:

  • assigns exec-server process trees to a Windows Job Object with KILL_ON_JOB_CLOSE;
  • closes the Job to reap surviving descendants;
  • keeps the legacy taskkill fallback no-window, bounded to two seconds, and single-flight;
  • trips a process-wide creation breaker on Windows error 164 so cleanup cannot amplify thread/resource exhaustion;
  • applies the same Job Object cleanup primitive to the rmcp-client stdio launcher.

The task-owned Windows regression starts a short-lived parent/child helper tree, verifies neither helper has a console window, closes the Job, waits for both exact process handles to exit, and asserts that the task-owned taskkill attempt counter did not increase. The error-164 injection test verifies that later process creation is suppressed after the breaker trips. Both tests pass on the rebased branch; the full codex-rmcp-client package also passes 109/109 tests.

Scope boundary: this does not modify or claim to fix the closed-source Desktop child-process snapshot worker, its PowerShell/WMI sampler, or every ChatGPT.exe cleanup producer discussed in this issue. It addresses the public exec-server/rmcp process-tree and fallback amplification paths. The compare also contains the separately reported completed-thread lifecycle fix from #33957; it can be split if maintainers invite a focused contribution.

Bulocka74 · 1 month ago

Additional Windows Codex Desktop diagnostics

Feedback/session ID: 019f7a60-2f25-7de2-be94-44b7ddb83f2b

Environment: Windows, Codex Desktop package 26.715.4045.0 (ChatGPT.exe file version 150.0.7871.124).

I reproduced the issue on Windows Codex Desktop by opening Codex and a project chat without entering a prompt or starting a task.

Confirmed correlated activity chain

ChatGPT.exe / codex.exe
-> repeated git.exe
-> repeated taskkill.exe
-> repeated Win32_Process WMI queries
-> sustained WmiPrvSE.exe CPU load

Across the completed ETW, WMI, and Process Monitor diagnostics, the measured values were:

  • Average WmiPrvSE.exe CPU: 18.6346%
  • Maximum WmiPrvSE.exe CPU: 35.7490%
  • Dominant repeated Git command: git.exe config --null --get core.fsmonitor
  • Dominant-command executions: 81
  • Average dominant-command interval: approximately 0.7267 seconds
  • Child git.exe processes: 171
  • Unique taskkill.exe PIDs: 153

The filtered Process Monitor data also showed repeated status, ls-files, diff, hash-object, rev-parse, and merge-base operations. The saved Process Monitor event timestamps span 58.257 seconds, although the intended manual capture duration was 30 seconds; intervals above were calculated from the observed timestamps.

Verdict and limitation

PARTIAL_CAUSE_CONFIRMED

The repeating Git operation and the tight temporal correlation with repeated taskkill.exe activity are confirmed. However, the selected Process Monitor filter did not retain the command-line arguments or target PID of taskkill.exe, so it does not yet prove that taskkill.exe was terminating those specific Git PIDs. The exact internal Codex component responsible for initiating the outer process chain is also not captured by this dataset.

Rejected hypotheses

The following were tested and rejected as necessary causes:

  • Git-Based Review mode
  • chat_processes.json
  • stale saved terminal states as a necessary cause

Most likely direction

The strongest remaining direction is a repository watcher or overlapping repository-state checks that lack single-flight protection or rate limiting. A per-repository single-flight guard would prevent a new Git state-refresh batch from starting while the previous batch is still active.

Related issue: #34005, which reports repeated git config --null --get core.fsmonitor activity.

Privacy and local artifacts

The raw PML is intentionally not being published because it may contain sensitive local metadata.

The following sanitized files exist locally but are not attached to this comment:

  • PROCMon-GIT-TASKKILL-REPORT.md
  • procmon-git-taskkill-safe.csv
liujianhao1226-beep · 1 month ago

Confirmation: same review-summary → git abort → taskkill storm on Windows Desktop 26.715.7063.0

This matches the chain described in #30820 / #33776 / #32113 / #27570. Local evidence below (no prompt content, no secrets).

Environment

  • OS: Windows 11, PowerShell
  • Codex Desktop package: OpenAI.Codex 26.715.7063.0 (OpenAI.Codex_2p2nqsd0c76g0)
  • Previously also reproduced on 26.715.4045.0; in-app update did not fix
  • Git itself is healthy in the affected repos (git status ~50–65 ms, no merge/rebase/index.lock, fsck --connectivity-only OK)

Symptom

  1. Open Codex Desktop → CPU rises quickly; Task Manager freezes/lags when refreshing process list
  2. Fully quit Desktop (window + tray) → process storm stops immediately
  3. Even when opening a clean worktree via codex://threads/new?path=..., background work still targets a previously saved local project in the sidebar

15s process sample (during active storm, post-update)

PID-diffing style sampler (unique short-lived PIDs over 15s):

| Metric | Value |
| --- | --- |
| Unique taskkill.exe | 262 (~17.5/s) |
| Unique git.exe | 226 (~15.1/s) |
| ChatGPT CPU seconds (15s window) | 5.88 |
| ChatGPT processes | 8 |

Baseline with Desktop fully exited: UniqueTaskkill=0, no ChatGPT/CodexMicroService.

Desktop log signature (exact fields)

Log root:

%LOCALAPPDATA%\Packages\OpenAI.Codex_2p2nqsd0c76g0\LocalCache\Local\Codex\Logs\

Repeated pattern:

requestKind=review-summary
source=review_model
failureReason=aborted
failureReason=abortedBeforeStart
failureReason=spawnFailed
failureReason=waitFailed
errorMessage="Git snapshot became stale"

Aborted Git subcommands observed:

  • git hash-object
  • git diff --no-index (per untracked file, e.g. scratch/*.py)
  • git diff --cached
  • git status
  • git diff against HEAD for a small set of tracked changes

Trigger cwd in current session (redacted base):

cwd=D:/AI/<local-project-A>

Also previously observed on a second local monorepo under the same parent folder.

Representative command shapes (same as other reports):

git -c core.hooksPath=NUL -c core.fsmonitor= hash-object -- <path>
git ... diff --no-index --raw --numstat -z -- /dev/null scratch/<file>.py
git ... diff --cached <HEAD> --no-renames --name-only -z

Also saw sidebar PR chip work against the same cwd:

requestKind=gh-pr-status
source=sidebar_task_pr_chip
cwd=D:/AI/<local-project-A>

Two-day log aggregation (local)

From recent non-empty Desktop logs:

  • review-summary-related Git complete lines: 4476
  • Early-cancel lines without cwd: 3706 (all failed/aborted)
  • One dirty monorepo cwd: 742 lines (710 failed/aborted)
  • Another dirty project cwd: 28 lines (all failed/aborted)
  • Git snapshot became stale: 55

Important negative findings (not root cause)

  • Not a merge/rebase conflict, index.lock, or corrupt object DB
  • Not continuous on-disk mutation: 10s mtime/size compare of dirty paths showed 0 changing files while snapshot still went stale
  • Not slow Git: git status ~65 ms in the hottest project
  • Defender / WMI / Task Manager behave as amplifiers/victims, not the primary producer (matches #33776 control experiment: kill ChatGPT → storm collapses)
  • Linked clean worktree for another project briefly looked healthy, then logs showed the storm was still scanning <local-project-A> restored from sidebar / local projects — clean worktree alone is not a durable fix

Global state / project registry shape

.codex-global-state.json contained multiple local-projects entries and a selected-project pointing at the dirty project. Desktop restored/scanned that project even when the user deep-linked into a different clean path. This aligns with #32113 (saved/inactive projects still scanned) and #30820 (startup restore → eager review-summary).

Workspace shape that amplifies (not required for the bug, but makes it severe)

Hot project approximately:

  • ~7 tracked modifications/deletions
  • large untracked tree under scratch/ (order of ~90+ files) plus other untracked dirs
  • .gitignore did not cover scratch/diagnostics dumps
  • No nested-repo fanout proven in this capture (unlike #27570), but untracked-per-file hash-object / diff --no-index is enough to feed the cancel/taskkill loop on Windows

A/B already tried (local)

| Attempt | Result |
| --- | --- |
| App update 26.715.4045 → 26.715.7063 | Still reproduces |
| Clean linked worktree deep-link | Current project quiet; other saved project still storms |
| Precise .git/info/exclude | Did not stop loop |
| git add -N intent-to-add | Did not stop; reverted |
| Fully exit Desktop | Storm stops immediately |

Request

Please treat this as another independent Windows confirmation on 26.715.7063.0 of:

  1. Eager/live review-summary / review_model over restored or saved local projects
  2. Per-file untracked Git fanout + abort
  3. taskkill /T /F cleanup feedback loop freezing process enumeration / Task Manager

Happy to validate a patched Desktop build quickly. Can also provide sanitized log excerpts on request.

athan614 · 1 month ago

Same issue here, ChatGPT/Codex windows app is stuck in a runaway process-enumeration loop with taskkill.exe whenever launched, causing WMI to get stuck at 25% CPU constantly, lagging my entire PC.

NI-SO · 1 month ago

+1

gbraunegg · 1 month ago

Same storm confirmed on Desktop 26.715.7063.0 (Windows 11 Pro 26200, 16-core)

Confirming the full chain on package OpenAI.Codex_26.715.7063.0_x64__2p2nqsd0c76g0
(codex.exe app-server launched with -c features.code_mode_host=true),
Windows 11 Pro build 26200. Symptoms: two WmiPrvSE instances (CIMWin32 and
WmiPerfInst provider hosts) at 83–88% of a core each, ~99% total system load
while an agent turn executed a test suite.

Measured spawn rate

PID-diffing Get-CimInstance Win32_Process every 300 ms during a single
active agent turn: 235 newly observed processes in 25 seconds (lower
bound), all churn parented by ChatGPT.exe. Captured command lines:

  • Multiple concurrent copies of:

powershell.exe -NoProfile -NonInteractive -Command "Get-CimInstance Win32_Process | Select-Object ProcessId,ParentProcessId | ConvertTo-Json -Depth 2"

  • A per-PID CPU sampler enumerating all perf-counter instances:

Get-CimInstance Win32_PerfFormattedData_PerfProc_Process ... followed by a
Win32_Process query with an OR-filter of ~60 PIDs

  • taskkill.exe /pid <helper> /t /f on each helper afterwards
  • Periodic `git.exe -c core.hooksPath=NUL -c core.fsmonitor= status --no-renames

--ignored=matching --untracked-files=normal --porcelain=v1 -z -- <watched files>`
polling files that legitimately change every ~30 s (service heartbeat JSON
files inside the workspace) — i.e. the repo-watch invalidation feeding the
loop is continuous by design in any workspace containing runtime state files.

WMI-Activity/Operational shows dozens of distinct short-lived ClientProcessIds
each issuing IWbemServices::CreateInstanceEnum - root\cimv2 : Win32_Process
(5858), matching the stale-client pattern in the OP.

Control experiments (both repeated twice)

  1. Killing ChatGPT.exe: total CPU dropped from ~99% to 8% within

seconds; WmiPrvSE to ~0; helper spawn rate to zero. Matches
@mweichse's control result.

  1. Untracked-file hygiene: sweeping the open workspaces to ~0

untracked-not-ignored files eliminated the git hash-object fan-out
(previously one process per untracked file — 51 untracked files in one
nested repo was enough to sustain churn when combined with concurrent
working-tree changes), but did not prevent the exec-monitoring /
snapshot-worker storm above. The two producers are independent; hygiene
only mitigates the hash-object component.

Notes

  • In-app update did not help (consistent with reports on 26.715.4045.0).
  • Headless codex exec (CLI, same machine, same repos) exhibits none of

this — no snapshot worker, no taskkill churn. The loop is Desktop-host only,
consistent with @BIOcanse's bundle analysis.

  • +1 to the Job-Object / bounded-cleanup remediation direction, and to

scoping process sampling to owned children instead of full-machine
Win32_Process + PerfProc enumeration.

suiyuebaobao · 1 month ago

The taskkill.exe/conhost.exe and WMI storm here is very close to the downstream behavior I measured. In my reproduction, repeated desktop Git refreshes were upstream, so this workaround is worth testing if the same parent chain is present.

I published a single-file Windows launcher for the original Microsoft Store/MSIX Codex app:

It starts Codex with a process-scoped GIT_DIR=NUL. On my machine, the stable trace changed from 247 new git.exe and 137 new taskkill.exe processes per 10 seconds to 0 and 0 over 15 seconds; WMI Provider Host fell from about 6.6% to 0.9%.

Usage: fully exit Codex from the Windows system tray, then run the script.

This is an unofficial workaround, not an upstream fix. It deliberately disables or limits Review, Changes, branch detection, commit, and other Git-backed Codex features. If the process storm in this issue has a different upstream cause, this may not apply. The launcher does not patch Codex, modify repositories or Git configuration, write system environment variables, require elevation, terminate processes, or remove chat history. Launching Codex normally restores Git integration.

MrUncleXiang · 1 month ago

Additional reproduction on Windows Desktop 26.715.9079.0

I reproduced the same taskkill.exe / conhost.exe process storm on a newer Codex Desktop build.

Environment
  • Windows 11 Pro for Workstations, build 26200, x64
  • Intel Core i7-12700
  • 32 GB RAM
  • Codex Desktop package: OpenAI.Codex_26.715.9079.0_x64
  • Bundled ChatGPT.exe: 150.0.7871.124

The machine had sufficient available RAM and no sustained disk queue at the beginning of the investigation.

Severe affected-state evidence

During a local Codex turn invoking read-only shell diagnostics:

  • One snapshot showed 15 live taskkill.exe processes and 22 conhost.exe processes.
  • A later sample showed 37 taskkill.exe and 46 conhost.exe.
  • Three seconds later, the counts changed to 22 and 30 respectively, showing continuous process creation and termination rather than a static residual set.
  • Windows process enumeration became unreliable. tasklist returned ERROR: Out of memory.
  • Even a trivial new cmd.exe process timed out while the operating system was stuttering.

In an earlier affected interval, the WMI Activity log recorded:

  • 844 Event ID 5858 failures involving Win32_Process
  • All 844 events occurred within approximately 70 seconds
  • 625 unique short-lived client PIDs were recorded

This is consistent with a high-rate short-lived process and WMI query storm.

Controlled exit/relaunch result

The user performed the following control test:

  1. Fully exited Codex Desktop.
  2. Windows responsiveness recovered and Task Manager showed a substantial resource reduction.
  3. Relaunched Codex Desktop.
  4. System resource usage increased substantially again.

This strongly associates the system-wide slowdown with the active Codex Desktop session.

Result after updating

Codex was updated from 26.715.8383.0 to 26.715.9079.0 and restarted.

The behavior still reproduced.

During a single diagnostic process, no additional shell command was started for 20 seconds:

| Time | taskkill.exe | conhost.exe |
|---|---:|---:|
| Start | 4 | 13 |
| 20 seconds later | 6 | 15 |

Therefore, cleanup-process activity was still continuing after the update.

At this early point after relaunch, the severe WMI failure storm had not yet returned, but the taskkill.exe / conhost.exe churn was already present.

Confounding factors checked
  • A third-party Codex history maintenance tool was not running.
  • It had no active startup entry.
  • No new ChatGPT.exe crash or application-hang event was required to reproduce the performance problem.
  • CPU, RAM and disk capacity alone did not explain the immediate exit/relaunch correlation.
Current workaround

Fully exiting Codex Desktop restores system responsiveness. Reinstalling or updating to 26.715.9079.0 did not resolve the underlying process-cleanup behavior.

No raw logs, usernames, local paths, repository names, prompts, conversation contents, account information or database files are included in this report.

Please let me know if sanitized Event ID 5858 samples or additional bounded process-count measurements would be useful.

KarmCraft · 1 month ago

Additional sanitized observation from another affected Windows system, extending the process/WMI evidence above.

Environment

  • Windows 11 x64, build 22631.7376
  • 32 logical processors
  • Codex Desktop Microsoft Store package: 26.715.8383.0
  • Bundled ChatGPT.exe: 150.0.7871.124

New signal: Program Compatibility Assistant co-spikes with WMI

During a system-wide UI stall, Task Manager showed:

  • WMI Provider Host: 31.3% whole-machine CPU (roughly 10 logical cores)
  • Service Host: Program Compatibility Assistant: 8.8%
  • Desktop Window Manager: 4.6%
  • Total system CPU: 57%

The PcaSvc host was simultaneously at approximately 394 MiB working set and 445 threads. Later in the same Codex session it fell to 97 threads but retained approximately 343 MiB. This looks bursty and may represent a secondary compatibility/cleanup amplification path rather than the root trigger.

A later point-in-time process snapshot contained:

  • 51 ChatGPT.exe processes: approximately 5.87 GiB aggregate working set and 1,963 threads
  • 32 conhost.exe processes
  • 4 taskkill.exe processes

Compatibility and crash-log correlation

Read-only Windows event-log review found:

  • 91 ChatGPT.exe Application Error events clustered across July 15–16, all with exception 0xc06d007f
  • Program Compatibility Assistant events for ChatGPT.exe involving DetectorShim_KernelDriver (4 events, July 18–21) and WrpKeyMitigation (2 events, July 20–21)
  • In the latest 24-hour WMI Operational window: 166 provider-start events and 76 Event ID 5858 failures; the recurring result was 0x80041032 from short-lived client PIDs

The exception code was already noted in the original report; the new part here is the crash count plus the direct, simultaneous PcaSvc resource growth and PCA mitigation events. This suggests checking whether the repeated helper termination/process-snapshot loop also drives Windows Program Compatibility Assistant work or leaves PCA resources behind.

No usernames, paths, repository names, prompts, credentials, or raw logs are included.

zhaojunmeng · 1 month ago

Additional root-cause evidence: RPC stdin disposal is the dominant taskkill producer

During the affected state, Codex Desktop repeatedly started short-lived, read-only Git processes followed by taskkill.exe /pid <Git PID> /t /f. Those taskkill processes issued Win32_Process queries, and WmiPrvSE CPU rose under the resulting call volume.

The following runtime evidence identifies why a normally completed, read-only Git RPC enters process-tree cleanup.

Reproduction
  • Application: Codex Desktop 26.715.10079.0 on Windows.
  • A Git working tree was prepared with 100 small, non-ignored untracked files.
  • Desktop was fully exited before each capture. The relevant instrumentation or ETW session was armed before relaunch, and Desktop was then allowed to restore and scan the workspace normally.
  • No repeated manual cancellation or deliberately delayed Git command was introduced. The process burst began during normal startup and workspace refresh.
Method

The Desktop main process was instrumented at three boundaries in one continuous reproduction:

  1. Git process spawn, including the supplied AbortSignal and killProcessTree option.
  2. RPC WritableStream stub disposal, including its reference count and closed state.
  3. Entry into killProcessTree, including the JavaScript call stack and whether it came from stdin.abort() or an explicit process kill().

A separate focused capture correlated individual RPC-stub disposal events with killProcessTree entries. Kernel-process and WMI ETW events were also correlated by process identity.

Results
  • 316 Git processes were spawned. All 316 had no AbortSignal.
  • killProcessTree ran 314 times.
  • 312 of those 314 calls (99.36%) came through the stdin WritableStream.abort() stack.
  • Only 2 calls came from an explicit process kill().
  • In the focused capture, 149 RPC stdin-stub disposal events corresponded to 149 killProcessTree calls. Every disposed stub had refcount == 0 and closed == false.
  • Only one review-subscription abort occurred. Of the 312 automatic stdin-abort events, 64 occurred before that abort and 248 after it. The repeated cleanup path was therefore already active independently of the subscription abort.
  • In the ETW capture, 223 taskkill processes mapped to 223 Win32_Process queries issued by those processes.

The 149-to-149 focused correlation and the captured call stacks distinguish this from temporal correlation between Git activity and taskkill: RPC stdin disposal directly entered the process-tree cleanup function.

Control flow

The relevant bundled-code behavior, with identifiers simplified, is equivalent to:

// Local child-process wrapper
const cleanup = () => killProcessTree(child);

return {
  stdin: new WritableStream({ write, close, abort: cleanup }),
  kill: cleanup,
};

// RPC WritableStream stub disposal
if (--state.refcount === 0 && !state.closed) {
  state.writer.abort(
    new Error("WritableStream RPC stub was disposed without calling close()")
  );
}

For these Git calls, killProcessTree is enabled. On Windows, that cleanup starts:

taskkill.exe /pid <Git PID> /t /f

The resulting path is:

read-only Git RPC exports stdin
-> the caller neither writes to nor closes stdin
-> the last RPC reference is released with closed == false
-> RPC stub disposal calls writer.abort()
-> stdin.abort invokes the shared process cleanup callback
-> killProcessTree starts taskkill /t /f
-> taskkill issues a Win32_Process query
-> WmiPrvSE CPU rises under high call volume

This stdin is the RPC-exposed child-process input stream. It is unrelated to Git's hash-object --stdin-paths batching option mentioned elsewhere in this issue.

Interpretation and claim boundary

In this reproduction, the dominant producer was not repeated Git timeout or review cancellation. Normal disposal of an unclosed stdin RPC stub was converted into process-tree termination, even for read-only Git commands that did not need stdin.

Repository refreshes and per-file operations still amplify the incident by creating more Git RPCs, and other cancellation or timeout paths may exist. However, neither repeated cancellation nor timeout is required to explain the measured taskkill volume above.

LinkiCangon · 1 month ago

Promising workaround: disable the sandbox private desktop

TL;DR: After a Windows reboot cleared an existing bad state, I added sandbox_private_desktop = false and fully restarted Codex. The Windows-wide DWM/input degradation did not recur during approximately 8 hours 52 minutes of normal work—well past this machine’s usual approximately four-hour failure window.

Cleanup activity still appeared briefly, so this should be treated as a compatibility mitigation, not a confirmed fix.

Workaround
[windows]
sandbox = "elevated"
sandbox_private_desktop = false

After changing the setting, fully exit and reopen Codex.

I verified that subsequent sandboxed commands ran on WinSta0\Default instead of the private desktop, while sandbox = "elevated" remained configured.

Tradeoff: this removes the additional private-desktop UI-isolation layer. It should be used only when needed for compatibility and reverted once the underlying issue is fixed.
Result

After approximately 8 hours 52 minutes of continued work:

  • No Windows-wide window-dragging or input lag recurred.
  • A 59-minute low-duty monitor sampled DWM for three seconds once per minute.
  • Mean DWM CPU across the 59 points was 7.58%; median was 6.7%.
  • The maximum per-minute three-second average was 22.2%.
  • The first 15-minute median was 7.7%; the last 15-minute median was 6.2%.
  • DWM handles, working set, and private memory showed no sustained growth.

The monitor caught two transient process-count samples:

  • 9 taskkill.exe / 20 conhost.exe
  • Later, 2 taskkill.exe / 12 conhost.exe

Both returned to the normal 0 / 9 by the next sample. Cleanup activity therefore still occurred, but it did not accumulate or coincide with sustained DWM degradation during this run.

Previous failure state

Environment:

  • Windows 10 Home 22H2, build 19045.6466
  • Codex Desktop 26.715.10079.0

Using the Windows \Process(dwm)\% Processor Time counter, where 100% is approximately one logical core:

  • During one severe episode, DWM averaged 108.4% over 10 seconds.
  • WMI provider processes averaged 20.4%.
  • Fully exiting Codex did not clear the degradation.
  • While all ChatGPT.exe and codex.exe processes were absent, DWM still averaged 73.8%.
  • After reopening Codex, DWM averaged 99.2%.
  • Restarting Windows restored DWM to a 7.9% average, and window dragging/input immediately returned to normal.

I did not capture taskkill.exe/conhost.exe counts during that pre-reboot failure, so I cannot confirm that it was exactly the same cleanup-process storm reported in this issue.

Scope and limitations

This is a single-machine before/after observation. The Windows reboot and workload/session differences are confounding variables, and I have not performed a reverse test by re-enabling the private desktop.

Codex-local renderer/state warnings and occasional brief pauses when switching threads were also not eliminated. The observed improvement was specifically that the previous Windows-wide DWM/input degradation did not return.

I would describe this as a promising mitigation—not proof that the private desktop is the root cause. I can provide a sanitized monitoring CSV or selected log excerpts if useful.

JOY · 1 month ago
sandbox_private_desktop = false

Tried today and it actually worked after a few minutes with 100% CPU load. It has been 3 hours so far.

sanzhang2 · 1 month ago

After updating the ChatGPT Windows app to version 26.721.3996.0, the issue appears to be resolved on my machine. Previously, enabling Remote Control caused multiple codex.exe processes to trigger sustained WmiPrvSE.exe CPU usage of around 20–30%. Since the update, WmiPrvSE.exe has remained consistently below 15% CPU usage, and the system is responsive again. I will continue monitoring it, but so far the new version seems to have fixed the problem.

bbingz · 1 month ago

Analysis (community)

Narrow residual only for the OSS goal / rollout-scan path that compounds large-thread pain on Windows Desktop threads such as #33776 (and related #34929). Under docs/contributing.md — invitation-only. No unsolicited PR.

Important boundary

This comment does not re-litigate the Electron taskkill / WMI storm. That loop is closed Desktop / app.asar territory and is not caused by the OSS path below. The OSS residual is a second, independent cost center: goal set/clear can unconditionally rescan the full legacy JSONL rollout even when SQLite metadata already exists.

Root-cause hypothesis

thread/goal/set and thread/goal/clear always call reconcile_thread_goal_rollout before mutating the goal store. That helper always invokes reconcile_rollout(..., builder=None, items=[]). On the empty-input branch, reconcile_rollout runs metadata::extract_metadata_from_rollout first and only afterwards looks up an existing SQLite thread row — so a fully materialized row cannot short-circuit the JSONL scan.

Verified at HEAD 4c43465133:

1. Caller always reconciles first.
codex-rs/app-server/src/request_processors/thread_goal_processor.rs:

  • thread_goal_set_inner → reconcile_thread_goal_rollout then set_thread_goal
  • clear path repeats the same order

2. Helper always full-reconciles with empty items.
reconcile_thread_goal_rollout resolves the rollout path, then:

reconcile_rollout(Some(state_db), path, provider, builder=None, &[], ...)

3. Extract-before-lookup in state_db.
codex-rs/rollout/src/state_db.rs reconcile_rollout empty-input branch: extract_metadata_from_rollout runs before ctx.get_thread(metadata.id). An existing metadata row therefore does not prevent a multi-GB JSONL reread. Community forensics (e.g. ~4 GiB rollout, sustained high codex.exe read bandwidth after WMI workarounds) fit this chain even when the taskkill storm is separately mitigated.

4. Fix surface is small and OSS.
StateRuntime / DB already expose thread row lookup. A pre-extraction fast path (“if metadata row present → mutate goal in SQLite; else slow reconcile”) is structurally available.

Ranked fix outline (OSS only):

  1. Before extract_metadata_from_rollout, probe SQLite for the thread id; if present, skip full JSONL scan for goal set/clear.
  2. Reserve full-rollout reconciliation for migration / missing-row repair only.
  3. Metrics: count full scans vs fast path; log rollout size when scan runs.
  4. Fail-first test: fixture with large JSONL + existing DB row → goal set does not re-read full file.

Again: please keep WMI/taskkill diagnosis on the Desktop side; this is the goal reconciliation residual only. Happy to draft the fast-path patch if maintainers want it. No unsolicited PR.

lilvli · 29 days ago

I observed a very similar system-wide rendering/stuttering issue on Windows.

Environment:

  • Affected Codex/ChatGPT Desktop build: OpenAI.Codex 26.721.4979.0, x64, Microsoft Store
  • Windows: x64, version 25H2, build 26200.8875
  • GPU 1: AMD Radeon 780M Graphics, driver 32.0.21030.2001
  • GPU 2: NVIDIA GeForce RTX 4060 Laptop GPU, driver 32.0.15.9159
  • OrayIddDriver virtual display device is also present

Symptoms:

  • After using Codex Desktop for an extended period, Windows developed severe system-wide visual stuttering.
  • The most obvious symptom was that dragging any application window became extremely choppy.
  • Closing the Codex/ChatGPT app did not restore normal rendering performance.
  • Restarting Windows immediately restored smooth window movement and normal responsiveness.
  • Before the restart, ChatGPT/Codex-related processes were using roughly 2 GB of memory, but the system still had approximately 13.5 GB of available RAM, so this did not appear to be ordinary memory exhaustion.

After restarting Windows, the app updated to OpenAI.Codex 26.721.11231.0. I have not yet confirmed whether the problem also reproduces on this newer build.

No project contents, conversation data, or private paths are included in this report.

meza · 26 days ago

Adding to the performance issues, I've managed to capture a trace purely minutes after starting the app. It has slowly become unusable over the past few days: 019fbd54-7b55-71d1-b2a2-d083a7e5c65c

henkosch · 26 days ago

Independent occurrence with DWM handle accumulation and Windows UI degradation

I observed a similar, although not identical, failure mode on another Windows system during extended Codex Desktop use.

All measurements below were collected locally during this incident. Values from the original issue are not repeated as if they were local measurements.

User-visible impact

After long-running Codex Desktop use, Windows UI responsiveness degraded progressively.

The most severe symptom was window movement: dragging ordinary application windows with the mouse became extremely slow and jerky, making the computer difficult to use.

Additional symptoms included:

  • slow or unreliable window rendering and switching;
  • intermittent difficulty loading or interacting with the Codex remote-thread UI;
  • the background Codex agent could continue running commands and modifying files even while the desktop UI was difficult to operate;
  • previous occurrences were only reliably cleared by restarting Windows.

This was gradual degradation rather than an immediate application or operating-system crash.

Environment

  • Windows 10 x64, build family 19045
  • 32 GB physical RAM
  • 12 logical processors
  • Codex Desktop package version: 26.721.4979.0
  • Bundled ChatGPT/Chromium executable version: 150.0.7871.128
  • Approximately 74 hours of system and application uptime at inspection
  • Native Windows workspace
  • One or more long-running, tool-heavy Codex sessions

No username, computer name, repository name, project information, file-system path, prompt, credential, or description of the agent's work is included here.

System responsiveness at inspection

The machine was not generally unresponsive and command execution was still fast:

  • Get-Process: approximately 77 ms
  • git status: approximately 183 ms
  • root-directory listing: approximately 63 ms
  • post-Explorer-restart verification command: approximately 82 ms

Approximately 10 GB of the installed 32 GB RAM was still available before closing VS Code. The system drive had approximately 46% free space.

A two-second CPU sample showed active Node.js worker processes as the primary CPU consumers, consistent with ongoing build or test activity. Codex itself consumed approximately 1% of total CPU during that sample.

This suggests that the desktop degradation was not caused by general CPU saturation, complete physical-memory exhaustion, slow command execution, or a full system drive.

DWM measurements

During the affected state, dwm.exe showed:

  • approximately 54,600–54,700 handles;
  • approximately 605–606 MB working set.

Representative readings collected during troubleshooting were:

  • initial inspection: approximately 54,682 handles;
  • after force-closing VS Code: approximately 54,672 handles;
  • after restarting Explorer: approximately 54,648 handles;
  • after the unsuccessful DWM termination attempts: approximately 54,697 handles.

The DWM handle count remained approximately stable during a short 12-second observation window. This indicates accumulated state rather than a rapidly increasing handle count at the exact moment of inspection.

No healthy post-reboot DWM baseline was recorded during this incident, so this report does not claim a measured before-and-after growth rate.

Application memory

Before mitigation:

  • Codex process group: approximately 2.0 GB combined working set and 2.1 GB combined private memory;
  • ChatGPT and ChatGPT Classic processes: approximately 5.1 GB combined working set and 7.3 GB combined private memory;
  • VS Code process group: approximately 4.74 GB combined working set;
  • WSL/Docker vmmem: approximately 2.3 GB working set.

Windows Error Reporting had previously recorded RADAR_PRE_LEAK_64 entries for:

  • ChatGPT Classic.exe;
  • a node.exe process.

These WER entries are reported as diagnostic observations. They suggest that Windows detected abnormal resource growth, but they do not by themselves establish the root cause of the DWM handle accumulation.

WMI evidence

The Microsoft-Windows-WMI-Activity/Operational event log contained 33 Event ID 5858 failures during the six hours preceding inspection.

Representative failures queried:

Win32_PerfFormattedData_PerfProc_Process

and ended with:

0x800706BA – Could not send status to client

This WMI error signature resembles the one discussed in the original issue, but the local process snapshot was different.

At the time of local inspection:

  • there was no active taskkill.exe accumulation;
  • 12 conhost.exe processes were present;
  • no evidence was collected showing hundreds of Codex-owned taskkill.exe or conhost.exe processes.

Therefore, this report does not claim that the exact subprocess-cleanup loop described in the original issue was active on this machine. It may be a related residual state or a different path producing similar WMI and DWM symptoms.

Display and event-log checks

No corresponding evidence was found for:

  • a DWM crash;
  • a display-driver reset;
  • DXGI device loss;
  • GPU out-of-memory failure;
  • a resource-exhaustion crash at the time of inspection.

The absence of these events does not rule out Chromium renderer, GPU-surface, handle, or other desktop-resource retention.

Mitigation attempts

Force-closing VS Code

The complete VS Code process group was force-closed with the user's authorization.

Results:

  • 24 Code.exe processes terminated;
  • no Code.exe process remained;
  • available physical memory increased from approximately 10.04 GB to 14.22 GB;
  • approximately 4.18 GB of physical memory was released;
  • Codex, ChatGPT, and the active background agent remained running;
  • DWM remained at approximately 54,672 handles and 605 MB.

This shows that VS Code accounted for substantial memory usage but was not solely responsible for the retained DWM state.

Attempting to restart DWM

The exact running DWM process was targeted with:

  • elevated Stop-Process -Force;
  • Windows taskkill /F.

Both attempts returned Access denied.

The DWM process remained running and responsive. No SYSTEM-level debugger, protected-process bypass, or other unsafe workaround was attempted because the computer was unattended and losing the graphical session would have been unacceptable.

Restarting Windows Explorer

explorer.exe was terminated and explicitly restarted.

Results:

  • old Explorer PID: 9576;
  • new Explorer PID: 21144;
  • Explorer working set decreased from approximately 169 MB to 54 MB;
  • Explorer handle count decreased from 2,973 to 655;
  • the new Explorer process responded normally;
  • Codex and ChatGPT remained running;
  • DWM remained effectively unchanged at approximately 54,648 handles and 606 MB.

Restarting Explorer successfully refreshed the Windows shell but did not release the DWM resources.

Full restart

A full Windows restart was not performed during this diagnostic session because a long-running background task was still active.

Historically, restarting Windows has been the only reliable way to restore normal window movement and desktop responsiveness after the degradation becomes severe.

Reproduction pattern

The approximate pattern observed across occurrences is:

  1. Start Codex Desktop on Windows.
  2. Keep one or more tool-heavy Codex sessions active for multiple days.
  3. Repeatedly use shell tools, including concurrent commands, commands with substantial output, cancellations, or timeouts.
  4. Continue working while Codex Desktop, ChatGPT/Chromium renderer processes, Node.js workers, and related development tools remain running.
  5. Windows window movement, switching, rendering, or remote-thread loading gradually becomes slow or unreliable.
  6. Inspect dwm.exe and observe unusually high retained handle and memory values.
  7. Closing large applications or restarting Explorer may release their own resources but does not necessarily restore DWM.
  8. Restarting Windows restores normal behavior temporarily.

The exact operation that initiates the resource retention has not been isolated.

Current assessment and limitations

The local evidence establishes that:

  • DWM retained approximately 54,600 handles and 605 MB working set;
  • long-running Codex/ChatGPT and development-tool processes retained several gigabytes of memory;
  • WER recorded RADAR_PRE_LEAK_64 for ChatGPT Classic and Node.js;
  • WMI produced repeated Event ID 5858 failures;
  • closing VS Code and restarting Explorer did not materially reduce DWM resource usage;
  • Windows UI responsiveness, especially mouse-driven window movement, became severely degraded.

The evidence does not yet establish whether the root cause is:

  • Codex process cleanup or cancellation;
  • ChatGPT Classic or another Chromium renderer;
  • retained graphics or GPU surfaces;
  • Node.js subprocesses;
  • a VS Code extension;
  • a display-driver interaction;
  • or an interaction between several long-running Electron applications.

The similarity to this issue is significant, especially the WMI failures, elevated DWM resources, progressive Windows UI degradation, and recovery after reboot. However, the absence of an active local taskkill.exe storm means the exact same cleanup defect has not been proven.

Expected behavior

Long-running Codex Desktop sessions should not:

  • cause unbounded DWM handle or working-set retention;
  • leave renderer, app-server, console, or process-query resources alive after tool calls complete;
  • cause repeated failed WMI queries against stale or exited processes;
  • degrade mouse-driven window movement or general Windows UI responsiveness;
  • require a complete Windows restart to restore usability.

A supported way to restart Codex renderers and the app-server while preserving thread state would also reduce the impact of this failure mode.

Diagnostic questions

  1. Can Codex Desktop retain process handles, WMI clients, console resources, or Chromium window surfaces even after the originating cleanup subprocesses have exited?
  2. Can a long-running or repeatedly compacted thread keep renderer or app-server resources alive indirectly?
  3. Does Codex Desktop version 26.721.4979.0 include any mitigation for the process-cleanup problem tracked in this issue?
  4. Which counters or logs would best distinguish subprocess-cleanup leakage from Chromium/GPU-surface retention?
  5. Is there a supported way to restart the Codex app-server and renderer processes without terminating active background work or losing thread state?

Feedback correlation

The same incident was submitted through Codex's in-app feedback mechanism.

Feedback session ID:

019fbd40-b02b-7772-b143-122169a00c36

Sanitized counter output and selected Windows event records can be provided if maintainers request them.

LinkiCangon · 26 days ago

Follow-up: after re-enabling sandbox_private_desktop = true on Microsoft Store package 26.727.6591.0, the same Windows-wide DWM/window-dragging degradation began to return during a long session, so the issue does not appear to be fixed in this build.

chess-oai contributor · 23 days ago

thanks for the detailed reports and investigation here: they were really helpful 💖

  • we’ve shipped fixes in 26.730.7989.0 that address some major sources of powershell/wmi storms
  • taskkill.exe can still be started by other process-cleanup paths, and we’re continuing to work on reducing that separately

if you’re still seeing sustained wmi cpu usage or system-wide lag after 26.730.7989.0, please share an updated /feedback id

bbingz · 22 days ago

Thanks for the update @chess-oai — confirming the boundary from my earlier comment so the OSS half does not get closed alongside the Desktop half.

My residual was explicitly not the powershell/WMI storm. It is the goal-reconciliation path, which can rescan the full legacy JSONL rollout on every thread/goal/set and thread/goal/clear even when a SQLite metadata row already exists. Re-verified on main today, still unremediated:

  • codex-rs/app-server/src/request_processors/thread_goal_processor.rs:130 and :248 still call reconcile_thread_goal_rollout before set_thread_goal (:143).
  • codex-rs/rollout/src/state_db.rs reconcile_rollout (:516) still runs extract_metadata_from_rollout (:543) before ctx.get_thread(metadata.id) (:556), so a fully materialized row cannot short-circuit the scan.

On multi-GB rollouts this surfaces as sustained codex.exe read bandwidth that survives the WMI mitigations — meaning some reporters may still see lag after 26.730.7989.0 from a different cause than the one you fixed, and their /feedback ids would point at disk I/O rather than WMI. Worth keeping in mind when reading the follow-ups you get.

The fast path is small: probe SQLite for the thread id before extraction, and reserve full-rollout reconciliation for migration / missing-row repair. Happy to draft it with the fail-first test (large JSONL + existing DB row → goal set does not re-read the file) if that would be useful — still no unsolicited PR.

chess-oai contributor · 21 days ago

@bbingz we have a fix en route for that one as well

bbingz · 21 days ago

@chess-oai thanks for the update — good to hear a fix is en route for the ChatGPT.exe taskkill/conhost storm. Standing by for the release notes; happy to re-verify on Windows once it ships.

grtninja · 3 days ago

Current-main source follow-up: hook cleanup still has an unreaped taskkill fallback

I checked current public source at 77b30a21e17d6ec6317f712ab6d044cb1bd152e7 because a separate Windows long-running Codex incident reached 498 processes / 8,304 threads / 239,618 system-wide handles and later Win32 1816 process-creation failures.

There is a direct source path that matches this report's taskkill.exe / conhost.exe accumulation pattern:

codex-rs/hooks/src/engine/command_runner.rs

impl Drop for ProcessTreeGuard {
    fn drop(&mut self) {
        ...
        #[cfg(windows)]
        {
            if let Some(job) = self.job.as_ref() {
                let _ = job.terminate();
            } else {
                let _ = std::process::Command::new("taskkill")
                    .args(["/PID", &process_id.to_string(), "/T", "/F"])
                    .stdin(Stdio::null())
                    .stdout(Stdio::null())
                    .stderr(Stdio::null())
                    .spawn();
            }
        }
    }
}

The fallback cleanup child is spawned from a Drop implementation and its handle is immediately discarded. There is no wait(), no bounded timeout/reap, and no postcondition that the cleanup subprocess itself exited. On Windows, taskkill.exe can in turn acquire a console host. Repeated fallback cleanup therefore has a plausible direct path to the exact process family reported here.

A second lifecycle detail in the same function is also worth auditing:

// Successfully completed hooks may intentionally leave detached helpers running.
if let Some(job) = process_tree_guard.job.as_ref() {
    let _ = job.preserve_descendants();
}
process_tree_guard.process_id = None;

Preserving descendants may be intentional for hooks whose contract is to launch a resident helper, but the generic successful-hook path does not visibly bind those preserved descendants to a durable owner/lease or later retirement path here. If ordinary hook commands can create helpers accidentally, this can turn successful hook completion into process ownership loss rather than cleanup.

Suggested regressions:

  1. Force the Windows Job Object creation/spawn-contained path to fail so the taskkill fallback is exercised repeatedly; assert no taskkill.exe/conhost.exe accumulation and wait for each cleanup child to terminate.
  2. Repeated hook timeout/cancellation with synthetic child+grandchild trees; require process/handle counts to return to baseline.
  3. Separate explicit resident_detached hook semantics from normal task-owned hooks; only the former should preserve descendants, and preserved processes should have a durable lifecycle owner rather than being anonymously detached.
  4. Long-run Windows hook test followed by a fresh CreateProcess to catch eventual Win32 1816/resource exhaustion.

This is source-level corroboration, not a claim that this one fallback explains every handle/process in the separate incident; the MCP generation leak and sandbox/lsass-handle family can compound the same host-exhaustion symptom.

Farer · 2 days ago

Hello @chess-oai,

I am an independent developer based in South Korea, and I use Codex extensively in my daily development work.

I personally use several mini PCs because they offer great value for the money. Some of them have relatively modest hardware specifications, and the Windows version of the ChatGPT Desktop app has been experiencing very noticeable performance issues and lag on these systems.

I have been patiently waiting for improvements to this issue, but it has remained unresolved for quite some time, so I wanted to respectfully ask if the team could take another look at it.

From what I can see, I am not the only person experiencing this problem. A number of other developers have reported similar Windows performance, process cleanup, WMI, and CPU-related issues.

I understand that the team has many issues and priorities to work through, but I would sincerely appreciate it if this Windows performance issue could be given somewhat higher priority.

I have now been waiting for a fix or meaningful update for more than two weeks, and the issue has a significant impact on using Codex comfortably on Windows, especially on lower-spec machines.

I sincerely hope the team can provide an update or improvement soon.

If possible, could you please let us know whether this issue is currently being worked on, or whether there is a target Desktop/MSIX build containing a fix?

Thank you very much for your hard work and for continuing to improve Codex.