Codex CLI busy-loops at ~100% CPU after controlling PTY is deleted

Open 💬 2 comments Opened Jun 15, 2026 by brucx

Summary

A stale Codex CLI process kept running after its controlling terminal disappeared and started consuming roughly one full CPU core. The process no longer had an active terminal, was not producing session logs, and did not appear to have active network connections. Diagnostics point to a codex-main thread busy-looping after stdin/stdout/stderr pointed at a deleted PTY.

This looks like the CLI is not treating EOF/EIO/error readiness from a detached PTY as a terminal condition, or is retrying it in a tight event loop.

Environment

  • Codex CLI: codex-cli 0.139.0
  • Binary path observed: ~/.codex/packages/standalone/releases/0.139.0-x86_64-unknown-linux-musl/bin/codex
  • OS: Ubuntu 24.04.4 LTS
  • Kernel: Linux 6.17.0-1013-gcp #13~24.04.1-Ubuntu SMP Thu Apr 16 02:10:24 UTC 2026 x86_64
  • Invocation: codex --yolo

Observed state

A stale process was found:

PID 186204  codex --yolo
CPU ~98%-110%
Started Thu Jun 11 15:09:23 2026
cwd /home/xiongtengyan
parent 185390: bash --rcfile /dev/fd/63

Its stdio file descriptors pointed to a deleted PTY:

0 -> /dev/pts/0 (deleted)
1 -> /dev/pts/0 (deleted)
2 -> /dev/pts/0 (deleted)

The active CPU-consuming thread was:

TID 186447
comm: codex-main
state: R

Thread sample:

TID 186447 utime=14002634 stime=18633967 state=R comm=(codex-main)

The process was not writing new session content. Its session log file remained unchanged:

~/.codex/sessions/2026/06/11/rollout-2026-06-11T15-09-26-019eb73b-2317-7cc0-8455-be755ff32039.jsonl
size: 999070
mtime: 2026-06-11 15:30:58 +0000

There were no TCP sockets associated with the process at the time of inspection; only local UNIX sockets were open.

CPU / syscall evidence

A 2 second /proc sample showed high CPU time growth and extremely high read syscall growth:

sample start:
utime=14201888 stime=18685605
syscr: 243943917947

sample end, ~2s later:
utime=14201967 stime=18685719
syscr: 243963619706

That is approximately:

  • utime + stime: +193 ticks in ~2 seconds, close to one CPU core
  • syscr: +1,172,137 read syscalls in ~2 seconds

This suggests a tight read/event-loop retry path rather than normal model/API activity.

Expected behavior

When the controlling PTY disappears or stdin/stdout/stderr return EOF/EIO/error readiness, Codex should exit cleanly or back off without consuming a CPU core.

Actual behavior

The process survived terminal teardown and kept a codex-main thread spinning at roughly 100% CPU until manually terminated.

Workaround

Manually killing the stale process worked:

kill 186204

Notes

strace was not installed on this host, so I could not capture the exact syscall loop. The /proc data above should still narrow this down to a busy-loop around terminal/read/event handling after PTY deletion.

View original on GitHub ↗

2 Comments

HuMoran · 21 days ago

Confirming this on macOS 14.6 / arm64 with codex-cli 0.146.0 — newer than any version in the related reports (#11077 was 0.98.0, this issue 0.139.0, #28794 0.140.0), so this is still unfixed as of 0.146.0.

One detail that I think matters for triage: my setup has no Node wrapper at all. The binary is a standalone Mach-O arm64 executable at ~/.local/bin/codex, launched directly by a Python parent (not codex.js). When the parent dies the native process is reparented to launchd (PPID 1) directly. That means the Layer-1 fix proposed in #14962 (the codex.js SIGHUP handler never calling process.exit()) would not help this shape — the PTY-revoked spin in the Rust process is sufficient on its own to produce the runaway.

Environment

  • codex-cli 0.146.0, standalone binary, 271 MB Mach-O arm64
  • macOS 14.6 (23G80), Mac mini M2, 8 cores
  • Multi-user server; several people ssh in and run codex interactively. Each dropped SSH session leaves one orphan.

Evidence

Orphans vs. healthy processes are cleanly separable on three axes:

| | orphaned | healthy |
|---|---|---|
| PPID | 1 | live parent |
| lsof -d 0,1 | 0 (revoked) 1 (revoked), fd 2 → /dev/null | /dev/ttysNNN |
| %CPU | ~33% each (see below) | 0.0% |

Healthy codex processes sit at a measured 0.0% CPU even while actively working, so CPU alone is a clean discriminator.

Per-process CPU, measured as a cumulative-CPU-time delta over 44 s of wall clock rather than ps %cpu:

23 orphans × ~14.8 s CPU / 44 s = ~33.5% each

That number is exactly 8 cores ÷ 23 processes = 34.8% — i.e. every orphan is trying to consume a full core and the scheduler is fair-sharing them down. Aggregate was 770% of the 800% available.

The hot thread, from ps -M:

PID   %CPU  STAT  STIME       UTIME       
552   36.1  R     735:32.30   318:08.03

70% of the burned time is kernel time, matching the system-wide split. sample(1) most often caught the tokio worker threads with kevent as the leaf frame, which is consistent with the tight-poll behaviour described here (on macOS it surfaces via kevent rather than the Linux read(0) = EIO). I could not capture a syscall trace to nail the exact call — fs_usage itself could not get scheduled on the saturated box.

System-wide, before and after killing the 23 orphans:

before:  29.75% user, 70.24% sys,  0.00% idle    load avg 30.35
after:    0.59% user,  1.41% sys, 97.99% idle    load avg  1.41

The oldest orphan had accumulated 3021 minutes (50.3 h) of CPU over 66.8 h of elapsed time — 75% of a core sustained for 2.8 days. All 23 exited cleanly on SIGTERM; none needed SIGKILL.

Second-order impact worth flagging

The reports so far describe this as a heat/battery/wasted-resources problem. On a shared server it is worse than that: the accumulated orphans took down an unrelated service. This box also serves SMB. With 0% CPU idle, smbd was starved badly enough that a single tree connect went from sub-millisecond to 133–400 ms:

11:57:12.125 → 11:57:12.258   connect_to_named_tree → bind_tree   (133 ms)
11:57:12.630 → 11:57:13.032                                        (400 ms)

Windows SMB clients time out and fail outright at that latency while macOS clients still limp along, so it presented as "our file server broke for Windows users" and cost a full debugging session before anything pointed at codex. After reaping the orphans, the same operation completed within a single millisecond.

Mitigation for anyone else hitting this

A periodic reaper, keyed on the three-way discriminator above. The %CPU > 10 term is what makes it safe — healthy codex processes are at 0.0%, so it cannot misfire on a live session:

/bin/ps -Ao pid,ppid,tty,%cpu,comm | /usr/bin/awk '
  $2==1 && $3=="??" && $4>10 && $5 ~ /\/codex$/ {print $1}' | xargs kill -TERM

I run that from a launchd daemon every 300 s. Verified end-to-end against a synthetic orphan (a compiled busy-loop binary named codex, double-forked to PPID 1 with no controlling TTY): it matched the decoy and left all 7 live codex processes untouched.

Expected behaviour

On persistent EOF/EIO/permanent-readiness from a revoked PTY, the TUI event loop should treat it as terminal and exit, rather than re-polling a descriptor that will never become un-ready. A backoff would cap the damage, but for a headless process with revoked stdio there is nothing left to render to — exiting seems strictly correct.

Related: #14962 (root-causes the crossterm side — crossterm-rs/crossterm#793), #11077, #28794.

HuMoran · 1 day ago

Update from the same box, two versions later: on 0.149.1 the CPU spin is gone — the failure to exit is not.

I'm the reporter of the earlier comment on this issue on 0.146.0. Same machine, same workload, 20 days later. Something between 0.146.0 and 0.149.1 removed the busy-loop. The orphaning itself is untouched, and the new shape is arguably harder to notice.

What changed

| | 0.146.0 (my earlier comment) | 0.149.1 (now) |
|---|---|---|
| %CPU per orphan | ~33% each, all 8 cores saturated, load 30 | 0.0% |
| Exits after PTY revoke | no | still no |
| PPID | 1 | 1 |
| lsof -d 0,1 | (revoked) | (revoked) |
| RSS per orphan | (not the binding constraint) | ~110 MB |
| Visible how | machine falls over | nothing, until memory runs out |

The regression window is unusually clean

This box runs a 300 s reaper I wrote after the earlier investigation, whose predicate was PPID==1 && TTY=="??" && %CPU>10. Its log is effectively a census of spinning orphans:

  14 2026-08-07
   3 2026-08-10
  18 2026-08-11
  13 2026-08-12
  31 2026-08-13
  11 2026-08-14
   1 2026-08-15
  17 2026-08-17
   8 2026-08-18
   4 2026-08-19
   8 2026-08-20
   5 2026-08-21
   8 2026-08-24
   1 2026-08-25   <-- last ever

142 spinning orphans over 18 days, then it stops dead.

0.149.1 was installed 2026-08-25 09:59. The final log entry is 2026-08-25 11:55, for a process that had started at 09:57:50 — i.e. two minutes before the upgrade, so that was the last pre-upgrade process. Not one process started under 0.149.1 has ever spun.

Meanwhile the leak rate is unchanged. Between 2026-08-26 10:27 and 2026-08-27 14:54 — roughly 28 h — the box accumulated 31 orphans, 3.4 GB RSS combined, every one of them at 0.0% CPU, invisible to the reaper. Same order of magnitude as the 31 killed on 2026-08-13. Every dropped terminal still leaks exactly one process; it just stopped announcing itself.

Current-state evidence

$ ps -Ao pid,ppid,tty,%cpu,comm | awk '$2==1 && $3=="??" && $4 ~ /codex$/'
  (31 processes, all %cpu 0.0)

$ lsof -p <orphan> -d 0,1,2 -a
codex  <pid>  0        (revoked)
codex  <pid>  1        (revoked)
codex  <pid>  2w  CHR  3,2  /dev/null

At least one of them still held an ESTABLISHED TLS connection to the API endpoint after ~29 h with no terminal attached — so these are not merely parked, they retain live sockets.

All 31 exited cleanly on SIGTERM; none needed SIGKILL. Same as last time.

Correcting my own earlier claim

In my 0.146.0 comment I wrote that healthy processes sit at 0.0% CPU and therefore "CPU alone is a clean discriminator." That is no longer true and anyone who built cleanup around it is now missing 100% of these. The predicate that still works, and that I've verified end-to-end against both a fabricated orphan and 31 real ones with zero collateral damage on live sessions:

PPID == 1 && TTY == "??" && comm ~ /codex$ && lsof -p PID -d 0 reports "(revoked)"

A legitimately backgrounded codex (exec mode, nohup) has fd 0 on /dev/null or a pipe, never (revoked), so this does not false-positive.

Worth flagging for anyone reproducing: (revoked) only appears when sshd tears the session down normally. kill -9 on the ssh client leaves fd 0 as /dev/ttysNNN, cmd & gets fd 0 redirected to /dev/null by the shell, and < /dev/tty shows as /dev/tty. The only thing that reproduced it here was bash -mc 'T=$(tty); ./prog < "$T" & sleep 2' followed by a clean exit.

Why this is still worth fixing rather than closing

The 0.0% CPU shape is quieter but not benign: on a shared machine 31 idle orphans is 3.4 GB of resident memory and 31 retained TLS sockets, accumulating for as long as nobody looks. The earlier version at least crashed the box loudly enough to get investigated within a day. This one wouldn't have been noticed at all if I hadn't gone looking.

Environment: codex-cli 0.149.1, standalone arm64 binary (no Node wrapper), macOS 14.6 (23G80), Mac mini M2, 8 cores. Multiple users ssh in and run codex interactively; each dropped session leaves one orphan.