Desktop: orphaned codex_chronicle after killed app causes lock-wait loop and high CPU

Open 💬 2 comments Opened Apr 25, 2026 by markmdev

What issue are you seeing?

After Codex Desktop was force-killed and restarted, an orphaned codex_chronicle process remained alive under launchd and continued holding the Chronicle lock. The restarted Desktop app repeatedly spawned a new Chronicle sidecar, which logged Waiting to acquire lock (owned by pid=...) over and over while the app/server processes showed high CPU and memory churn.

Killing only the orphaned codex_chronicle process immediately removed that lock-wait loop and substantially reduced the runaway CPU behavior. No user threads were archived or deleted.

This seems like a Desktop lifecycle cleanup bug: when the main app is killed or crashes, the Chronicle sidecar can survive as an orphan and block the next app instance.

Environment

  • Codex Desktop: 26.422.30944 build 2080
  • Codex CLI: codex-cli 0.117.0
  • macOS: 26.0.1 (25A362)
  • Architecture: arm64
  • Related issue: #19166 appears related to large-thread instability / Array buffer allocation failed, but this report is specifically about the orphaned Chronicle lock holder after restart.

Evidence

Before mitigation, process state included an orphaned Chronicle process from the previous killed app instance:

PID    PPID  STAT  %CPU  RSS    ELAPSED  COMMAND
3430   1     S     0.0   12832  48:59    /Applications/Codex.app/Contents/Resources/codex_chronicle

The restarted app was repeatedly spawning a new sidecar that could not acquire the lock:

[AppServerConnection] Starting local app-server sidecar argsCount=0 command=/Applications/Codex.app/Contents/Resources/codex_chronicle cwd=null hostId=local
[AppServerConnection] app_server_sidecar_stderr hostId=local message="codex_chronicle starting\nWaiting to acquire lock (owned by pid=3430)..."
[AppServerConnection] app_server_sidecar_stderr hostId=local message="Waiting to acquire lock (owned by pid=3430)..."
[AppServerConnection] Stopping local app-server sidecar hostId=local pid=...

The log repeated that lock-wait message approximately every 10 seconds until the orphan was killed.

Before mitigation, the app had also entered a heavy thread hydration loop in the same restart window:

~660 method=thread/read
~658 method=thread/resume
~649 method=thread/unsubscribe
~656 maybe_resume_success

The active workload included several very large local threads and failed memory backfill jobs. This may be a separate stressor, but it amplified the failure:

memory_stage1 ... error ... stream disconnected before completion: Incomplete response returned, reason: max_output_tokens
memory_stage1 ... error ... Codex ran out of room in the model's context window. Start a new thread or clear earlier history before retrying.

Mitigation performed

  1. Rotated/deleted large local Codex log files to reduce disk churn.
  2. Killed only the stale orphaned codex_chronicle process and the old orphaned Crashpad handler from the previous app instance.
  3. Marked failed memory_stage1 retries as exhausted so the background summarizer would not immediately retry oversized failed jobs.

After killing the orphaned Chronicle process:

  • No codex_chronicle process remained outside the current app tree.
  • The Waiting to acquire lock loop stopped.
  • CPU dropped from runaway levels to a much more normal range.
  • Threads remained available; nothing was archived or deleted.

Expected behavior

If Codex Desktop is killed or crashes, any owned codex_chronicle sidecar should either:

  • exit with the parent app,
  • release its lock reliably,
  • or be detected and replaced safely by the next app launch.

The next Desktop launch should not repeatedly spawn sidecars that wait on a lock owned by a stale orphaned process.

Actual behavior

The orphaned codex_chronicle survived the killed app, held the lock, and caused the restarted app to repeatedly start/stop a Chronicle sidecar that could not acquire the lock. This correlated with high CPU/log churn and made the app unstable until the orphan was manually killed.

Possible fix direction

  • Ensure codex_chronicle is tied to the parent app lifecycle on macOS.
  • Store enough metadata with the Chronicle lock to detect stale/orphaned owners.
  • On Desktop startup, if the lock owner is parented to launchd and no current Codex parent owns it, either reclaim the lock or terminate/restart the sidecar.
  • Consider adding a bounded retry/backoff for sidecar startup when the lock is held.

View original on GitHub ↗

2 Comments

markmdev · 4 months ago

Follow-up after further debugging: the later high-CPU issue turned out to be a separate Desktop heartbeat target-thread resume/unsubscribe loop, now tracked in #19563.

I am leaving this issue scoped to the original orphaned codex_chronicle lock-wait problem, because that failure mode was real and had different evidence/mitigation: a stale codex_chronicle process survived the killed app, held the Chronicle lock, and the lock-wait loop stopped after killing only that orphaned process.

loremcdmx · 1 month ago

Confirming this, with an orphan that survived four days and several app restarts — and with what looks like a concrete downstream symptom worth linking (#27395).

Environment

  • Codex Desktop: ChatGPT.app 26.721.41059, Codex Framework 150.0.7871.128
  • macOS 26 (Darwin 27.0.0), Apple Silicon, 18 cores / 48 GB RAM
  • Observed 2026-07-26

The orphan

  PID  PPID  STARTED                      ELAPSED       COMMAND
 1261     1  Wed Jul 22 07:56:48 2026     04-00:10:05   .../ChatGPT.app/Contents/Resources/codex_chronicle

Reparented to launchd (PPID 1) and still holding the lock four days later, across multiple app launches. The current app instance was PID 8239, started 07:40:21 that morning; its own chronicle (PID 8355, started 07:40:31) spun for 4 h 31 min without ever acquiring the lock:

[AppServerConnection] app_server_sidecar_stderr message="codex_chronicle starting\nWaiting to acquire lock (owned by pid=1261)..."

That line appears 5,074 times in a single day of desktop logs.

Downstream symptom — likely a root cause for #27395

While the lock was held, the app-server request queue stayed saturated. inFlightRequestCount was pinned at its cap of 6, and everything else expired in the queue:

[AppServerConnection] app_server_client_request_queue_rejected inFlightRequestCount=6
    method=app/list queueWaitMs=30001 reason=queue-expired
[electron-message-handler] Request failed error={"code":-32001,"message":"App server request expired while queued"}
    method=thread/list durationMs=60003
[electron-message-handler] mcp_request_timeout method=turn/start pendingCount=12 timeoutMs=30000
[electron-message-handler] [Composer] submit failed errorMessage=Timeout errorName=Error followUp=local mode=local

Same method=app/list, skills/list, config/read, thread/list starvation, and the same user-facing "Error submitting message" banner described in #27395.

Before / after killing only PID 1261

| Window | [Composer] submit failed | expired while queued |
|---|---|---|
| 13:38–14:06 (before) | 4 | 47 |
| 14:10–14:19 (after) | 0 | 0 |

The waiting chronicle acquired the lock and started writing at 14:12:11, about 20 seconds after the kill. The log kept receiving entries throughout the "after" window (last entry 14:18:18 while sampling), so the zeros are a real recovery and not an artifact of logging having stopped.

No threads were archived or lost, matching the original report.

Suggested fix

Two options, either of which would close this:

  1. Liveness-checked lock. Record the owner PID plus a start-time/boot-id in the lock, and let a new sidecar steal it when the owner is gone or when the owning app instance has exited. A PID alone is not enough — here the PID was very much alive, just orphaned from an app that no longer existed.
  2. Lifecycle cleanup. Terminate the chronicle sidecar on app exit, and have an orphaned sidecar exit on its own once its parent app instance disappears rather than being reparented to launchd indefinitely.

A cheap mitigation in the meantime: after N failed lock acquisitions, surface it in the UI instead of silently spinning. Right now the only visible effect is that the app quietly stops working, which is what makes this expensive to diagnose — the actual cause is 5,000 lines deep in a log file most users will never open.

Related: #27395, #26454.