Codex Desktop SSH reconnect creates many stale remote app-server processes across multiple Linux hosts, eventually causing fork/exec failures

Open 💬 3 comments Opened Jun 24, 2026 by Yoon-Jeong-ho
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

What version of the Codex App are you using?

Codex Desktop for macOS 26.616.81150 (CFBundleVersion 4306)

What subscription do you have?

Not specified in this public report; can provide privately if required.

What platform is your computer?

Local client:

macOS 26.5.1 build 25F80
arm64
Codex Desktop 26.616.81150

Remote hosts:

Multiple Linux SSH hosts, mostly Ubuntu 22.04/24.04 class systems
Remote connection type: Codex Desktop -> SSH remote connection
Remote Codex CLI observed on reachable hosts: codex-cli 0.135.0

What issue are you seeing?

When Codex Desktop reconnects/restores SSH remote connections, it appears to start codex app-server sessions across many configured remote hosts. Some of those remote app-server processes remain after connection failures and become orphaned under PID 1. Repeated reconnects then accumulate enough remote processes/threads that the remote account can no longer fork shells or accept SSH exec requests.

This is not only a single failed SSH connection. It affected a fleet of SSH remote hosts after Codex Desktop restart/reconnect.

Observed symptoms:

Uncaught Exception:
Error: WebSocket was closed before the connection was established
at e.terminate (/Applications/Codex.app/Contents/Resources/app.asar/.vite/build/src-BEdCFtv8.js:370:32003)
at /Applications/Codex.app/Contents/Resources/app.asar/.vite/build/main-DLo8G5hp.js:299:1450
at async Av.runWithSshStartupGate (.../main-DLo8G5hp.js:300:7070)
at async UI.connectAndWaitForOpen (.../src-BEdCFtv8.js:364:48602)
at async UI.startAppServerProcess (.../src-BEdCFtv8.js:364:46900)

Remote SSH failures after authentication succeeds:

Authenticated to <remote-host> using "publickey".
exec request failed on channel 0

On one host before it fully failed:

bash: fork: retry: Resource temporarily unavailable
app_server_count=62

Shortly afterward, that host also started returning:

exec request failed on channel 0

Fleet-level evidence

After closing/cleaning stale processes once, Codex Desktop restart/reconnect caused remote app-server processes to respawn. A later read-only check showed:

<host-08>: app-server 2
<host-11>: app-server 5
<host-12>: app-server 9
<host-13>: app-server 9
<host-14>: app-server 9
<host-15>: app-server 9
<host-16>: app-server 11
<host-18>: app-server 62, then fork failures, then SSH exec failure
<host-20>: app-server 9
<host-09>, <host-17>, <host-19>: SSH auth succeeded but exec failed

On the host with 62 app-server matches, the process list included many old orphaned app-server processes:

PID      PPID   ELAPSED      STAT CMD
447411      1   2-16:00:21  Sl   node .../bin/codex app-server --listen unix://...
447429 447411   2-16:00:20  Sl   .../bin/codex app-server --listen unix://...
625657      1   3-17:52:22  Sl   node .../bin/codex app-server --listen unix://...
625712 625657   3-17:52:22  Sl   .../bin/codex app-server --listen unix://...

This suggests failed/restarted Desktop SSH websocket sessions can leave long-lived orphaned remote app-server processes.

Relevant Codex Desktop log evidence

The local app log showed Codex Desktop trying many remote SSH hosts in a burst:

Starting app-server connection hostId=remote-ssh-discovered:<host-16> transport=websocket
ssh_websocket_v0.ensure_remote_app_server sshAlias=<host-16>

Starting app-server connection hostId=remote-ssh-discovered:<host-17> transport=websocket
ssh_websocket_v0.ensure_remote_app_server sshAlias=<host-17>

Starting app-server connection hostId=remote-ssh-discovered:<host-18> transport=websocket
ssh_websocket_v0.ensure_remote_app_server sshAlias=<host-18>

Starting app-server connection hostId=remote-ssh-discovered:<host-19> transport=websocket
ssh_websocket_v0.ensure_remote_app_server sshAlias=<host-19>

Starting app-server connection hostId=remote-ssh-discovered:<host-20> transport=websocket
ssh_websocket_v0.ensure_remote_app_server sshAlias=<host-20>

For one already-failing host, logs showed an extremely high reconnect attempt count:

app_server_connection.reconnect_started hostId=remote-ssh-discovered:<host-09> reconnectAttempt=7615 trigger=timer
Starting app-server connection hostId=remote-ssh-discovered:<host-09> transport=websocket
Authenticated to <remote-host> using "publickey".
exec request failed on channel 0

Steps that appear to reproduce or trigger the problem

I do not yet have a minimal single-host reproduction, but the observed sequence was:

  1. Configure multiple Linux SSH remotes in Codex Desktop.
  2. Use Codex Desktop with those remotes over time.
  3. Some remote app-server/WebSocket bootstraps fail or time out.
  4. Restart or reopen Codex Desktop.
  5. Codex Desktop attempts to reconnect/restore many SSH remotes.
  6. Remote codex app-server --listen unix://... processes accumulate across hosts.
  7. Some app-server processes remain orphaned under PID 1 for days.
  8. Affected hosts eventually show fork: Resource temporarily unavailable.
  9. After that, SSH public-key authentication still succeeds, but SSH exec fails with exec request failed on channel 0.

Expected behavior

Codex Desktop should:

  • Bound SSH remote reconnect attempts and avoid unbounded retry loops.
  • Avoid restoring or connecting to every discovered SSH host unless the user requested that.
  • Cleanly terminate remote codex app-server and codex app-server proxy processes after failed SSH websocket startup.
  • Avoid leaving orphaned remote app-server processes under PID 1.
  • Detect remote fork/process exhaustion and stop retrying that host.
  • Provide a UI or CLI action to stop all Desktop-created remote app-server/proxy sessions for a host.
  • Surface a controlled connection error instead of an uncaught WebSocket exception.

Actual behavior

Codex Desktop repeatedly attempts remote SSH websocket app-server startup. Failed or stale remote app-server processes can remain alive. On a multi-host SSH setup, this can create many remote app-server processes across the fleet and eventually exhaust per-user process/thread resources on the remote hosts.

Impact

High impact for shared Linux development/GPU servers:

  • Remote SSH exec can fail even though public-key authentication succeeds.
  • Users see exec request failed on channel 0 and cannot start shells through normal SSH exec.
  • Shells can fail with fork: Resource temporarily unavailable.
  • Manual cleanup is difficult once the account can no longer fork.
  • On shared servers, this can interfere with other operational tasks and makes Desktop remote auto-reconnect unsafe.

Related issues

This looks related to, but not identical to:

  • #27338: WebSocket uncaught exception when SSH remote bootstrap fails.
  • #24542: Desktop remote proxy respawns unmanaged app-server.
  • #28913: stale remote-control app-server reused by local CLI.
  • #29138: Desktop Connections leaves stale backend state.
  • #26382: canceled/background Codex work and remote app-server/proxy process cleanup issues.

The distinct part here is fleet-wide SSH remote reconnect/app-server fan-out leading to remote Linux fork/exec failure.

Additional notes

The issue body intentionally redacts:

  • remote hostnames
  • IP addresses
  • usernames
  • private key names
  • private filesystem paths

I can provide additional sanitized logs if needed.

이 이슈는 코덱스로 작성되었습니다.

View original on GitHub ↗

3 Comments

github-actions[bot] contributor · 2 months ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #29138

Powered by Codex Action

Yoon-Jeong-ho · 2 months ago

Thanks for the duplicate suggestion. I reviewed #29138 and agree it is related, but I do not think this issue is a duplicate.

#29138 focuses on a single remote backend launched with a scrubbed environment, then later reused by local CLI sessions, causing environment divergence.

This issue focuses on a different failure mode:

  • Codex Desktop SSH reconnect/restoration fans out across multiple configured Linux SSH hosts.
  • Remote codex app-server processes are left orphaned under PID 1 for days.
  • Reconnect attempts keep accumulating app-server processes across the fleet.
  • The accumulation eventually causes remote process/thread exhaustion.
  • Affected hosts then show bash: fork: retry: Resource temporarily unavailable.
  • SSH public-key authentication still succeeds, but SSH exec fails with exec request failed on channel 0.

So the common area is stale remote app-server lifecycle management, but the impact and reproduction pattern are different: fleet-wide reconnect fan-out and remote Linux fork/exec failure.

Happy to keep this linked to #29138, but I think this issue should remain open to track the multi-host reconnect/resource-exhaustion case separately.

이 댓글은 코덱스로 작성되었습니다.

MichealWangYZ · 2 months ago

I hit what appears to be the same Codex Desktop remote SSH app-server accumulation issue, with an additional downstream failure mode around SQLite corruption and quota exhaustion.

Environment details:

Remote Codex version: 0.142.1
Remote OS: AlmaLinux 9.7 x86_64
Remote access path: Codex Desktop SSH remote connection
Codex state/control path: ~/.codex on NFSv4 home, mount had local_lock=none

Observed process state on the remote host:

546 codex app-server --listen unix:// processes
546 matching /bin/sh bootstrap parent processes

The bootstrap command shape matched the Desktop SSH app-server bootstrap:

/bin/sh -c '... mkdir -p ${CODEX_HOME:-$HOME/.codex}/app-server-control && ... nohup codex app-server --listen unix:// > .../app-server.log 2>&1 &'

The parent shells stayed alive in do_wait, waiting on their child app-server. The app-server children were long-lived and many appeared to be listening on the same apparent socket path:

~/.codex/app-server-control/app-server-control.sock

Timestamps showed repeated launches over several days, often in bursts around ~90 seconds apart.

Downstream impact:

  • Multiple app servers had the same ~/.codex/logs_2.sqlite* / state_5.sqlite* files open.
  • codex doctor reported SQLite integrity failures / disk I/O errors.
  • The app-server log repeatedly moved damaged DBs into ~/.codex/db-backups and rebuilt them.
  • ~/.codex/db-backups grew to about 5.4 GB and exhausted the remote home quota.
  • codex resume then failed with quota exceeded plus damaged local database errors.
  • The app-server log also showed model refresh 429 Too Many Requests, likely amplified by the many app-server instances.

I originally opened #30548 with more detail, but this looks like the same root issue as this ticket, so I’m closing that as a duplicate and leaving the extra reproduction details here.