app-server: fd exhaustion is reported as a missing requirements file at TUI bootstrap; daemon may not inherit the shell's fd limit

Codex CLI version

0.147.0 (CLI) / 0.146.0 (running app-server — see "Version skew" below)

Platform

Linux (Ubuntu 24.04, systemd)

What happened

Launching the TUI failed at bootstrap with:

Error: configRequirements/read failed during TUI bootstrap: configRequirements/read failed:
failed to read configuration layers: Failed to read requirements file
/etc/codex/requirements.toml: No file descriptors available (os error 24) (code -32603)

/etc/codex/ does not exist on this machine at all. A healthy run returns ENOENT there and continues without complaint. The message names a file that is not the problem, and reports a startup path that is not where the failure occurred.

The actual cause was the long-lived codex ... app-server --listen unix:// daemon sitting at its file-descriptor ceiling. The TUI does not read requirements.toml itself — it asks the daemon over the control socket. The saturated daemon could not complete open(), received EMFILE, and forwarded it as JSON-RPC -32603.

This is the same mislabeling class as #36755 (skill loader reports EMFILE as "invalid SKILL.md files"), at a different call site. Here it is arguably worse: #36755 at least names files that exist, whereas this points at an absent path and an unrelated subsystem.

Measurements

The daemon had been running 8 days:

Max open files            1024                 1048576              files    (soft/hard)
open fds                  1022
child processes            230

Descriptor accounting closed exactly, at 4 per child (3 stdio pipes + 1 pidfd):

690  pipe          <- 230 children x 3
230  anon_inode:[pidfd]
 80  regular files (state_*.sqlite WAL/shm, thread_history, rollout jsonl, logs)
 13  socket
  9  eventpoll/eventfd/inotify
----
1022 / 1024

The 230 children grouped into 43 identical sets — one set per session started over those 8 days, with ~6 stdio MCP servers per set. None were ever reaped. SIGTERM to the daemon reclaimed all of them.

Because the leaked handle is a pidfd, the children stay valid and never appear as zombies. ps showed one zombie process system-wide. The leak is invisible except as descriptor pressure, which is part of why the surfaced error is so far from the cause.

The leak itself is already reported in #26984 (fd/pipe leak, orphan children) and #30408 (per-thread MCP processes never cleaned up). This report is about two things those do not cover.

Issue 1: the error names the wrong subsystem

EMFILE is a resource-exhaustion condition, not a configuration problem, but the configRequirements/read path reports it as a failure to read a specific requirements file. Anyone hitting this will investigate /etc/codex/requirements.toml — a file that need not exist — instead of the daemon's descriptor use.

Suggested handling, consistent with what #36755 asks for at its own call site:

• Distinguish EMFILE/ENFILE from ENOENT and from parse failures, and say so: "the app-server has exhausted its file descriptors" rather than naming an optional config path.
• Do not report an optional, absent config file as a read failure at all when the underlying errno is a resource error.

Issue 2: the daemon may not inherit the launching shell's fd limit

The usual advice for the leak (ulimit -n <higher> before starting Codex) may not reach the daemon at all, which makes the workaround suggested in the existing threads unreliable on Linux.

The app-server detaches into its own login-session scope with PPID 1, so in that
case it takes systemd's DefaultLimitNOFILESoft (built-in default 1024)
rather than anything from the calling shell. Observed directly: launching from a
shell with ulimit -n 262144 produced a daemon with a 1024 soft limit.

This is not fully deterministic — on a later respawn the daemon did land in an
existing session scope and inherited that scope's higher limit. Which limit you
get appears to depend on whether the daemon is started fresh and detached or
forked within an existing session, which makes the ulimit workaround
unreliable rather than simply ineffective. Either way the user has no way to
tell which they got without inspecting /proc/<pid>/limits.

Reproduction:

# a login shell reports a high limit ...
$ bash -lc 'ulimit -Sn'
1048576

# ... but anything systemd starts gets the built-in default
$ systemd-run --user --pipe --wait sh -c 'ulimit -Sn'
1024

# and the daemon can land in the latter category despite the launching shell
$ bash -c 'ulimit -n 262144; codex app-server daemon version >/dev/null'
$ pid=$(pgrep -x codex | while read p; do
      tr '\0' ' ' < /proc/$p/cmdline | grep -q 'app-server --listen' && echo $p
  done | head -1)
$ awk '/Max open files/{print $4}' /proc/$pid/limits
1024

Suggested handling: on startup the app-server could raise its own RLIMIT_NOFILE soft limit toward the hard limit (a standard setrlimit bump — the hard limit here was 1048576, so no privilege is required), and log the effective value. That alone would turn a multi-day outage into a non-event even while the leak persists.

Issue 3 (minor): daemon restart cannot restart an unmanaged daemon

When the daemon was auto-spawned on demand rather than started via the daemon manager:

$ codex app-server daemon restart
Error: app server is running but is not managed by codex app-server daemon

There is no documented path from that state back to a healthy one. SIGTERM to the pid works and the daemon respawns on next use, but the CLI does not suggest it. Either restart should be able to adopt/replace an unmanaged daemon, or the error should say what to do instead.

Version skew

codex app-server daemon version reported:

{"cliVersion":"0.147.0","appServerVersion":"0.146.0"}

The daemon kept running its old binaries across a CLI upgrade, carrying the accumulated leak with it. Worth considering whether the CLI should signal (or handle) a daemon older than itself, since a long-lived daemon is exactly the process most likely to have accumulated problems that a restart would clear.

Related

• #26984 — MCP stdio servers leak pipe fds + orphan child processes (same leak, macOS)
• #30408 — MCP server processes leak: per-thread processes never cleaned up (same leak, Desktop)
• #36755 — skill loader mislabels EMFILE as invalid SKILL.md (same mislabeling class, different call site)