Stale app-server survives CLI update and causes 0.147.0 TUI to use pre-0.147 `spawn_agent` model filtering

Open 💬 2 comments Opened Aug 8, 2026 by keyhr

What version of Codex CLI is running?

codex-cli 0.147.0

Platform

Ubuntu 24.04.4 LTS, Linux x86_64.

Codex installation:

running standalone on linux-x86_64

current executable:
/home/keyhr/.codex/packages/standalone/releases/0.147.0-x86_64-unknown-linux-musl/bin/codex

version:
0.147.0

The SHA256 of the installed binary exactly matches the rust-v0.147.0 GitHub release binary:

cb0a15567e9a60a5820d54b0f6ae86d504dc3805c1eab21a47f70e3eb7b73a40  installed codex
cb0a15567e9a60a5820d54b0f6ae86d504dc3805c1eab21a47f70e3eb7b73a40  rust-v0.147.0 release binary

What issue are you seeing?

After updating Codex CLI from 0.146.0 to 0.147.0, a normal TUI session still behaves as if it is using the pre-0.147 multi-agent model filtering.

Specifically, from a gpt-5.6-sol parent, spawning gpt-5.6-luna fails:

Unknown model gpt-5.6-luna for spawn_agent.
Available models: gpt-5.6-sol, gpt-5.6-terra

However, starting the same 0.147.0 binary with:

codex --disable tui_app_server -m gpt-5.6-sol

makes the exact same Luna spawn succeed:

Started `/root/luna_spawn_check`
...
LUNA_SPAWN_OK

This appears to be caused by a stale 0.146.0 app-server surviving the CLI update and being reused by the normal TUI path.

Process inspection shows:

PID=12089
codex -c features.code_mode_host=true app-server --listen unix://
/home/keyhr/.codex/packages/standalone/releases/0.146.0-x86_64-unknown-linux-musl/bin/codex

while the currently installed CLI is 0.147.0.

There is also a 0.147.0 app-server proxy process:

codex app-server proxy
/home/keyhr/.codex/packages/standalone/releases/0.147.0-x86_64-unknown-linux-musl/bin/codex

So the system can end up with a 0.147.0 frontend/proxy while a 0.146.0 app-server remains alive.

Why this is particularly visible in 0.147.0

rust-v0.147.0 includes #36892, "Support leaf models in multi-agent v2".

Before that change, a V2 parent such as gpt-5.6-sol rejected the V1/leaf gpt-5.6-luna model. After #36892, Luna should be accepted as a leaf worker.

The effective model catalog on the affected machine is correct:

gpt-5.6-sol     list    v2    true
gpt-5.6-terra   list    v2    true
gpt-5.6-luna    list    v1    true

The bundled model catalog reports the same values.

So this is not a stale model catalog.

The stale app-server makes a 0.147.0 CLI exhibit exactly the old behavior:

Available models: gpt-5.6-sol, gpt-5.6-terra

Reproduction

  1. Have Codex 0.146.0 installed and an app-server running.
  1. Upgrade Codex to 0.147.0 without manually terminating the existing app-server.
  1. Verify the shell CLI is updated:
codex --version
# codex-cli 0.147.0
  1. Start a normal Sol session:
codex -m gpt-5.6-sol
  1. Call:
spawn_agent(
  model="gpt-5.6-luna",
  fork_turns="none",
  reasoning_effort="low",
  message="Return exactly LUNA_SPAWN_OK"
)
  1. Observe:
Unknown model gpt-5.6-luna for spawn_agent.
Available models: gpt-5.6-sol, gpt-5.6-terra
  1. Inspect running processes:
for pid in $(pgrep -f 'codex.*app-server'); do
  echo "PID=$pid"
  tr '\0' ' ' < /proc/$pid/cmdline
  echo
  readlink -f /proc/$pid/exe
done
  1. Observe that an app-server from the old 0.146.0 installation is still running.
  1. Start the same installed 0.147.0 binary while bypassing the app-server-backed TUI:
codex --disable tui_app_server -m gpt-5.6-sol
  1. Repeat exactly the same spawn_agent call.
  1. It succeeds:
LUNA_SPAWN_OK

Additional A/B verification

On macOS a Homebrew-installed Codex 0.147.0 successfully spawns Luna from a Sol parent.

The parent session log contains:

{
  "name": "spawn_agent",
  "arguments": {
    "model": "gpt-5.6-luna",
    "reasoning_effort": "low"
  }
}

and the resulting child session confirms:

{
  "model": "gpt-5.6-luna",
  "effort": "low"
}

So this is not a silent fallback to Sol.

On the affected Linux machine, a local source build of rust-v0.147.0 also fails through the normal TUI path, but the official 0.147.0 binary succeeds when tui_app_server is disabled. This further points to app-server reuse/version skew rather than the 0.147.0 executable itself.

Feature state

Both working and failing environments report:

multi_agent       stable  true
multi_agent_v2    stable  false

so the difference is not explained by that feature flag.

Expected behavior

When the active CLI is upgraded, the normal TUI should not silently execute turns through an incompatible older app-server.

At minimum, Codex should detect:

CLI version != app-server version

and either:

  1. restart/replace the stale app-server automatically,
  2. refuse to reuse it and start a matching app-server, or
  3. show a clear warning that the active session is running against a different app-server version.

A user seeing:

OpenAI Codex (v0.147.0)

should not unknowingly get 0.146.0 backend behavior.

Workaround

This immediately avoids the problem:

codex --disable tui_app_server

Manually terminating the stale 0.146.0 app-server and allowing a current one to start should also resolve the version skew.

Related issues / changes

  • #34721 — old app-server/remote-control daemon survives a CLI update
  • #24034 — active CLI and app-server can resolve to different versions
  • #36892 — 0.147.0 adds leaf-model support for Multi-Agent V2

This may share the same root cause as #34721 / #24034, but this report demonstrates a concrete functional regression in a normal TUI session: a stale app-server causes a feature added in the currently displayed CLI version to remain unavailable.

View original on GitHub ↗

2 Comments

hauomu · 20 days ago

Cross-reference from Windows Desktop: I can reproduce the same Sol/Terra-only Luna rejection after explicitly ruling out a stale 0.146 app-server.

On one Windows 11 host/account:

  • Stable Desktop package: OpenAI.Codex 26.803.5235.0
  • fresh Desktop-generated official model cache: client_version = 0.147.0
  • catalog: Sol list/v2, Terra list/v2, Luna list/v1
  • no model_catalog_json override
  • all ChatGPT.exe and codex.exe processes were force-terminated and verified absent
  • Stable Desktop was relaunched from scratch and a brand-new thread used

Desktop still reports:

Unknown model gpt-5.6-luna.
Available models: gpt-5.6-sol and gpt-5.6-terra.

By contrast, standalone codex-cli 0.147.0 on the same machine successfully spawns Luna from Sol using the same official catalog.

I posted the full A/B reproduction in #34964. This suggests the stale-server mechanism documented here is a real cause of the symptom, but not the only one: Windows Desktop 0.147's app-server/spawn path can still exhibit the pre-#36892 allowlist behavior even after a complete process teardown/restart with a freshly generated 0.147 catalog.

A particularly strong clue is that a local catalog copy with only Luna's multi_agent_version changed v1 -> v2 makes Desktop spawning succeed again. That points toward an app-server/Desktop-specific model-filtering path still requiring exact V2 metadata rather than accepting visible V1 models as leaf workers.

jdcodes1 · 9 days ago

This is the persistent-daemon version-skew family: the TUI you launch is 0.147.0, but it connects to the long-lived codex app-server daemon over the control socket — and that daemon keeps executing whatever binary it was started from until something restarts it. An update replaces packages/standalone/current on disk but doesn't restart a running daemon, so spawn_agent model filtering (server-side logic in the daemon process) stays pre-0.147 while your shell's codex --version says otherwise. Same underlying mechanism as #37754 (old daemon serving list_turns errors after update) — and #37893 documents the worst case, where the daemon becomes unmanageable so even codex app-server daemon restart can't fix it ("running but not managed").

Immediate workaround: codex app-server daemon restart (or kill the codex app-server process) after upgrading — your spawn should then accept gpt-5.6-luna.

Repo-side fix worth centralizing across these issues: on connect, the client already probes the daemon's version over the socket; when it doesn't match the client binary, either auto-restart the daemon (safe when no live turns) or at minimum surface "daemon is running 0.146.0, restart to apply the update" instead of letting stale behavior masquerade as current-version bugs. Verifying probe_app_server_version (app-server-daemon/src/lib.rs#L80) against the client version at TUI bootstrap is the one-line detection.