TUI freezes indefinitely (input + rendering dead, Esc unresponsive) when an in-flight turn's await stalls — main select! loop awaits handlers inline

Summary

The TUI can freeze completely and indefinitely during a turn: rendering stops (the Working (Xm Ys) elapsed timer stops ticking), keyboard input is dead (Esc to interrupt does nothing), and the only way out is to kill the process. The session itself is fine on disk and codex resume recovers it, but the running instance is unrecoverable.

I hit this on a long turn (a /review that fanned out parallel tool calls / sub-threads). The turn header sat at Working (58m 20s • esc to interrupt) with the timer frozen at the same second for many minutes.

Environment

• codex-rs, recent main.
• macOS 26.5 (arm64), running inside a terminal emulator (kitty 0.47.2).
• Release build (stripped/optimized).

Observed symptoms

• Working (Xm Ys) elapsed timer frozen at a fixed value (does not advance ~1×/s as it normally does).
• Screen does not redraw at all (captured the terminal grid twice several seconds apart — byte-identical).
• Esc (the documented interrupt key) has no effect.
• SIGWINCH (terminal resize) does not trigger a redraw.
• Process CPU near-zero; the session rollout file on disk had stopped growing.

Diagnosis (thread states)

sample/lldb on the hung process show every thread parked in a kernel wait — zero forward progress:

• main → __ulock_wait (it is pthread_join-ing the codex-main thread; expected).
• codex-main → __psynch_cvwait — i.e. Runtime::block_on(<top-level future>) parked, waiting for a waker that never fires.
• tokio-rt-worker ×N → parked on condvars (idle, no runnable tasks); one in kevent (the I/O driver).
• reqwest-internal-sync-runtime → kevent.
• sqlx-sqlite-worker-* ×9 → all in semaphore_wait.

At the time of the hang there were ~25 ESTABLISHED TLS connections to the model endpoint that were silent (no bytes flowing) — consistent with one or more streaming responses that stalled mid-stream.

Root cause analysis

The top-level TUI loop awaits each branch's handler inline inside the select! (codex-rs/tui/src/app.rs, around line 1158 on recent main):

loop {
    let control = select! {
        Some(event) = app_event_rx.recv()          => handle_event(...).await,
        active = /* active_thread_rx.recv() */      => handle_active_thread_event(...).await,
        event  = tui_events.next()                  => handle_tui_event(...).await,   // keyboard (incl. Esc) + redraw ticks
        app_server_event = app_server.next_event()  => handle_app_server_event(...).await,
    };
    // ...
}

select! drives exactly one branch's future at a time. If a chosen handler .awaits on something that never resolves, the entire loop is blocked — the tui_events.next() branch (line ~1184) is never polled again, so:

• keyboard input (including Esc → interrupt) is never read, and
• frame/redraw ticks are never serviced (hence the frozen elapsed timer and dead screen).

In other words, an in-flight turn handler that stalls takes down the whole UI, and there is no watchdog/timeout around the loop and no way for Esc to preempt an already-running handler .await.

The SSE reader does have an idle timeout (codex-api/src/sse/responses.rs, process_sse → timeout(idle_timeout, stream.next()) around line 446; default appears to be ~5 min, with the idle branch emitting ApiError::Stream("idle timeout waiting for SSE")). Since the UI stayed frozen for ~58 min, the stalled .await that wedged the loop is on a path that is not covered by that idle timeout — most likely the orchestration of the parallel sub-threads / waiting for the full set of tool-call outputs, or a channel/sqlx await with no timeout. (The 9 blocked sqlx-sqlite-worker threads suggest the thread-store/DB path may be involved, but I could not confirm the exact awaited resource from a stripped binary.)

Why it's unrecoverable once wedged

Because the wedge is the top-level future parked awaiting an inner future whose wakeup will never arrive. There is no in-process or external (debugger) way to surgically wake a specific parked tokio future in a stripped release binary, so the only recovery is kill + codex resume.

Suggested fixes

1. Don't await long/turn handlers inline in the select! loop. Spawn turn/sub-thread processing as background tasks that only feed events back via the existing channels, so tui_events (input + redraw) is always serviced and Esc/interrupt stays responsive even during a stalled turn.
2. Make Esc/interrupt able to preempt an in-flight turn (e.g. an AbortHandle/cancellation token for the active turn that the input path can trigger), rather than depending on the handler returning.
3. Add a watchdog/overall timeout (or at least surface a "no progress for N seconds, press Esc to abort" state) around turn handling, and ensure idle/read timeouts cover the parallel-tool-call / sub-thread orchestration path, not just the single SSE read.
4. Consider a defensive idle timeout on the thread-store/sqlx awaits so a stuck DB pool can't silently freeze the UI.

Repro (approximate)

1. Start a turn that streams a response and/or fans out parallel tool calls / sub-threads (e.g. /review on a large change set).
2. Cause the model stream to stall mid-response (e.g. flaky/half-open upstream connection that keeps the socket ESTABLISHED but sends no further bytes).
3. The TUI freezes: elapsed timer stops, screen stops redrawing, Esc is ignored; kill required.

Happy to provide more detail on the thread dumps if useful.