›_ Codex Issues
Codex Issues / #24527

TUI typing lag: synchronous palette and terminal-size queries block input

Resolved 💬 7 comments Opened May 26, 2026 by mehulparmariitr Closed Aug 3, 2026
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.
[!IMPORTANT] Reviewable two-fix implementation: https://github.com/mehulparmariitr/codex/pull/1 Direct upstream PR creation was rejected by the repository's invitation-only permissions. Maintainers: please either invite mehulparmariitr to submit this branch to openai/codex or use the linked commits as the basis for an OpenAI-owned PR.

Summary

Codex CLI intermittently displays typed characters late, including in fresh and resumed sessions. Instrumented codex-cli 0.145.0 traces isolated two independent synchronous blockers on the TUI input/render path:

  1. FocusGained performed sequential OSC foreground/background palette queries before input polling resumed.
  2. Ordinary frames repeatedly queried terminal size synchronously, including inside autoresize().

Both can block for seconds under terminal or machine contention. The proposed branch removes these two blockers while preserving focus color refresh, resize behavior, and a post-reflow recovery check.

Reproduction

Observed on macOS arm64 in Ghostty and IDE terminals, without requiring a long transcript:

  1. Start or resume Codex.
  2. Leave it idle or switch focus away and back.
  3. Type into the composer.
  4. Intermittently, characters appear tens of milliseconds to several seconds late.

Concurrent Codex TUIs can slow at the same time, so transcript length is neither necessary nor sufficient.

Confirmed evidence

| Boundary | Direct measurement | Conclusion |
|---|---:|---|
| Focus-time palette refresh | Maximum 15,082.767 ms; another capture was 4,127.795 ms, matching two sequential two-second waits plus overhead | Confirmed synchronous input blocker |
| Terminal-size lookup | Individual calls reached 9,282.232 ms, 6,116.971 ms, 5,547.107 ms, and 4,448.370 ms at the four ordinary draw call sites | Confirmed synchronous draw blocker |
| Already-readable input behind size lookup | One direct pair measured 4,394.705 ms inside size lookup and 4,327.577 ms before the queued key was consumed; the key then drew in milliseconds | Confirms input waits behind the blocked TUI task |
| Inner terminal draw | Repeated slow autoresize() samples resolved to another size lookup; render callback, buffer diff, backend draw, and backend flush had no samples at or above 50 ms in that capture | Supports removing size queries rather than optimizing rendering |
| Load amplification | During a controlled CPU-heavy build, two independent TUIs produced 332 nearest slow-size pairs within 10 ms; 156 pairs exceeded 100 ms in both processes | Machine pressure amplifies the synchronous boundaries |

Composer mutation, model/network work, MCP calls, SQLite, and transcript reflow were not required for the captured maximums.

Implementation

Branch: https://github.com/mehulparmariitr/codex/tree/fix/tui-input-lag-blocking-queries
Upstream comparison: https://github.com/openai/codex/compare/main...mehulparmariitr:codex:fix/tui-input-lag-blocking-queries

| Fix | Commit | Behavior |
|---|---|---|
| Reuse terminal size across draws | ec956c42bd | Carries dimensions from resize events, reuses the cached size through pre-render/layout/reflow/draw, and retains one explicit post-reflow recovery sample |
| Bound focus palette refresh | 8e6f543b07 | Pauses and recreates the Crossterm event source, uses the existing batched foreground/background probe with one shared 100 ms deadline, and preserves cached colors on timeout/error |

The fixes are separate commits for review and rollback but are combined because they address the same user-visible input-latency issue.

Verification

  • Regression: unchanged-size draws do not query backend size.
  • Regression: the explicit post-reflow size recheck is drained correctly.
  • Regression: focus gain recreates the event source and the following key is delivered.
  • Fresh just test -p codex-tui: 3,252 passed, 4 skipped, and one unrelated resume test timed out under heavy load; that exact test passed immediately when rerun alone.
  • Scoped Clippy and formatting passed.

In the latest patched diagnostic session:

  • 720 completed key events
  • p50 1.523 ms, p95 6.534 ms, p99 11.559 ms
  • exactly one terminal-size sample during setup, not repeated on ordinary draws
  • no focus, key, or runtime stall

Two keys still took 62.457 ms and 71.991 ms; 60.236-70.106 ms was spent after draw notification but before the draw handler ran. Composer, rendering, and terminal output remained below 1 ms. This is a separate event-loop/OS-scheduling residual under high machine load, not either fixed synchronous query.

Scope

This PR removes the two directly proven Codex blockers. It does not claim to eliminate:

  • OS/terminal pixel-presentation delay,
  • host key-repeat behavior, or
  • all event-loop scheduling delay under severe machine contention.

Those paths remain measurable follow-ups rather than speculative additions to this PR.

Related reports and prior work

  • Issues: #33037 (focus freeze), #23702 (PTY backlog), #21978 (stale size), #23489 and #35309 (OSC probing), #21945 (reflow), #31838 (other input latency), #19790 (input ownership), #20213 (multi-terminal freezes), #13795 (typing framerate), and #31420 (reply leakage).
  • Upstream PR history: #4673 introduced focus refresh; #5935 addressed Crossterm query ownership; #9947 attempted pause/resume; #20654 and #23175 introduced bounded/batched probes; #18575 introduced the measured per-draw resize query.

Requested maintainer action

Please review https://github.com/mehulparmariitr/codex/pull/1 and either:

  1. invite the same branch as an upstream PR, or
  2. open an OpenAI-owned PR using the two linked commits.

View original on GitHub ↗

7 Comments

github-actions[bot] contributor · 3 months ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #23702

Powered by Codex Action

bbingz · 1 month ago

Analysis (community)

Thanks for tracking this — attempting a residual-focused analysis aligned with docs/contributing.md (invitation-only: analysis in-thread, no unsolicited PR).

  • Issue under review: #24527 — TUI input lag while typing — worsens with session length (macOS arm64, terminal-agnostic)
  • Theme / radar cluster: TUI keystroke lag long sessions (tui-lag)
  • Reporter posture: static + local evidence where available; not claiming a maintainer invite.

Observed (from issue + public residual signal)

Bug Report: Typing Lag in Codex TUI ### Description While typing in the Codex CLI prompt, there is a noticeable input lag — keystrokes appear on screen with a delay. The lag is not consistent: it seems to worsen as the session grows longer (more turns / more context), which aligns with the O(n) TUI rebuild described in #21945. This is specifically keystroke-level lag (characters lagging behind as I type), not response time lag. Happens across all terminals — confirmed in Ghostty,

Root-cause hypothesis

Long-session keystroke lag is likely O(n) work on the UI thread (history paint, full re-layout, or blocking I/O on each key), distinct from skill-popup Enter (#34857) and resume/fork identity (#34972).

High-level fix outline (not a PR)

  • Profile input path: event → buffer → render under large scrollback.
  • Coalesce paint; virtualize history; never do disk/network on key path.
  • Fail-first: synthetic 10k-turn transcript + keystroke latency assertion.

Related

#24527 (thin analysis density), #13795, #11678.

Questions for maintainers

  1. Is the intended long-term policy documented for this area (especially sandbox gitdir / log TRACE defaults)?
  2. Preferred residual anchor issue if several open threads describe the same mechanism?
  3. If the high-level outline above is roughly aligned, would an invited PR with failing-first tests be welcome later — or is an internal fix preferred?

Happy to refine with more measurements or code pointers. No unsolicited PR from me.

bbingz · 1 month ago

Analysis (community)

Residual analysis for #24527 (TUI input lag grows with session length) — invitation-only. Distinct from skill-popup Enter (#34857) and resume/fork identity (#34972). No unsolicited PR.

Deterministic evidence (code + runtime)

Code (5dd992acd3):

  • Per-token TRACE in the hot markdown path: codex-rs/tui/src/markdown_stream.rs L78

tracing::trace!("push_delta: {delta:?}"); — formats the entire delta on every push.

  • Finalize path also TRACE-logs full source (markdown_stream.rs ≈L177).
  • TUI process attaches optional SQLite log layer (codex-rs/tui/src/lib.rs log_db: Option<log_db::LogDbLayer>), so those TRACE events compete with keystroke/render work for CPU and disk.
  • Chat history rendering keeps growing committed cells (codex-rs/tui/src/chatwidget.rs architecture: history cells + live tail), so paint cost scales with transcript length.

Runtime (codex-cli 0.146.0-alpha.6):

  • Local logs_2.sqlite shows codex_tui::markdown_stream = 15085 TRACE rows — third-highest TRACE target on this host.
  • Combined with codex_api::sse::responses (37252) and codex_app_server::outgoing_message (19558), streaming sessions generate continuous sink pressure while the user is typing.

Root-cause hypothesis

Keystroke latency is multi-factor:

  1. Unbounded transcript paint / lack of virtualization as history grows.
  2. Hot-path tracing::trace! of full deltas into a TRACE-default SQLite sink (LogDbLayer) on the same process.
  3. Coalescing gaps: every delta can force both format work and eventual DB insert batching (LOG_FLUSH_INTERVAL = 2s).

This is a TUI UX residual with a measurable logging co-factor, not a pure “make the model faster” report.

High-level fix outline (not a PR)

  1. Remove or gate push_delta TRACE behind an explicit debug flag (never format full delta at TRACE-by-default).
  2. Profile input → render with disk sink disabled; set p95 keystroke bound.
  3. Virtualize history cells; coalesce paints on resize/stream.
  4. Fail-first: synthetic large transcript + streaming deltas; assert p95 key latency and that markdown_stream TRACE count stays 0 under default filter.

Questions for maintainers

  1. Preferred harness location inside codex-tui for keystroke latency?
  2. Is demoting codex_tui::markdown_stream TRACE acceptable even before global default changes?
  3. Would an invited PR for virtualization/coalescing + TRACE gate be welcome?

No unsolicited PR from me.

bbingz · 1 month ago

Analysis (community) — short follow-up

Thanks @mehulparmariitr for the additional measurements on #24527 (still on codex-cli 0.145.0, ~2-day session, ~8 MB / 2.6k+ rollout records). That evidence is more useful than another restatement of the original report.

Root-cause hypothesis (updated)

Long-session keystroke lag on #24527 is multi-factor: primary unbounded transcript paint/layout on the composer path; amplifier full-delta TRACE + optional SQLite LogDbLayer on the same TUI process. Reporter data falsifies DB file size as the sole cause.

What your note changes in the residual picture

I previously pointed at a multi-factor path on main (5dd992acd3 / nearby):

  1. History paint / committed cells scaling with transcript (codex-rs/tui chatwidget-style cell list + live tail) — O(n) on the UI path as the session grows.
  2. Hot-path tracing::trace!("push_delta: {delta:?}") in codex-rs/tui/src/markdown_stream.rs (~L78) formatting full deltas, plus optional LogDbLayer SQLite sink under a TRACE-default filter (codex-rs/state/src/log_db.rs default_filter()).

Your controls help falsify pure “log DB file size” as the sole cause:

  • disable_paste_burst = true
  • no large workspace_roots
  • logs_2.sqlite reduced 282 MB → ~40 KB while keystroke lag remains

So the residual should be framed as:

  • Primary (best fit): unbounded transcript layout/paint (and related Markdown/wrap/composer work) on the input path as history grows — also consistent with full text present in rollout JSONL while the TUI visually truncates.
  • Amplifier (still real on 0.146.x TRACE-default builds, not sufficient alone): full-delta TRACE + SQLite sink pressure on the same process as the TUI. Shrinking an already-written DB does not remove the hot-path trace! formatting or streaming insert churn during active turns.

That matches your request list: key→frame latency vs transcript size; whether history/Markdown/layout runs on composer updates; whether TRACE/SQLite remains on the critical path; large-transcript p95 keystroke regression.

High-level fix outline (still not a PR)

  1. Virtualize / window committed history cells; coalesce paints so key events never walk full transcript.
  2. Gate or drop full-payload push_delta TRACE by default (log length/kind only).
  3. Fail-first: synthetic multi-k-turn transcript + p95 keystroke bound; assert markdown_stream TRACE count is 0 under default sink filter.
  4. Separate display-path bugs (truncation with complete JSONL) from model/API completeness.

Questions for maintainers

  1. Preferred owner surface for keystroke p95: codex-tui integration tests vs a micro-benchmark crate?
  2. Is demoting codex_tui::markdown_stream TRACE acceptable ahead of any global logs_2 default-level change?
  3. If the outline aligns, would an invited PR (virtualization/coalescing + TRACE gate + fail-first latency test) be welcome — or is this tracked internally already?

Happy to refine with more code pointers. No unsolicited PR from me. Invitation-only per docs/contributing.md.

mehulparmariitr · 1 month ago

Canonical report and reviewable fix

@fcoury-oai @etraut-openai, the issue description is now the concise canonical report, including the direct evidence, both fixes, post-fix validation, and the remaining scheduling limitation.

Direct upstream PR creation was rejected by repository permissions. Please either invite mehulparmariitr to submit this branch upstream or use the two commits as the basis for an OpenAI-owned PR.

mehulparmariitr · 1 month ago

Upstream status: focus fixed, terminal-size blocker remains

@fcoury-oai @etraut-openai, thank you for reviewing this area. Felipe's merged focus fix is:

That change removes focus-time palette queries and fixes one of the two synchronous blockers confirmed in this issue.

The remaining confirmed blocker is repeated synchronous Terminal::size() work during ordinary draws. The reviewable single-fix implementation is:

It is based on main with #35649 already present, contains no replacement focus logic, and keeps the private diagnostic instrumentation out of the public diff. The size change carries resize dimensions through the draw pipeline, reuses the last known size for unchanged scheduled frames, refreshes once after resume, and retains a delayed post-reflow recovery sample.

Direct evidence included size calls up to 9.282 s and an already-readable key waiting 4.328 s behind a 4.395 s size lookup. With the remaining patch applied, a PTY smoke run recorded 293 ordinary draws with zero terminal-size queries.

Because direct upstream PR submission is invitation-gated, could either of you please review or port this remaining change, or advise the appropriate path for getting it into an OpenAI-owned PR?

bbingz · 22 days ago

This closed on 2026-08-03 without a comment — one tracking question so the second half does not get lost. @fcoury-oai's #35649 (976129f) removed the focus-time palette queries, which was blocker 1 of the 2 confirmed in this thread; @mehulparmariitr noted on 2026-07-29 that repeated synchronous Terminal::size() work during ordinary typing was still present. Was that covered by a later change, or should it be split into its own issue? Happy to re-verify against current main either way.