TUI /resume picker can block on global rollout scan despite cwd filter

Open 💬 7 comments Opened May 10, 2026 by majiayu000
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

What version of Codex CLI is running?

codex-cli 0.130.0

Local source checkout used for code-path inspection: openai/codex at cac5354455.

What subscription do you have?

N/A for this report. This is local CLI/TUI resume picker behavior before a model request is made.

Which model were you using?

N/A. The observed latency is in local session listing / resume picker loading.

What platform is your computer?

Darwin 25.2.0 arm64 arm

What terminal emulator and version are you using (if applicable)?

VS Code integrated terminal, vscode 1.109.5.

What issue are you seeing?

The TUI /resume picker can be slow in profiles with many local rollout files because the cwd-filtered resume list still goes through a filesystem-first scan of the global rollout tree.

This is a narrower CLI/TUI variant of the larger local-history performance class discussed in openai/codex#18693, and it also overlaps with resume listing correctness issues such as openai/codex#20165 and openai/codex#21619. The specific problem here is the first-screen /resume list path: even when the picker is scoped to the current working directory, the storage layout and listing path can force global rollout scanning before the user sees the picker results.

The current flow appears to be:

  1. /resume opens the TUI resume picker.
  2. The picker builds a thread/list request with a cwd filter and source filter.
  3. The request still sets use_state_db_only: false.
  4. The backend uses filesystem-first listing so it can repair / validate SQLite metadata.
  5. For updated_at ordering, the rollout list path must scan files because updated time is not encoded in filenames.
  6. After selecting a session, cold resume can additionally load the full rollout JSONL into memory.

Relevant code paths from current main:

  • codex-rs/tui/src/resume_picker.rs: thread_list_params includes cwd/source filters but sets use_state_db_only: false.
  • codex-rs/rollout/src/recorder.rs: list_threads_with_db_fallback performs filesystem-first listing and read-repair before returning filtered listings.
  • codex-rs/rollout/src/list.rs: the UpdatedAt sort path documents that it must scan files up to the scan cap because updated_at is not encoded in filenames.
  • codex-rs/thread-store/src/local/read_thread.rs and codex-rs/rollout/src/recorder.rs: cold resume history loading ultimately calls RolloutRecorder::load_rollout_items, which uses tokio::fs::read_to_string(path) for the rollout.

Local measurement from my profile:

~/.codex/sessions rollout files: 4829
~/.codex/sessions size: 2.1G
largest rollout file: 141,283,019 bytes
state_5.sqlite threads: 4541 total, 330 cli/vscode, 1 for the current cwd
raw stat/sort of ~/.codex/sessions rollout files: 10.70s

For comparison, Claude Code stores transcripts under per-project directories:

~/.claude/projects/<sanitized-cwd>/<session-id>.jsonl

On the same machine, scanning one heavy Claude project directory and sorting by mtime took about 0.74s. The point is not that Claude's whole profile is smaller; the full ~/.claude/projects tree has 6317 jsonl files. The faster common path comes from physical per-project partitioning, so "continue current directory" does not need to inspect all projects first.

What steps can reproduce the bug?

  1. Accumulate many Codex local rollout files under ~/.codex/sessions, especially across multiple projects.
  2. Make sure only a small subset belongs to the current working directory.
  3. Open Codex TUI in one project directory.
  4. Run /resume.
  5. Observe that the resume picker can take noticeable time to show or refresh results.
  6. Inspect the listing path: the picker passes a cwd filter, but because use_state_db_only is false, the backend can still scan the global rollout tree and repair/validate metadata before returning the page.

A local way to estimate the worst-case scan pressure is:

find ~/.codex/sessions -type f -name 'rollout-*.jsonl' | wc -l
find ~/.codex/sessions -type f -name 'rollout-*.jsonl' \
  | while IFS= read -r f; do stat -f '%m %z %N' "$f"; done \
  | sort -nr \
  | head -25 >/dev/null

What is the expected behavior?

Opening /resume in the TUI should make the common current-directory picker path fast and bounded by current-project metadata, not by the number of rollout files across all projects.

In particular, if the SQLite state DB already has current cwd/source/provider metadata, the first page should be able to render from that index without waiting for a global filesystem scan/repair.

Additional information

Suggested phased fix:

  1. For the default TUI /resume current-cwd list, try use_state_db_only: true first.
  2. If the DB returns no usable results, errors, or the user explicitly selects a global/show-all mode, fall back to the existing filesystem scan path.
  3. Move filesystem repair/reconciliation for normal picker opening to a background task so it does not block the first visible page.
  4. Longer term: add a per-cwd sidecar/index or encode enough metadata to avoid global rollout scans for project-scoped listing.
  5. Separately, consider making cold resume avoid synchronously reading very large rollout files in full before the UI can recover.

This keeps the existing correctness fallback while making the common interactive path closer to a DB-first, project-scoped resume picker.

If this direction matches the maintainers' intent, I am happy to send a small invited PR for the Phase 1 change.

View original on GitHub ↗

7 Comments

github-actions[bot] contributor · 2 months ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #20864
  • #21619

Powered by Codex Action

majiayu000 · 2 months ago

Thanks for the duplicate check. I reviewed #20864 and #21619.

I think this issue is related but narrower:

  • #20864 is about Codex Desktop startup/UI lag from scanning ~/.codex/sessions.
  • #21619 is about resume --all omitting resumable sessions due to scan limits.
  • This issue is specifically about the TUI /resume picker first page: even with a cwd filter, the picker sends use_state_db_only=false, so the app-server can still take the filesystem-first scan/repair path before returning current-directory results.

So the proposed Phase 1 fix here is intentionally small: make the default current-cwd TUI /resume picker try the SQLite indexed path first, and keep filesystem scan as a fallback / repair path.

betaHi · 2 months ago

I hit a similar issue with many local Codex sessions.

This does not fix the underlying /resume performance bug, but I made a small local-only TUI workaround for browsing/searching/resuming sessions:
https://github.com/betaHi/codex-session-browser

Sharing as a UX reference/workaround.

minsing-jin · 2 months ago

I have a proposed fix available in my fork, but PR creation is restricted to collaborators on this repository.\n\nBranch: https://github.com/minsing-jin/codex/tree/fix/resume-picker-indexed-filter-results\nCommit: https://github.com/minsing-jin/codex/commit/a45684b388280879b6b995ccf0e2d46eb3bbda2b

Gusarich · 1 month ago

Having the same issue for a long time! Below are the measurements on my machine.

~/.codex/sessions rollout files: 60,930
~/.codex/sessions size: 50G
~/.codex/state_5.sqlite size: 11G
state DB threads total: 60,650

raw stat/sort over rollout files: 12.43s real
DB current-cwd count query: 0.00s real

0xdevalias · 1 month ago

Partially adjacent: rust-v0.140.0 includes #27031, which speeds cold resume by reusing the history-bearing StoredThread loaded during the running-thread probe. That may improve latency after selecting a large session.

  • #27031

It does not appear to address the /resume picker's first-page listing path or the global rollout scan despite cwd filtering, which is the main issue described here. So this is probably not resolved, but post-selection resume latency may be improved on 0.140+.

bobaoapae · 6 days ago

I can reproduce this on Windows with a substantially larger local history, and I traced the current TUI path on main (f90e7deea6).

Environment and local scale

  • Codex CLI: 0.144.4
  • OS: Microsoft Windows NT 10.0.26200.0 x64
  • Terminal: Windows Terminal / PowerShell
  • Active rollout JSONL files: 28,062
  • Rows in state_5.sqlite: 28,062

On this profile, a read-only SQLite COUNT(*) completes in roughly 10 ms, and representative filtered/paginated state-DB queries complete in approximately 3–14 ms. By contrast, opening /resume can take several minutes before showing useful results, regardless of the launch directory.

Confirmed cause

The picker already has cursor pagination at the UI layer, but every page is currently requested with ThreadListParams.use_state_db_only = false. This selects the filesystem scan-and-repair path before the indexed results can be displayed. Consequently, the time to first useful frame is coupled to the global rollout population rather than the requested page.

This is specifically picker latency, not thread/resume hydration after selecting a thread.

Proposed correction

I implemented and tested a narrow DB-first policy in the TUI:

  1. Represent the page source with typed cursors: StateDb(Option<String>) and ScanAndRepair(Option<String>).
  2. Request the initial page and subsequent indexed pages with useStateDbOnly: true.
  3. Preserve the complete session list through normal cursor pagination; there is no row cap or silent history truncation.
  4. After indexed pages are exhausted, retain scan-and-repair as a fallback.
  5. Do not start that expensive fallback merely because the indexed page is short or empty. Trigger it only through explicit downward navigation or an active search.
  6. If SQLite is empty or unavailable and therefore provides no selectable row, pressing Down still starts scan-and-repair, so the fallback remains reachable.
  7. Keep the existing cross-page deduplication when scan results overlap indexed results.

This makes the common first-page path proportional to the requested SQLite page while preserving access to unindexed rollouts and all existing sessions.

Coverage

The TUI tests cover:

  • initial state-DB request;
  • state-DB cursor continuation;
  • state-DB exhaustion and transition to scan-and-repair;
  • scan cursor continuation and exhaustion;
  • no eager scan prefetch;
  • search-triggered fallback;
  • explicit navigation recovery from an empty/unavailable state DB;
  • existing cross-page deduplication behavior.

Local validation:

  • just test -p codex-tui resume_picker::tests::: 93/93 passed
  • just fix -p codex-tui: passed
  • full codex-tui run: 3030 passed, with 6 unrelated Windows environment/locale failures (number formatting, %USERPROFILE% snapshot expectations, and Kitty image handling)
  • Rust formatting and git diff --check: passed

Patch available for review

I have not opened an unsolicited PR because docs/contributing.md requires an explicit maintainer invitation. If this DB-first/fallback policy matches the intended direction, I would be happy to open the prepared PR.

Related: #22411 documents the same scan-vs-index cost at the app-server level.