thread/list filesystem fallback can hide older matching threads after hitting 10,000-file scan cap

Open 💬 2 comments Opened Apr 2, 2026 by afurm

What version of Codex CLI is running?

codex-cli 0.116.0

What subscription do you have?

Plus

Which model were you using?

N/A — this appears to be a local thread listing / resume-history bug, not model-specific

What platform is your computer?

Darwin 25.4.0 arm64 arm

What terminal emulator and version are you using (if applicable)?

Codex desktop app terminal

What issue are you seeing?

thread/list can falsely report that there are no more matching threads when the server is using filesystem fallback and the rollout scan hits the hard 10,000-file cap before reaching older matching sessions.

Impact:

  • history UIs can appear incomplete
  • filtered thread/list calls can return data = [] with nextCursor = null even though older matching threads still exist
  • fallback latest-session lookup can stop early and behave as if no matching session exists

This appears to affect the rollout-scanning path rather than the SQLite-backed path.

From reading the local source:

  • hard cap is MAX_SCAN_FILES = 10000 in codex-rs/rollout/src/list.rs
  • the traversal code only marks more_matches_available = true after hitting the cap if at least one matching item was already found
  • so if the newest 10,000 rollout files are all filtered out by provider/source/cwd, the page can terminate as if exhausted even though older matching threads still exist

Relevant code:

  • codex-rs/rollout/src/list.rs around the scan cap handling in:
  • traverse_directories_for_paths_created
  • traverse_directories_for_paths_updated
  • traverse_flat_paths_created
  • traverse_flat_paths_updated

In each case the logic is effectively:

let reached_scan_cap = scanned_files >= MAX_SCAN_FILES;
if reached_scan_cap && !items.is_empty() {
    more_matches_available = true;
}

That !items.is_empty() condition appears incorrect for filtered queries.

What steps can reproduce the bug?

I have not yet reproduced this from a normal end-user session; this report is based on source analysis of the local checkout.

A synthetic repro should look like this:

  1. Create more than 10,000 rollout files under ~/.codex/sessions/....
  2. Ensure the newest 10,000 files do not match a chosen filter, for example:
  • different model_provider, or
  • different source, or
  • different cwd
  1. Ensure there are older rollout files beyond that scan window that do match the filter.
  2. Call thread/list with that filter.
  3. Observe that the filesystem fallback path can return:
  • data = []
  • nextCursor = null

even though older matching threads still exist.

High-level repro shape:

  • newest 10,000 rollouts: provider = skip_provider
  • older rollouts after that: provider = target_provider
  • query: thread/list with modelProviders = ["target_provider"]

Expected:

  • pagination should continue, or the response should otherwise indicate that older matches may still exist

Actual:

  • the filesystem path can terminate as if exhausted once the scan cap is hit before any matching item is found

I found the relevant control flow here:

  • codex-rs/rollout/src/list.rs
  • codex-rs/rollout/src/recorder.rs fallback listing path

What is the expected behavior?

When filesystem scanning hits the 10,000-file cap before finding a matching thread, thread/list should not claim the result set is exhausted.

Expected behavior:

  • if the scan cap is hit before the server can prove exhaustion, the response should preserve a continuation path rather than returning nextCursor = null
  • older matching threads should remain discoverable even when many newer non-matching rollouts exist
  • fallback latest-session/history lookup should not stop early and behave as if there is no matching session

Additional information

Additional code references:

  • hard cap:
  • codex-rs/rollout/src/list.rs
  • filesystem fallback entrypoint:
  • codex-rs/rollout/src/recorder.rs
  • SQLite-backed list path does not appear to have this exact failure mode:
  • codex-rs/rollout/src/state_db.rs

I also checked existing open issues/PRs and did not find a direct report for this specific scan-cap behavior.

This looks especially relevant for users with:

  • large local thread history
  • provider/source/cwd filters
  • fallback listing paths when SQLite is unavailable, incomplete, or bypassed

View original on GitHub ↗

2 Comments

afurm · 4 months ago

Sharper repro

This looks reproducible with a minimal filtered-history setup on the filesystem fallback path:

  1. Create more than MAX_SCAN_FILES rollout files.
  2. Make the newest MAX_SCAN_FILES files all fail the filter, for example:
  • model_provider = "skip_provider"
  1. Add at least one older rollout beyond that window that does match:
  • model_provider = "target_provider"
  1. Call thread/list with:
  • limit = 1
  • modelProviders = ["target_provider"]

Expected:

  • the API should preserve a continuation path so older matching threads remain discoverable

Actual:

  • the filesystem path can return data = [] and nextCursor = null once the scan cap is hit before any matching item is found

Why I think this happens

In the rollout traversal code, hitting the scan cap only sets more_matches_available when at least one matching item was already collected.

The relevant pattern appears in all four traversal variants in codex-rs/rollout/src/list.rs:

let reached_scan_cap = scanned_files >= MAX_SCAN_FILES;
if reached_scan_cap && !items.is_empty() {
    more_matches_available = true;
}

That means filtered queries can terminate as "exhausted" even when the scan merely stopped early.

Fix direction

I think the continuation token should be based on the last scanned (ts, id) rather than the last returned item when the scan cap is hit.

Today, even if more_matches_available were forced to true, build_next_cursor(&items) still cannot produce a cursor when items is empty.

So the likely fix is:

  • track the last scanned cursor candidate during traversal
  • when reached_scan_cap is true, emit nextCursor from that last scanned position, even if no matching items were returned on that page

Regression test idea

A focused regression test in either:

  • codex-rs/rollout/src/tests.rs
  • or codex-rs/app-server/tests/suite/v2/thread_list.rs

could assert that a filtered query with many newer non-matching rollouts does not falsely terminate at the scan cap.

If this direction looks right to the maintainers, I am ready to prepare a focused regression test and fix for it if invited.

afurm · 4 months ago

@etraut-openai I wanted to check whether the analysis and proposed fix direction here align with the intended behavior.

If this matches how you’d want thread/list / fallback pagination to behave, I’d be happy to prepare a focused regression test and a small PR for it if invited.

If not, I’d also appreciate any guidance on the expected behavior or preferred fix direction so I can adjust the proposal.