app-server loads ALL session files on every thread/list call — causes high CPU, slow startup, and background token waste

Additional environment detail:

The affected setup is the official VS Code extension:

• Extension identifier: openai.chatgpt
• Name: Codex – OpenAI’s coding agent
• Version: 26.42.71525
• Auto update: disabled
• Running in a remote SSH workspace, so the app-server/session files are on the server side, not on my local laptop.

A bounded check of current openai/codex main suggests that one inspected thread/list path is already metadata/head-only. So this may be specific to the extension/app-server version above, triggered by the SSH remote extension setup, or caused by another app-server path used during sidebar refresh/reconnect.

The practical workaround that removed the high CPU behavior was archiving old ~/.codex/sessions files on the remote server and keeping only a few recent sessions.

I’m happy to provide sanitized file counts/sizes and logs after I rest.

I opened a separate issue with additional local evidence that may overlap with this all-session/list loading path: #24510. It includes state_5.sqlite active metadata size, the observed thread-list query/index plan, logs_2.sqlite TRACE/log evidence, long-running goal-session rollout stats, and a temporary mitigation script gist.

I can confirm this issue on a current Codex Desktop → Remote SSH → Linux app-server → Android remote-control setup, with a much larger session population and a clean before/after remediation.

Environment

• Codex Desktop client observed by app-server: 26.707.61608
• Linux app-server runtime: 0.144.1
• Linux PATH CLI: 0.142.3
• Remote client: codex_chatgpt_android_remote (API v2)
• Transport: ChatGPT remote-control relay to a Unix-socket app-server on Linux

Reproduction and timing

The Linux state store had:

• 49,133 active threads matching source='exec' AND has_user_event=0
• 3,215,116,658 bytes of rollout JSONL for those threads
• no missing, duplicate, or out-of-tree rollout paths
• healthy state_5.sqlite

During one Android connection attempt, app-server logs recorded five consecutive thread/list requests on new remote-control connections. They took 34s, 36s, 35s, 39s, and 41s before Android retried. Every resumed rollout reported parse errors: 0.

A direct app-server protocol comparison on the same profile showed:

• default scan-and-repair thread/list: 45.77s first run, 39.17s second run
• after compacting several large interactive rollouts: still 30.06s
• thread/list with useStateDbOnly: true: 0.173s

The generated protocol schema explicitly describes omitted/false useStateDbOnly as scanning JSONL rollouts to repair metadata. The Android request omitted this optimization.

Controlled remediation

I made a consistent SQLite backup and manifest, then reversibly archived only inactive rows matching:

source = 'exec'
AND has_user_event = 0
AND archived = 0

Rollouts updated during the preceding hour were excluded to avoid touching active automation.

Results:

• 49,090 inactive no-user exec threads archived
• 3,212,078,641 bytes moved out of the active sessions tree
• active JSONL count reduced to 246
• 46 recent no-user exec sessions left active
• current and backup SQLite databases both passed PRAGMA quick_check
• every manifest source/destination/count/size reconciled

Immediately afterward, the Android app connected and loaded all conversations successfully. No app-server restart, Linux reboot, account change, or network change was needed.

Why automatic maintenance appears necessary

Even after cleanup, 46 no-user exec sessions had been created or updated within the preceding hour. This profile will regress without retention/archival.

Expected behavior would be one or more of:

1. Completed no-user exec threads are automatically archived or excluded from interactive thread discovery.
2. Remote/mobile thread/list uses the state-DB-only path and reconciles rollouts asynchronously.
3. Filesystem repair work is bounded/incremental and cannot exceed the remote client's timeout.
4. Thread retention/compaction prevents unbounded accumulation.

I have not attached raw logs, rollout contents, account identifiers, hostnames, or local project paths, but can provide additional sanitized aggregate diagnostics if useful.

Hi, thanks for documenting this.

I noticed one current downstream detail that may be relevant to the same rollout-scan root cause:

• At audited upstream commit 5157493, latest_thread_cwd calls tokio::fs::read_to_string on the complete rollout and scans its lines in reverse to recover the latest TurnContext working directory.
• In resolve_resume_thread_id, codex exec resume --last performs that read serially for each candidate returned by thread/list until the CWD matches.
• The same path computes latest_thread_cwd before checking args.all, so resume --last --all still reads the first candidate rollout completely even though no CWD filtering is required.

This appears to be a separate client-side full-file pass after thread/list. Improving the app-server listing and repair path alone may therefore leave codex exec resume --last reopening candidate rollout files to recover metadata that could potentially come from the state DB or thread projection.

I have not measured the incremental latency of this second pass. The evidence above establishes only the current control flow and the fact that its work scales with the number and size of candidates inspected.

Would it make sense for exec resume to short-circuit --all before calling latest_thread_cwd and otherwise consume an authoritative latest-CWD metadata projection instead of reopening each candidate rollout?

I checked all relevant issues, comments, pull requests, discussions, and release notes; this evidence is not already reported.

I am only reporting this finding and am not proposing a pull request unless a maintainer invites one.

Disclosure

Investigated thoroughly with GPT-5.6 Sol (runtime-default reasoning effort), using Oh My Pi as the agent framework.

This report is not generic or unreviewed AI-generated output. Its claims were checked against the cited evidence, and it includes the relevant detail intended to help maintainers resolve the issue.

If reports like this are not useful to the project, please let me know and I will refrain from submitting similar ones. My intent is to help without wasting maintainer time or energy or discouraging their work.

Thank you for your work.

Author of this issue here.

I see #31305 ([thread-store] let stores own derived metadata) has been open since early July. If that work is paused, or if its scope is deliberately narrower than this issue, I'd rather take a small piece off your hands than duplicate anything.

The part I have in mind is the one @MikeeI described above, which looks separable from the app-server listing path: latest_thread_cwd reads a whole rollout and scans it in reverse, resolve_resume_thread_id does that serially per candidate, and resume --last --all still pays for the first candidate even though no CWD filtering is needed.

Two things I'd want to be honest about before anyone spends time on this. I can't reproduce the scale reported in this thread — my own profile is small, so the evidence above is rnul's and MikeeI's, not mine. And the change I'm describing is control-flow you can read and cover with a test, not something I could validate under real load.

I'd only start if a maintainer says it's useful and not already in flight. Otherwise I'm happy to leave it be.