Codex Desktop App becomes laggy because it scans all `~/.codex/sessions` rollout files instead of respecting Desktop-visible session index/state

Open 💬 24 comments Opened May 3, 2026 by 8th-block
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

What version of the Codex App are you using (From “About Codex” dialog)?

Version 26.429.30905 (2345)

What subscription do you have?

Pro and Business

What platform is your computer?

Darwin 25.3.0 arm64 arm

What issue are you seeing?

Summary

Codex Desktop App becomes slow/unresponsive when ~/.codex/sessions contains many rollout JSONL files.

The important point: these session files are not only created by Codex Desktop. They are also created by other Codex clients that share the same CODEX_HOME, including CLI, codex exec, app-server clients, VS Code-style clients, and subagents.

Desktop appears to scan/read the entire ~/.codex/sessions folder, even though ~/.codex/session_index.jsonl contains a much smaller set of sessions that seem to be the intended Desktop-visible history.

Clearing or moving files out of ~/.codex/sessions immediately fixes the Desktop UI lag.

Environment

  • Codex Desktop App: Version 26.429.30905 (2345)
  • Codex CLI: codex-cli 0.128.0
  • OS: macOS
  • Platform: Apple Silicon / Darwin
  • CODEX_HOME: default ~/.codex

Actual behaviour

Codex Desktop App becomes laggy as ~/.codex/sessions grows.

In my local case:

find ~/.codex/sessions -type f -name '*.jsonl' | wc -l
# 109

du -sh ~/.codex/sessions
# 78M

wc -l ~/.codex/session_index.jsonl
# 22

The Desktop app becomes responsive again after clearing/moving files out of ~/.codex/sessions.

What steps can reproduce the bug?

  1. Use Codex Desktop App with the default CODEX_HOME.
  2. Also use Codex CLI / app-server / other Codex clients that share the same CODEX_HOME.
  3. Let ~/.codex/sessions accumulate many rollout JSONL files.
  4. Open Codex Desktop App.
  5. Observe Desktop UI lag/slowness.
  6. Move or clear old files from ~/.codex/sessions.
  7. Reopen Codex Desktop App.
  8. Observe that the lag is resolved.

What is the expected behavior?

Codex Desktop App should not eagerly scan or parse every rollout JSONL file under ~/.codex/sessions.

Desktop should only load the sessions it needs for its UI, using a bounded/indexed source of truth such as:

  • ~/.codex/state_5.sqlite
  • ~/.codex/session_index.jsonl
  • Desktop-visible source filters
  • lazy loading when a specific thread is opened

The Desktop UI should not degrade just because other Codex clients share the same CODEX_HOME.

Evidence / investigation

The sessions folder contains rollout files from multiple sources, not just Desktop-created sessions.

Observed source breakdown included:

vscode: 64
exec: 22
cli: 3
subagent / guardian / thread_spawn: multiple

Meanwhile:

~/.codex/session_index.jsonl: 22 entries
~/.codex/sessions: 109 rollout files

Archived sessions are physically moved out of the hot sessions tree into:

~/.codex/archived_sessions/

and state_5.sqlite marks them as archived. This suggests that files physically present under ~/.codex/sessions are on the Desktop hot path.

Additional information

Suggested fix

Make Codex Desktop App avoid full eager scans of ~/.codex/sessions.

Possible approaches:

  • Use state_5.sqlite or session_index.jsonl as the bounded source for Desktop history.
  • Filter by Desktop-relevant sources before touching rollout JSONL contents.
  • Lazy-load rollout files only when opening a specific thread.
  • Avoid scan-and-repair work across all rollout files during normal Desktop startup or history rendering.
  • Add a way to hide/archive non-Desktop sessions from Desktop history.

The core issue is that Codex Desktop App appears to pay an O(number/size of all rollout files) cost for a shared sessions directory that is written by many Codex clients.

View original on GitHub ↗

24 Comments

github-actions[bot] contributor · 3 months ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #20802
  • #19517
  • #20103
  • #19483
  • #20269

Powered by Codex Action

ebarti · 3 months ago

I stumbled upon this in the past too - I even lost threads in Codex because of this. Since then, if I invoke codex programmatically or via the SDK I always set a different CODEX_HOME env var. The painful thing is you need to make sure the authentication in this secondary CODEX_HOME is valid

8th-block · 3 months ago

I just use a clean-up script for now. Use it at your own risk :-) it will probably break in the future when the app changes - they move fast

#!/usr/bin/env zsh
set -euo pipefail

usage() {
  cat <<'EOF'
Usage:
  archive-codex-non-desktop.zsh [--dry-run]

Archives Codex rollout sessions older than OLDER_THAN_HOURS where the first
JSONL session_meta payload has one of these originators:
  - symphony-orchestrator
  - repoprompt
  - reportprompt
  - codex-tui
  - codex_sdk_ts

Environment:
  CODEX_HOME          Defaults to "$HOME/.codex"
  OLDER_THAN_HOURS    Defaults to 12
EOF
}

dry_run=0
case "${1:-}" in
  --dry-run)
    dry_run=1
    ;;
  -h|--help)
    usage
    exit 0
    ;;
  "")
    ;;
  *)
    echo "Unknown argument: $1" >&2
    usage >&2
    exit 2
    ;;
esac

for dep in jq sqlite3 find stat date head mv mkdir cp; do
  if ! command -v "$dep" >/dev/null 2>&1; then
    echo "Missing required command: $dep" >&2
    exit 1
  fi
done

CODEX_HOME="${CODEX_HOME:-$HOME/.codex}"
SESSIONS_DIR="$CODEX_HOME/sessions"
ARCHIVE_DIR="$CODEX_HOME/archived_sessions"
DB="$CODEX_HOME/state_5.sqlite"
OLDER_THAN_HOURS="${OLDER_THAN_HOURS:-12}"

if ! [[ "$OLDER_THAN_HOURS" == <-> ]]; then
  echo "OLDER_THAN_HOURS must be a positive integer, got: $OLDER_THAN_HOURS" >&2
  exit 1
fi

originators=(
  "symphony-orchestrator"
  "repoprompt"
  "reportprompt"
  "codex-tui"
  "codex_sdk_ts"
)

[[ -d "$SESSIONS_DIR" ]] || { echo "Missing sessions dir: $SESSIONS_DIR" >&2; exit 1; }
[[ -f "$DB" ]] || { echo "Missing state DB: $DB" >&2; exit 1; }

cutoff_epoch=$(( $(date +%s) - OLDER_THAN_HOURS * 3600 ))
originator_filter_json="$(printf '%s\n' "${originators[@]}" | jq -R . | jq -s .)"

if (( dry_run )); then
  echo "Dry run: no files or database rows will be changed."
else
  mkdir -p "$ARCHIVE_DIR"
  cp "$DB" "$DB.bak.$(date +%Y%m%d-%H%M%S)"
fi

matched=0
archived=0
skipped=0

find "$SESSIONS_DIR" -type f -name '*.jsonl' -print0 |
while IFS= read -r -d $'\0' src; do
  mtime="$(stat -f %m "$src")"
  (( mtime < cutoff_epoch )) || continue

  meta="$(
    head -n 1 "$src" | jq -cer --argjson allowed "$originator_filter_json" '
      select(.type == "session_meta")
      | .payload as $p
      | select($allowed | index($p.originator))
      | {
          id: $p.id,
          originator: $p.originator,
          timestamp: $p.timestamp,
          cwd: $p.cwd
        }
    ' 2>/dev/null
  )" || continue

  matched=$(( matched + 1 ))

  id="$(jq -r '.id' <<< "$meta")"
  originator="$(jq -r '.originator' <<< "$meta")"
  cwd="$(jq -r '.cwd // ""' <<< "$meta")"
  dest="$ARCHIVE_DIR/$(basename "$src")"

  if [[ -e "$dest" ]]; then
    echo "SKIP exists: originator=$originator id=$id dest=$dest"
    skipped=$(( skipped + 1 ))
    continue
  fi

  if (( dry_run )); then
    echo "WOULD archive originator=$originator id=$id src=$src cwd=$cwd"
    continue
  fi

  mv "$src" "$dest"

  id_sql="${id//\'/\'\'}"
  dest_sql="${dest//\'/\'\'}"

  sqlite3 "$DB" "
    UPDATE threads
    SET archived = 1,
        archived_at = strftime('%s','now'),
        rollout_path = '$dest_sql'
    WHERE id = '$id_sql';
  "

  archived=$(( archived + 1 ))
  echo "Archived originator=$originator id=$id"
done

if (( dry_run )); then
  echo "Dry-run complete."
else
  echo "Archive complete."
fi
8th-block · 3 months ago

Those of you running heavily automated coding agents which generate lots of sessions. Use a separate CODEX_HOME and copy your auth in there from your regular codex home. HT(🎩): @ebarti

ebarti · 3 months ago

But I totally agree this should be something better handled natively by Codex. Not sure why they would think having programmatic and interactive sessions in the same session storage was a good idea 😓

8th-block · 3 months ago

it is simple. They are overworked and have different priorities ... been there

etraut-openai contributor · 3 months ago

All client instances (including the CLI and app) share a database (~/.codex/state_5.sqlite) that stores an indexed table of sessions. This eliminates the need for clients to scan all of ~/.codex/sessions at startup. If you are seeing large scans of the sessions directory, then something is very wrong. Perhaps the database has become corrupted, and Codex is attempting to rebuild it.

dm-8thblock · 3 months ago
All client instances (including the CLI and app) share a database (~/.codex/state_5.sqlite) that stores an indexed table of sessions. This eliminates the need for clients to scan all of ~/.codex/sessions at startup. If you are seeing large scans of the sessions directory, then something is very wrong. Perhaps the database has become corrupted, and Codex is attempting to rebuild it.

Thanks for your response I am merely posting my observations (& best guess what went wrong really). I can consistently reproduce the issue by running a large number of sessions via my local symphony, watch Codex App degrade and become laggy, then prune all my Symphony sessions stored in ~/.codex/sessions which then resolves fully the unresponsiveness with the Codex app snapping back to working well aagain. Tried on my Mac Studio and MacBook - same problem. Just FYI and hope this helps!

etraut-openai contributor · 3 months ago

How are you "pruning your sessions"? There is currently no app server API for deleting sessions; there is an API for archiving sessions. If you're deleting or otherwise manipulating the files under ~/.codex/sessions behind the back of Codex, that might explain why Codex is going in to "rebuild the database" mode when it starts.

dm-8thblock · 3 months ago
How are you "pruning your sessions"? There is currently no app server API for deleting sessions; there is an API for archiving sessions. If you're deleting or otherwise manipulating the files under ~/.codex/sessions behind the back of Codex, that might explain why Codex is going in to "rebuild the database" mode when it starts.

I was using that script above I posted earlier . Also just getting rid of the ~/.codex/sessions folder solved the lag initially during the investigation. that is how I narrowed down which piece was causing a problem for me

ebarti · 3 months ago

@etraut-openai I see extremely degraded performance since the last update, but I have been running Codex agents in a separate CODEX_HOME directory via the Codex SDK for a while now.

valdineipigajunior · 3 months ago

Here, the same thing happens. After a few days of conversation, the specific chat becomes very slow, both to open and minimize while it's processing data. If you open a new chat in the project, it will always be fast, until you chat for a while, then the slowness returns. It's quite amateurish how Codex handles this. The problem is that when you start a new chat, it always starts out "dumb," and you have to keep summarizing the slow chat and pasting it into the new, clean chat.

dm-8thblock · 3 months ago
Here, the same thing happens. After a few days of conversation, the specific chat becomes very slow, both to open and minimize while it's processing data. If you open a new chat in the project, it will always be fast, until you chat for a while, then the slowness returns. It's quite amateurish how Codex handles this. The problem is that when you start a new chat, it always starts out "dumb," and you have to keep summarizing the slow chat and pasting it into the new, clean chat.

Do not diss other people’s work! They are busy enough and now they have to listen to people like you who can’t conduct themselves in a respectful manner.

john0123412 · 1 month ago

Additional repro on Windows — same index vs. full-directory-scan pattern
Environment: Codex Desktop 26.707.3748.0 (codex-cli 0.144.0-alpha.4), Windows 11 25H2, x64, 39.7GB RAM (~23GB free), disk queue 0 — resource exhaustion ruled out.
Symptom: severe UI lag on Windows too — noticeable delay on typing and scrolling, not just macOS.
I checked my local ~/.codex/sessions directory and found the same pattern described in this issue:

session_index.jsonl (the file Desktop-visible history actually reads from): 65KB
Full sessions directory: 326MB across 162 rollout JSONL files, largest single file 40MB

This matches OP's finding that the index is tiny while the actual scanned directory is much larger, which supports the theory that Desktop is full-scanning every historical rollout file under sessions rather than only reading session_index.jsonl.
Process-level data: 9 Codex-Desktop-related processes running, 3 of them holding ~820MB RAM with sustained 40%+ CPU. I have sandbox = "elevated" configured as well, though I haven't isolated whether that specifically compounds the lag or is incidental — flagging it in case it's useful, not asserting causation.
Possibly related (unconfirmed same root cause, flagging for triage rather than claiming they're the same bug):

Discussion #29949 — multiple Windows users reporting general Desktop lag since June; mechanism there looks more like renderer/GPU CPU spikes and isn't clearly tied to the sessions-directory scan specifically.
#11984 — long-session renderer memory growth up to 5GB+; this looks like single-session bloat rather than many-old-files full-scan, so may be a distinct mechanism with overlapping symptoms.

Expected behavior: Desktop should only load/reference the sessions listed in session_index.jsonl rather than scanning every historical rollout file under sessions.

lyy0709 · 1 month ago

The CLI hits this same scan through codex-core, and I can add the code-level mechanism plus some real-world numbers (traced on codex-cli 0.144.4, paths unchanged on current main).

Where the scan comes from

  1. State-DB init gates startup on backfill: rollout/src/state_db.rs → try_init_with_roots_inner calls wait_for_backfill_gate, which runs backfill_sessions inline in a loop until BackfillStatus::Complete, up to STARTUP_BACKFILL_WAIT_TIMEOUT = 30s (1s poll). On timeout it returns an error, state-DB init fails for that launch, and the next launch resumes scanning from the watermark — so users see repeated ~30s startup stalls until the backfill eventually converges.
  2. backfill_sessions (rollout/src/metadata.rs) walks ~/.codex/sessions and ~/.codex/archived_sessions, and for every rollout calls extract_metadata_from_rollout → RolloutRecorder::load_rollout_items, which reads and parses the entire rollout JSONL into memory just to derive listing metadata (title/preview/timestamps/usage).
  3. The DB filename is version-pinned — state/src/lib.rs: pub const STATE_DB_FILENAME: &str = "state_5.sqlite" — so every schema bump starts a fresh DB with empty backfill_state and re-scans the whole tree from scratch. The scan is not a one-time cost; it comes back with upgrades.

Numbers from one real ~/.codex (macOS, CLI): 1,030 rollout files totalling 1.8 GB, largest single files 203 MB and 163 MB. A fresh backfill therefore reads ~2 GB from disk and materializes >200 MB single files in memory. The whole-file load_rollout_items materialization is probably also a contributor to the ingestion RSS blowups reported in #31040, and the 30s gate behavior matches #23787 / #28087.

Possible directions

  • Don't gate interactive startup on backfill completion: claim the lease, run the backfill in the background, and serve session listings best-effort as rows land (the gate currently buys listing completeness at the cost of a hard startup stall).
  • Extract listing metadata without materializing whole rollouts: the fields needed come from the head SessionMeta line plus a bounded tail (last timestamps/usage), so a capped head+tail read would replace the full-file parse; alternatively skip/degrade files above a size threshold.
  • Carry backfill_state (or at least the completed watermark) forward across schema bumps so a version upgrade doesn't rescan every historical session.
helenananaa · 1 month ago

Additional Windows A/B reproduction: GPT-5.6 Ultra subagent fan-out makes the full-session scan cost explode.

Environment

  • Codex Desktop: 26.707.12708.0
  • Windows 11 x64, build 26200
  • 31.7 GiB RAM
  • Default shared CODEX_HOME
  • SQLite quick_check passed; this was not database corruption

Strong model/mode correlation

I analyzed the persisted thread graph in state_5.sqlite (root threads plus thread_spawn_edges) after several days of normal heavy use.

Across 289 root trees / 816 total thread nodes:

| Cohort | Root trees | Total nodes | Avg nodes/root | Max nodes | Multi-node roots |
|---|---:|---:|---:|---:|---:|
| >=80% gpt-5.6-* + ultra | 20 | 537 | 26.85 | 180 | 19/20 |
| All other model/effort combinations | 269 | 279 | 1.04 | 5 | 3/269 |

The largest trees were:

  • 180 nodes: 180/180 gpt-5.6-sol + ultra
  • 97 nodes: 97/97 gpt-5.6-sol + ultra
  • 43 nodes: 43/43 gpt-5.6-sol + ultra
  • 36 nodes: 36/36 gpt-5.6-sol + ultra
  • 28 nodes: 28/28 gpt-5.6-sol + ultra
  • 22 nodes: 22/22 gpt-5.6-sol + ultra

For the 180-node tree, thread_source was 1 user root + 179 subagent children. The 97-node tree was 1 root + 96 subagents, and the 43-node tree was 1 root + 42 subagents. Nested depth-2 children were also present.

The tokens_used values in the state DB are apparently cumulative and extremely large (the 180-node tree summed to about 127.4B), so token count may not be a directly comparable usage metric. The node counts, model/effort fields, spawn edges, and rollout files are unambiguous.

Older long gpt-5.5 + high sessions could have large token counters, but were typically single-node threads. The sharp tree fan-out appeared after using GPT-5.6 Ultra heavily.

A/B recovery

Before cleanup:

  • 633 active rollout JSONL files
  • about 1.84 GiB under the active sessions tree
  • global Desktop lag, mouse/cursor stutter, frequent Not Responding states
  • Windows Task Manager repeatedly showed the physical disk at 100%
  • switching an unrelated conversation could freeze the entire app

Recovery procedure:

  1. Archive affected roots with the official codex archive <root-thread-id> command (recursive archive).
  2. Keep full backups.
  3. Move archived rollout files outside CODEX_HOME, because archived files are also on the backfill scan path described in this issue.
  4. Leave only current/recent small trees in the hot sessions directory.

After cleanup:

  • 15 active rollout files
  • 10.3 MiB active session footprint
  • UI immediately responsive
  • a 15-second cold-start sample showed physical disk average 3.26%, max 6.38%, max queue length 0.06
  • main window remained Responding=True

Reducing local text/GPU caches did not explain the durable recovery. The decisive variable was removing the large multi-agent rollout forest from the hot scan path.

Interpretation

This looks like an algorithmic scalability bug in backfill/list/hydration, not a GPT-5.6 inference bug by itself. GPT-5.6 Ultra is a strong amplifier because it creates dozens or hundreds of child rollouts per user-visible root. If startup/listing cost is proportional to all rollout bytes and all child files, a few Ultra sessions can turn an otherwise normal profile into an O(total rollout history) startup and UI workload.

It also creates a bad product tradeoff: keeping history accessible makes Desktop unusable, while moving it outside CODEX_HOME restores performance but removes it from the UI.

Suggested invariants

  • A user-visible root should be listable from bounded DB metadata without reading every descendant rollout.
  • Child/subagent threads should be represented as a summarized tree count in the root list, not eagerly hydrated individually.
  • Rollout metadata extraction should use the head plus a bounded tail/index, never whole-file materialization.
  • Interactive startup should not block on full backfill.
  • Archived rollout bodies should not be parsed on the normal startup/list path.
  • Add per-root diagnostics: descendant count, total rollout bytes, and a warning/repair action for pathological trees.
  • Provide cold archive/search/restore so history remains usable without remaining on the hot scan path.

This overlaps with #31198 (subagent rollout amplification), #32443 (many spawn edges/hydration), and #25779 (unbounded session-state meta issue), but the A/B above specifically confirms the sessions/backfill performance mechanism in this issue on a current Windows build.

drkvj · 28 days ago

Additional macOS reproduction on a current Desktop build, with measurements captured immediately after a fresh restart.

Environment

  • Codex Desktop: 26.727.40816 (bundle build 6067)
  • macOS on Apple Silicon
  • Default shared CODEX_HOME
  • No service incident was active on the OpenAI status page during capture

Fresh-restart behavior

Restarting did not resolve the problem. Within roughly 3–5 minutes of launch:

  • codex ... app-server stayed around 60–100% CPU
  • app-server RSS reached approximately 3.5–3.7 GB
  • one renderer reached approximately 1.2–1.3 GB RSS
  • basic Desktop interactions remained sluggish

This makes the restart recommendation ineffective for this profile: it releases the old processes, but the replacement app-server immediately returns to heavy work.

Session inventory

  • ~/.codex/sessions: 19,593 rollout JSONL files
  • total rollout size: 9.2 GB
  • session_index.jsonl: 3,164 rows, 417 KB
  • largest individual rollouts: approximately 1.2 GB, 1.2 GB, 1.1 GB, 908 MB, with several more in the 795–892 MB range

The large mismatch between the Desktop-visible index and the full rollout tree is consistent with the full-directory/backfill mechanism described above.

Process evidence

A read-only macOS sample of the hot app-server showed activity in:

  • serde_json string parsing
  • Tokio async filesystem reads
  • SQLite worker activity
  • response/websocket telemetry paths

No raw sample, logs, paths, task titles, prompts, repository names, screenshots, or transcript contents are attached because they may contain private project data.

The machine had about 195 GB free disk space, so this was not caused by a nearly full volume. OpenAI service status was also operational, making a local history/backfill path more likely than model or network latency.

Interpretation

This appears to be the same algorithmic scalability issue: after restart, interactive startup/listing or hydration performs work proportional to the accumulated rollout corpus rather than the small visible-session index. The extremely large individual active rollouts may additionally overlap with #21134.

Useful product behavior would be:

  • never parse entire multi-hundred-MB or GB rollouts for list metadata
  • keep backfill off the interactive startup path
  • use bounded head/tail metadata extraction or stored database metadata
  • avoid eagerly hydrating descendants and archived session bodies
  • provide a supported cold-storage mechanism that preserves searchable history without keeping every rollout on the hot scan path
  • warn when a session tree or total active history is large enough to degrade Desktop
harryshawk · 26 days ago

Additional reproducible evidence from Codex Desktop on macOS.

Feedback Ref ID: 019fbf2d-6cdf-74d3-9ba9-b8649ce6931c

Environment

  • macOS 26.5.2, Apple Silicon
  • Codex session storage on an external APFS volume under /Volumes/Expansion/CodexHome/sessions

Observed behavior

After the prompt “I have the OAI finance plugin,” the Codex app-server launched five overlapping rg processes. Each searched the entire session archive using progressively longer versions of the same prompt:

rg -l --fixed-strings --ignore-case --no-ignore --glob '*.jsonl' -- '<progressively longer prompt prefix>' /Volumes/Expansion/CodexHome/sessions

Earlier searches were not cancelled when newer searches began. The five processes:

  • persisted for roughly 4–10 minutes;
  • were direct children of the Codex app-server, not sub-agents;
  • consumed approximately 1.4–2.2 GB RSS combined;
  • spent substantial time in disk-I/O wait;
  • caused severe UI and typing lag, including delayed character echo.

Pausing the task and its sub-agents did not stop these searches.

Secondary diagnostic failure

When asked what the five processes were, the task initially claimed they were retained sub-agents from earlier work and said it was closing completed agents. That diagnosis was factually incorrect: PID/process-tree inspection showed five active rg children of the app-server. The task corrected itself only after the independent process evidence was presented.

Ruled out

A newly mounted external NTFS drive was unrelated. It was mounted read-only with Spotlight indexing disabled, and the runaway processes targeted only the Codex session archive on the APFS volume. No thermal, disk-space, or malware signal was observed.

Expected behavior

  • Use an indexed or bounded session lookup.
  • Cancel or replace an older search when a newer search supersedes it.
  • Pausing a task should cancel its associated background retrieval work.
  • Diagnostics should distinguish sub-agent records from operating-system processes.

This appears related to the broader runaway-rg behavior reported in #13717, but this reproduction specifically involves Codex Desktop session-history retrieval and an external CODEX_HOME session store.

joeblowma · 25 days ago

Ohhh! So _that's_ what _that_ was... in my experience:

Environment

  • Windows 11 25H2
  • Windows Store Codex app install (no idea what version, scrapped when I saw this happen)

Observed behavior

Opened the app, did nothing except check usage, found it consumed 3% of weekly plus usage. Even saw the spike on a usage graph earlier today... that I can't seem to even find again now.

Expected behavior

  • Opening an app and doing nothing should cost exactly that much, nothing
  • bare minimum: transparency. Throw a yes/no dialog, make it possible to "nope out" even if that means the app closes

Additional comments

Uninstalled. Then spent a good while hunting around for the stuff it _didn't_ uninstall, well outside the installation directory. Nearly 2GB of electrum and other backing apps/data, still possibly more I've yet to discover.

Caveat: I'm not going to reinstall this app to find out if it's fixed. Ever. It's enough work already just keeping codex-cli functional. Thanks for the fish!

shleder · 11 days ago

The shared ~/.codex/sessions inventory here is a useful scale/discovery test. I’m field-testing codex-rescue==0.1.0a4 against large mixed-source session stores.

If that state still exists, could you run:

pipx install codex-rescue==0.1.0a4
codex-rescue sessions

and then a read-only check on one representative rollout:

codex-rescue doctor --json "/path/to/a-representative-rollout.jsonl"

This won’t fix Desktop’s eager scanning behavior, but it can validate whether Rescue discovers the mixed store safely and whether representative persisted histories are structurally healthy.

Please share only sanitized counts/status/findings—no raw rollouts, DBs, prompts, tool output, credentials, repository data, session IDs, or private paths.

joeblowma · 11 days ago

@shleder state hasn't changed aside from a few more cli sessions. Thanks for making me look, if the GUI parsed all of those with only 3% inference use... I guess that was actually kinda efficient 😆

codex-rescue sessions showed me 20 sessions and emitted nothing that looked like a problem. Not that I'm sure what I'd be looking for.

Still have a bunch of rollouts going back to late Feb. Did a quick powershell script that spews the resulting doctor output to a file named after/beside the rollout, and ran doctor against all of them.

Basically every session at or before May 6 (my next session after that was may 19) is loaded with UNKNOWN_OPERATIONAL_SCHEMA. I have a couple "status": "OVERSIZED_PAYLOAD", a single UNFINISHED_TOOL_CALL, the rest say HEALTHY.

Anything in particular you'd like me to look for? Really should clean these up, 77 items weighing in at 126MB, largest single file 22.4MB.

Script I used

$WorkPath = "C:\Users\YourUserName\.codex\sessions"

Get-ChildItem -Path "$($WorkPath)\*.jsonl" -Recurse | ForEach-Object {
    Write-Host "Found file: $($_.FullName)"
    $OutputFile = $_.FullName+"_doctor.txt"
    $out = (codex-rescue doctor --json $_.FullName 2>&1 | Out-String).Trim()
    $out | Out-File -FilePath $OutputFile -Encoding utf8
}
thedjpetersen · 5 days ago

Additional exact Cmd-K reproduction on the current unified ChatGPT/Codex desktop build.

Environment

  • ChatGPT/Codex Desktop: 26.803.61601 (bundle build 6396)
  • macOS 26.5.2, Apple Silicon
  • Large mixed-source local Codex history
  • Synthetic queries only; no raw history, paths, titles, or logs attached

Exact trigger and timing

  1. Press Cmd-K.
  2. Type a short query.
  3. The renderer uses in-memory title/metadata matching for the first two characters.
  4. At the third character it enables content results and, after a 200 ms debounce, invokes local app-server thread/search.
  5. A direct JSON-RPC benchmark against the app's bundled app-server reproduced the stall without Electron or network search:
  • initialize: ~65 ms
  • synthetic zzz content search: ~8.1 s
  • unrelated synthetic no-match: ~0.45 s

Profiled hot path

A read-only macOS sample during thread/search showed:

  • codex_rollout::search::content_match_snippet
  • Tokio async file reads / kernel read
  • serde_json parsing
  • regex matching

This confirms Cmd-K's third-character transition is reparsing rollout JSONL and building snippets on the interactive search path. It is not a GPU/renderer issue: the backend RPC alone reproduces it. It is not primarily network search either.

The renderer query code also supplies no abort signal or latest-query-wins cancellation. A typist who pauses for >=200 ms can therefore start overlapping content scans, consistent with the incremental-search process evidence already reported above.

Suggested fix

  • Keep title/metadata results immediate.
  • Put searchable user/assistant text in a bounded FTS index; exclude large tool/image payloads.
  • Cancel superseded thread/search requests, or enforce latest-query-wins in app-server.
  • Until indexing is available, require an explicit content-search action or a longer idle threshold instead of scanning at character 3.

This narrows #20864 from general history scanning to a deterministic Cmd-K call site and backend hot path. cc @thsottiaux

thedjpetersen · 5 days ago

Deeper profile from the installed macOS build 26.803.61601 (6396), renderer build commit a59c4c74cf79511086d3f7eb123d5d5052067680.

I extracted the shipped ASAR and traced the exact path:

  • cel("root", query) enables content results at 3 characters (hel = 3).
  • stl debounces with Ctl = 200, then creates a distinct React Query key per term and calls every host with search-threads-for-host.
  • That queryFn ignores React Query's signal; the host searchThreads method sends thread/search as a background request with no cancellation path.
  • Each superseded term therefore continues scanning rollout files. The UI hides stale results by comparing debounced/current text, but it does not stop the work.
  • The no-result component (utl) is a disabled "No matches" item; there is no create-chat action carrying the typed phrase.

Native app-server measurements using the installed binary and synthetic non-matching terms:

  • Before local history cleanup: one 3-character search took 8.08 s across ~486 active rollouts.
  • After archiving 456 inactive sessions: warmed single searches are 49–77 ms.
  • One-search control profile: 61.9 ms, 35 process threads.
  • 20 deliberately overlapping searches (stress test): median 600 ms, p90 1.53 s, max 1.68 s, and 141 process threads.
  • The overlapping sample is dominated by directory traversal, Tokio file reads/blocking-pool contention, regex DFA work, serde_json parsing, and codex_rollout::search::content_match_snippet.
  • At a 200 ms cadence after cleanup the searches no longer overlap (all 49–77 ms), which validates cancellation/backpressure as the amplification fix rather than the sole base-cost fix.

The long-lived live app also retained the pre-cleanup state: after ~69 hours its primary renderer was 1.26 GB RSS, app-server 1.08 GB, and Electron main 593 MB. A restart is required for the local mitigation to fully take effect.

Recommended product changes:

  1. Index content search instead of scanning/parsing rollout JSONL on every term.
  2. Make command-menu search latest-only: accept the React Query signal, cancel the IPC/app-server request, and check cancellation between files. At minimum serialize/coalesce by host.
  3. Keep input urgent with useDeferredValue/startTransition; memoize QWa and eGa, which currently rebuild/filter/sort loaded chat metadata in stl on each input render.
  4. Delay content search longer than title/command filtering (or require explicit chat-search mode); three characters + 200 ms is too aggressive for a filesystem scan.
  5. Replace disabled "No matches" with "New chat: {query}" and prefill the new composer.

@thsottiaux this includes the exact shipped call chain and native contention profile. Happy to provide sanitized sample output privately if useful.

shleder · 3 days ago

Your large shared-session-store results are preserved in bounded discovery tests. Vetto scans recursively beyond the legacy 20-item window and fails closed on explicit entry, per-session, and aggregate-byte budgets.

Migration update: active Codex Rescue development has moved to Vetto: https://github.com/shleder/vetto. The standalone shleder/codex-rescue repository remains public as compatibility history. User installation is now npm install --global @shleddy/vetto@next, with recovery under vetto rescue. The boundary remains read-only/copy-only and does not modify session JSONL or vendor SQLite. The newest diagnostic changes are merged into Vetto main and will be included in a later npm alpha; no source install is requested.