Codex Desktop App becomes laggy because it scans all `~/.codex/sessions` rollout files instead of respecting Desktop-visible session index/state

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• #20802
• #19517
• #20103
• #19483
• #20269

Powered by Codex Action

I stumbled upon this in the past too - I even lost threads in Codex because of this. Since then, if I invoke codex programmatically or via the SDK I always set a different CODEX_HOME env var. The painful thing is you need to make sure the authentication in this secondary CODEX_HOME is valid

I just use a clean-up script for now. Use it at your own risk :-) it will probably break in the future when the app changes - they move fast

#!/usr/bin/env zsh
set -euo pipefail

usage() {
  cat <<'EOF'
Usage:
  archive-codex-non-desktop.zsh [--dry-run]

Archives Codex rollout sessions older than OLDER_THAN_HOURS where the first
JSONL session_meta payload has one of these originators:
  - symphony-orchestrator
  - repoprompt
  - reportprompt
  - codex-tui
  - codex_sdk_ts

Environment:
  CODEX_HOME          Defaults to "$HOME/.codex"
  OLDER_THAN_HOURS    Defaults to 12
EOF
}

dry_run=0
case "${1:-}" in
  --dry-run)
    dry_run=1
    ;;
  -h|--help)
    usage
    exit 0
    ;;
  "")
    ;;
  *)
    echo "Unknown argument: $1" >&2
    usage >&2
    exit 2
    ;;
esac

for dep in jq sqlite3 find stat date head mv mkdir cp; do
  if ! command -v "$dep" >/dev/null 2>&1; then
    echo "Missing required command: $dep" >&2
    exit 1
  fi
done

CODEX_HOME="${CODEX_HOME:-$HOME/.codex}"
SESSIONS_DIR="$CODEX_HOME/sessions"
ARCHIVE_DIR="$CODEX_HOME/archived_sessions"
DB="$CODEX_HOME/state_5.sqlite"
OLDER_THAN_HOURS="${OLDER_THAN_HOURS:-12}"

if ! [[ "$OLDER_THAN_HOURS" == <-> ]]; then
  echo "OLDER_THAN_HOURS must be a positive integer, got: $OLDER_THAN_HOURS" >&2
  exit 1
fi

originators=(
  "symphony-orchestrator"
  "repoprompt"
  "reportprompt"
  "codex-tui"
  "codex_sdk_ts"
)

[[ -d "$SESSIONS_DIR" ]] || { echo "Missing sessions dir: $SESSIONS_DIR" >&2; exit 1; }
[[ -f "$DB" ]] || { echo "Missing state DB: $DB" >&2; exit 1; }

cutoff_epoch=$(( $(date +%s) - OLDER_THAN_HOURS * 3600 ))
originator_filter_json="$(printf '%s\n' "${originators[@]}" | jq -R . | jq -s .)"

if (( dry_run )); then
  echo "Dry run: no files or database rows will be changed."
else
  mkdir -p "$ARCHIVE_DIR"
  cp "$DB" "$DB.bak.$(date +%Y%m%d-%H%M%S)"
fi

matched=0
archived=0
skipped=0

find "$SESSIONS_DIR" -type f -name '*.jsonl' -print0 |
while IFS= read -r -d $'\0' src; do
  mtime="$(stat -f %m "$src")"
  (( mtime < cutoff_epoch )) || continue

  meta="$(
    head -n 1 "$src" | jq -cer --argjson allowed "$originator_filter_json" '
      select(.type == "session_meta")
      | .payload as $p
      | select($allowed | index($p.originator))
      | {
          id: $p.id,
          originator: $p.originator,
          timestamp: $p.timestamp,
          cwd: $p.cwd
        }
    ' 2>/dev/null
  )" || continue

  matched=$(( matched + 1 ))

  id="$(jq -r '.id' <<< "$meta")"
  originator="$(jq -r '.originator' <<< "$meta")"
  cwd="$(jq -r '.cwd // ""' <<< "$meta")"
  dest="$ARCHIVE_DIR/$(basename "$src")"

  if [[ -e "$dest" ]]; then
    echo "SKIP exists: originator=$originator id=$id dest=$dest"
    skipped=$(( skipped + 1 ))
    continue
  fi

  if (( dry_run )); then
    echo "WOULD archive originator=$originator id=$id src=$src cwd=$cwd"
    continue
  fi

  mv "$src" "$dest"

  id_sql="${id//\'/\'\'}"
  dest_sql="${dest//\'/\'\'}"

  sqlite3 "$DB" "
    UPDATE threads
    SET archived = 1,
        archived_at = strftime('%s','now'),
        rollout_path = '$dest_sql'
    WHERE id = '$id_sql';
  "

  archived=$(( archived + 1 ))
  echo "Archived originator=$originator id=$id"
done

if (( dry_run )); then
  echo "Dry-run complete."
else
  echo "Archive complete."
fi

Those of you running heavily automated coding agents which generate lots of sessions. Use a separate CODEX_HOME and copy your auth in there from your regular codex home. HT(🎩): @ebarti

But I totally agree this should be something better handled natively by Codex. Not sure why they would think having programmatic and interactive sessions in the same session storage was a good idea 😓

it is simple. They are overworked and have different priorities ... been there

All client instances (including the CLI and app) share a database (~/.codex/state_5.sqlite) that stores an indexed table of sessions. This eliminates the need for clients to scan all of ~/.codex/sessions at startup. If you are seeing large scans of the sessions directory, then something is very wrong. Perhaps the database has become corrupted, and Codex is attempting to rebuild it.

All client instances (including the CLI and app) share a database (~/.codex/state_5.sqlite) that stores an indexed table of sessions. This eliminates the need for clients to scan all of ~/.codex/sessions at startup. If you are seeing large scans of the sessions directory, then something is very wrong. Perhaps the database has become corrupted, and Codex is attempting to rebuild it.

Thanks for your response I am merely posting my observations (& best guess what went wrong really). I can consistently reproduce the issue by running a large number of sessions via my local symphony, watch Codex App degrade and become laggy, then prune all my Symphony sessions stored in ~/.codex/sessions which then resolves fully the unresponsiveness with the Codex app snapping back to working well aagain. Tried on my Mac Studio and MacBook - same problem. Just FYI and hope this helps!

How are you "pruning your sessions"? There is currently no app server API for deleting sessions; there is an API for archiving sessions. If you're deleting or otherwise manipulating the files under ~/.codex/sessions behind the back of Codex, that might explain why Codex is going in to "rebuild the database" mode when it starts.

How are you "pruning your sessions"? There is currently no app server API for deleting sessions; there is an API for archiving sessions. If you're deleting or otherwise manipulating the files under ~/.codex/sessions behind the back of Codex, that might explain why Codex is going in to "rebuild the database" mode when it starts.

I was using that script above I posted earlier . Also just getting rid of the ~/.codex/sessions folder solved the lag initially during the investigation. that is how I narrowed down which piece was causing a problem for me

@etraut-openai I see extremely degraded performance since the last update, but I have been running Codex agents in a separate CODEX_HOME directory via the Codex SDK for a while now.

Here, the same thing happens. After a few days of conversation, the specific chat becomes very slow, both to open and minimize while it's processing data. If you open a new chat in the project, it will always be fast, until you chat for a while, then the slowness returns. It's quite amateurish how Codex handles this. The problem is that when you start a new chat, it always starts out "dumb," and you have to keep summarizing the slow chat and pasting it into the new, clean chat.

Here, the same thing happens. After a few days of conversation, the specific chat becomes very slow, both to open and minimize while it's processing data. If you open a new chat in the project, it will always be fast, until you chat for a while, then the slowness returns. It's quite amateurish how Codex handles this. The problem is that when you start a new chat, it always starts out "dumb," and you have to keep summarizing the slow chat and pasting it into the new, clean chat.

Do not diss other people’s work! They are busy enough and now they have to listen to people like you who can’t conduct themselves in a respectful manner.

Additional repro on Windows — same index vs. full-directory-scan pattern
Environment: Codex Desktop 26.707.3748.0 (codex-cli 0.144.0-alpha.4), Windows 11 25H2, x64, 39.7GB RAM (~23GB free), disk queue 0 — resource exhaustion ruled out.
Symptom: severe UI lag on Windows too — noticeable delay on typing and scrolling, not just macOS.
I checked my local ~/.codex/sessions directory and found the same pattern described in this issue:

session_index.jsonl (the file Desktop-visible history actually reads from): 65KB
Full sessions directory: 326MB across 162 rollout JSONL files, largest single file 40MB

This matches OP's finding that the index is tiny while the actual scanned directory is much larger, which supports the theory that Desktop is full-scanning every historical rollout file under sessions rather than only reading session_index.jsonl.
Process-level data: 9 Codex-Desktop-related processes running, 3 of them holding ~820MB RAM with sustained 40%+ CPU. I have sandbox = "elevated" configured as well, though I haven't isolated whether that specifically compounds the lag or is incidental — flagging it in case it's useful, not asserting causation.
Possibly related (unconfirmed same root cause, flagging for triage rather than claiming they're the same bug):

Discussion #29949 — multiple Windows users reporting general Desktop lag since June; mechanism there looks more like renderer/GPU CPU spikes and isn't clearly tied to the sessions-directory scan specifically.
#11984 — long-session renderer memory growth up to 5GB+; this looks like single-session bloat rather than many-old-files full-scan, so may be a distinct mechanism with overlapping symptoms.

Expected behavior: Desktop should only load/reference the sessions listed in session_index.jsonl rather than scanning every historical rollout file under sessions.

The CLI hits this same scan through codex-core, and I can add the code-level mechanism plus some real-world numbers (traced on codex-cli 0.144.4, paths unchanged on current main).

Where the scan comes from

1. State-DB init gates startup on backfill: rollout/src/state_db.rs → try_init_with_roots_inner calls wait_for_backfill_gate, which runs backfill_sessions inline in a loop until BackfillStatus::Complete, up to STARTUP_BACKFILL_WAIT_TIMEOUT = 30s (1s poll). On timeout it returns an error, state-DB init fails for that launch, and the next launch resumes scanning from the watermark — so users see repeated ~30s startup stalls until the backfill eventually converges.
2. backfill_sessions (rollout/src/metadata.rs) walks ~/.codex/sessions and ~/.codex/archived_sessions, and for every rollout calls extract_metadata_from_rollout → RolloutRecorder::load_rollout_items, which reads and parses the entire rollout JSONL into memory just to derive listing metadata (title/preview/timestamps/usage).
3. The DB filename is version-pinned — state/src/lib.rs: pub const STATE_DB_FILENAME: &str = "state_5.sqlite" — so every schema bump starts a fresh DB with empty backfill_state and re-scans the whole tree from scratch. The scan is not a one-time cost; it comes back with upgrades.

Numbers from one real ~/.codex (macOS, CLI): 1,030 rollout files totalling 1.8 GB, largest single files 203 MB and 163 MB. A fresh backfill therefore reads ~2 GB from disk and materializes >200 MB single files in memory. The whole-file load_rollout_items materialization is probably also a contributor to the ingestion RSS blowups reported in #31040, and the 30s gate behavior matches #23787 / #28087.

Possible directions

• Don't gate interactive startup on backfill completion: claim the lease, run the backfill in the background, and serve session listings best-effort as rows land (the gate currently buys listing completeness at the cost of a hard startup stall).
• Extract listing metadata without materializing whole rollouts: the fields needed come from the head SessionMeta line plus a bounded tail (last timestamps/usage), so a capped head+tail read would replace the full-file parse; alternatively skip/degrade files above a size threshold.
• Carry backfill_state (or at least the completed watermark) forward across schema bumps so a version upgrade doesn't rescan every historical session.

Additional Windows A/B reproduction: GPT-5.6 Ultra subagent fan-out makes the full-session scan cost explode.

Environment

• Codex Desktop: 26.707.12708.0
• Windows 11 x64, build 26200
• 31.7 GiB RAM
• Default shared CODEX_HOME
• SQLite quick_check passed; this was not database corruption

Strong model/mode correlation

I analyzed the persisted thread graph in state_5.sqlite (root threads plus thread_spawn_edges) after several days of normal heavy use.

Across 289 root trees / 816 total thread nodes:

| Cohort | Root trees | Total nodes | Avg nodes/root | Max nodes | Multi-node roots |
|---|---:|---:|---:|---:|---:|
| >=80% gpt-5.6-* + ultra | 20 | 537 | 26.85 | 180 | 19/20 |
| All other model/effort combinations | 269 | 279 | 1.04 | 5 | 3/269 |

The largest trees were:

• 180 nodes: 180/180 gpt-5.6-sol + ultra
• 97 nodes: 97/97 gpt-5.6-sol + ultra
• 43 nodes: 43/43 gpt-5.6-sol + ultra
• 36 nodes: 36/36 gpt-5.6-sol + ultra
• 28 nodes: 28/28 gpt-5.6-sol + ultra
• 22 nodes: 22/22 gpt-5.6-sol + ultra

For the 180-node tree, thread_source was 1 user root + 179 subagent children. The 97-node tree was 1 root + 96 subagents, and the 43-node tree was 1 root + 42 subagents. Nested depth-2 children were also present.

The tokens_used values in the state DB are apparently cumulative and extremely large (the 180-node tree summed to about 127.4B), so token count may not be a directly comparable usage metric. The node counts, model/effort fields, spawn edges, and rollout files are unambiguous.

Older long gpt-5.5 + high sessions could have large token counters, but were typically single-node threads. The sharp tree fan-out appeared after using GPT-5.6 Ultra heavily.

A/B recovery

Before cleanup:

• 633 active rollout JSONL files
• about 1.84 GiB under the active sessions tree
• global Desktop lag, mouse/cursor stutter, frequent Not Responding states
• Windows Task Manager repeatedly showed the physical disk at 100%
• switching an unrelated conversation could freeze the entire app

Recovery procedure:

1. Archive affected roots with the official codex archive <root-thread-id> command (recursive archive).
2. Keep full backups.
3. Move archived rollout files outside CODEX_HOME, because archived files are also on the backfill scan path described in this issue.
4. Leave only current/recent small trees in the hot sessions directory.

After cleanup:

• 15 active rollout files
• 10.3 MiB active session footprint
• UI immediately responsive
• a 15-second cold-start sample showed physical disk average 3.26%, max 6.38%, max queue length 0.06
• main window remained Responding=True

Reducing local text/GPU caches did not explain the durable recovery. The decisive variable was removing the large multi-agent rollout forest from the hot scan path.

Interpretation

This looks like an algorithmic scalability bug in backfill/list/hydration, not a GPT-5.6 inference bug by itself. GPT-5.6 Ultra is a strong amplifier because it creates dozens or hundreds of child rollouts per user-visible root. If startup/listing cost is proportional to all rollout bytes and all child files, a few Ultra sessions can turn an otherwise normal profile into an O(total rollout history) startup and UI workload.

It also creates a bad product tradeoff: keeping history accessible makes Desktop unusable, while moving it outside CODEX_HOME restores performance but removes it from the UI.

Suggested invariants

• A user-visible root should be listable from bounded DB metadata without reading every descendant rollout.
• Child/subagent threads should be represented as a summarized tree count in the root list, not eagerly hydrated individually.
• Rollout metadata extraction should use the head plus a bounded tail/index, never whole-file materialization.
• Interactive startup should not block on full backfill.
• Archived rollout bodies should not be parsed on the normal startup/list path.
• Add per-root diagnostics: descendant count, total rollout bytes, and a warning/repair action for pathological trees.
• Provide cold archive/search/restore so history remains usable without remaining on the hot scan path.

This overlaps with #31198 (subagent rollout amplification), #32443 (many spawn edges/hydration), and #25779 (unbounded session-state meta issue), but the A/B above specifically confirms the sessions/backfill performance mechanism in this issue on a current Windows build.