›_ Codex Issues
Codex Issues / #21211

Thread navigation/loading slows from unbounded metadata and eager large-history hydration

Open 💬 20 comments Opened May 5, 2026 by clairernovotny
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

Supersedes openai/codex#21154. The original root cause was threads.title becoming the full first user message, but the local diagnostics now point to a broader performance problem in thread navigation and thread loading:

  • unbounded thread metadata can bloat the SQLite thread-list/navigation path
  • a SQLite-only trim is temporary because reconciliation can repopulate titles from JSONL history
  • thread/list can still be huge when first_user_message is treated as a full prompt/history field instead of a bounded preview
  • opening a large thread can still be slow because the UI path eagerly hydrates/replays too much history

User impact

Codex Desktop becomes sluggish when switching chats and when opening/loading large threads. The visible UX impact is not just a database query delay: the app can appear stuck while the renderer parses, allocates, reconciles, and renders large thread payloads.

In local testing, trimming pathological titles helped chat switching a lot. Loading large threads was still slow afterward, which suggests there are at least two hot paths:

  1. list/navigation metadata payload size
  2. initial thread-open history hydration/rendering

Title bloat impact

I benchmarked an affected local SQLite DB backup against the same row set after shortening only pathological active titles. This isolates title length as the variable.

Active rows

<!-- linear:table-colwidths:133,133,133,133,133,133 -->
| DB | Active rows | Active title chars | Active first_user_message chars | Max title chars | Active titles > 120 |
| -- | -- | -- | -- | -- | -- |
| Bad backup | 134 | 14,610,549 | 14,614,033 | 675,773 | 94 |
| Same rows after title repair | 134 | 3,588 | 14,614,033 | 120 | 0 |
| Current repaired DB | 158 | 5,154 | 14,614,486 | 120 | 0 |

Thread-list style query

Query approximating the active thread navigation/list path that needs titles:

SELECT id, title, source, cwd, updated_at_ms
FROM threads
WHERE COALESCE(archived,0)=0
ORDER BY updated_at_ms DESC, id DESC
LIMIT 200;

Measured over 80 iterations:

<!-- linear:table-colwidths:133,133,133,133,133,133 -->
| DB | Rows | Result payload bytes | SQLite query median | SQLite query p95 | JSON encode median |
| -- | -- | -- | -- | -- | -- |
| Bad backup | 134 | 15,605,594 | 8.12 ms | 10.30 ms | 46.34 ms |
| Same rows after title repair | 134 | 25,074 | 0.15 ms | 0.19 ms | 0.14 ms |
| Current repaired DB | 158 | 34,343 | 0.17 ms | 0.19 ms | 0.17 ms |

Impact from title repair on the same rows:

  • Result payload: 15.6 MB -> 25 KB, about 622x smaller.
  • SQLite read median: 8.12 ms -> 0.15 ms, about 54x faster.
  • JSON encode median: 46.34 ms -> 0.14 ms, about 331x faster.

This is before Electron IPC, UI reconciliation, rendering, and any extra chat-switching work.

Full list item including preview

If the list path also includes first_user_message, the duplicated title roughly doubles the heavy payload:

SELECT id, title, first_user_message, source, cwd, updated_at_ms
FROM threads
WHERE COALESCE(archived,0)=0
ORDER BY updated_at_ms DESC, id DESC
LIMIT 200;

<!-- linear:table-colwidths:133,133,133,133,133,133 -->
| DB | Rows | Result payload bytes | SQLite query median | SQLite query p95 | JSON encode median |
| -- | -- | -- | -- | -- | -- |
| Bad backup | 134 | 31,196,414 | 14.70 ms | 17.75 ms | 90.73 ms |
| Same rows after title repair | 134 | 15,615,894 | 7.12 ms | 8.55 ms | 46.22 ms |
| Current repaired DB | 158 | 15,626,192 | 7.08 ms | 8.50 ms | 45.40 ms |

So the title bug doubled the heavy list item payload: one copy in first_user_message, another copy in title.

Reconciliation can repopulate bad titles

SQLite is not the only source of truth. Codex can rebuild or reconcile thread metadata from rollout JSONL files and session_index.jsonl.

The local failure mode appears to be:

  • the state extractor reads rollout events
  • the first EventMsg::UserMessage can populate both first_user_message and fallback title
  • later ThreadNameUpdated events can overwrite the title
  • reconciliation/upsert writes the resulting metadata back into SQLite

That means a DB-only trim can be undone later if the underlying JSONL history has no later good ThreadNameUpdated event. In practice, the durable local repair needed both:

  • trim threads.title in SQLite
  • append a later thread-name update to the affected JSONL histories so reconciliation keeps the bounded title

thread/list can still be too large after title repair

Bounding titles helps, but thread/list can still pay for full prompt-sized first_user_message values when those are mapped into Thread.preview.

One local active DB snapshot after repopulation showed:

active rows: 158
active title chars: 13,980,313
active first_user_message chars: 14,614,486
active titles >120 chars: 67
active first_user_message >120 chars: 105
active first_user_message >10k chars: 74
max first_user_message: 675,773 chars

This was not only RepoPrompt. Grouped by source:

vscode rows=86 first_user_message_chars=13,981,761 max=675,773 gt120=75 gt10k=54
exec   rows=22 first_user_message_chars=627,261    max=44,430  gt120=22 gt10k=20

Direct app-server probe using /Applications/Codex.app/Contents/Resources/codex app-server:

thread/list archived=false modelProviders=[] useStateDbOnly=true  limit=20   2.56s response=4,505,704 bytes
thread/list archived=false modelProviders=[] useStateDbOnly=true  limit=100  7.95s response=14,628,806 bytes
thread/list archived=false modelProviders=[] useStateDbOnly=false limit=100  8.26s response=14,627,363 bytes

This suggests first_user_message should either be bounded as a preview field for list/read summaries, or the full field should not be selected/sent in thread/list.

Opening large threads is a separate hot path

Direct app-server timings show that metadata-only reads are fast, but full turn reads can be very expensive:

48.7 MB rollout, image-heavy:
thread/read includeTurns=false  62.8 ms     response=850 bytes
thread/read includeTurns=true   11.65 s     response=20,654,619 bytes

45.9 MB rollout, compaction-heavy:
thread/read includeTurns=false  27.8 ms     response=875 bytes
thread/read includeTurns=true   3.39 s      response=6,363,354 bytes

1.4 MB rollout with giant first user message/title/preview:
thread/read includeTurns=false  415.9 ms    response=704,374 bytes
thread/read includeTurns=true   814.0 ms    response=1,414,448 bytes

The large rollouts are not all prompt-import cases. Large contributors included image generation events, compaction payloads, function/tool outputs, MCP tool results, and exec output.

Separate CDP profiling inside Codex Desktop showed the UI layer can still be the bottleneck even when app-server calls are quick:

tested thread switch: about 13.4s to settle in the UI
direct app-server calls for comparable tested thread data: tens of ms
renderer heap spike during switch: >200 MB transient
long tasks during switch: repeated ~1.8-2.0s tasks

So the remaining thread-open UX issue appears to be renderer hydration/reconciliation/allocation, not only DB or app-server I/O.

UX requirement

The fix should keep the current user experience or improve it:

  • switching to a thread should show usable content quickly
  • older turns should continue loading automatically
  • users should not have to click an extra "load older messages" control just to make opening a thread fast
  • full history should remain available for scrollback, search, copy, and context inspection
  • active assistant output and status should not be delayed behind background hydration

Suggested direction

I do not want to over-specify the implementation, but the fixes likely need to cover these areas:

  • enforce a single bounded display-title invariant across all title producers and SQLite write boundaries
  • make reconciliation preserve bounded titles instead of restoring the first-message fallback
  • keep thread/list payloads bounded by treating previews as previews, not full prompt/history fields
  • make initial thread open staged/paged: metadata plus newest visible turns first, older history loaded automatically afterward
  • avoid sending heavyweight replay payloads in the initial UI load when a placeholder/expand-on-demand representation would preserve the UX
  • add performance regression tests or fixtures for large titles, large previews, and large rollout histories

View original on GitHub ↗

20 Comments

github-actions[bot] contributor · 2 months ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #21154
  • #21119

Powered by Codex Action

clairernovotny · 2 months ago

Additional local measurements point to two separate hot paths beyond the original threads.title duplication.

1. thread/list is still huge when first_user_message is full prompt text

Current active DB state on the same machine:

active rows: 158
active title chars: 13,980,313
active first_user_message chars: 14,614,486
active titles >120 chars: 67
active first_user_message >120 chars: 105
active first_user_message >10k chars: 74
max first_user_message: 675,773 chars

This is not only RepoPrompt. Grouped by source, active first_user_message bytes are mostly vscode, but there are also exec rows with long previews:

vscode rows=86 first_user_message_chars=13,981,761 max=675,773 gt120=75 gt10k=54
exec   rows=22 first_user_message_chars=627,261    max=44,430  gt120=22 gt10k=20

Direct app-server probe using /Applications/Codex.app/Contents/Resources/codex app-server:

thread/list archived=false modelProviders=[] useStateDbOnly=true  limit=20   2.56s response=4,505,704 bytes
thread/list archived=false modelProviders=[] useStateDbOnly=true  limit=100  7.95s response=14,628,806 bytes
thread/list archived=false modelProviders=[] useStateDbOnly=false limit=100  8.26s response=14,627,363 bytes

So bounding title helps thread switching, but thread/list still pays for full preview text because the state DB select includes threads.first_user_message, and app-server maps it into Thread.preview.

Suggested fix: treat first_user_message as a bounded preview field in list/read summaries, or stop selecting/sending the full field in thread/list. Preserve full user content in rollout/history; do not duplicate it into the list summary payload.

2. Opening a thread can still be slow because thread/read includeTurns=true loads and serializes the full rollout history

Direct app-server timings:

48.7 MB rollout, image-heavy:
thread/read includeTurns=false  62.8 ms     response=850 bytes
thread/read includeTurns=true   11.65 s     response=20,654,619 bytes

45.9 MB rollout, compaction-heavy:
thread/read includeTurns=false  27.8 ms     response=875 bytes
thread/read includeTurns=true   3.39 s      response=6,363,354 bytes

1.4 MB rollout with giant first user message/title/preview:
thread/read includeTurns=false  415.9 ms    response=704,374 bytes
thread/read includeTurns=true   814.0 ms    response=1,414,448 bytes

The large rollouts are not all prompt-import cases. The largest active rollout contributors I saw include:

48.7 MB rollout: image_generation_end + image_generation_call ~40 MB combined
45.9 MB rollout: compacted entries ~32 MB, user/assistant messages ~10.9 MB
33.7 MB rollout: function_call_output ~14.3 MB, mcp_tool_call_end ~10.7 MB, compacted ~3.2 MB
12.5 MB rollout: function_call_output ~3.9 MB, image_generation event/call ~4.8 MB, exec_command_end ~1.6 MB

Suggested fixes:

  • Make thread open lazy/page turns by default rather than returning all turns on every thread/read includeTurns=true path.
  • Use thread/turns/list or a similar paginated endpoint for history hydration.
  • Avoid sending heavyweight replay payloads in initial UI load: image blobs, large compaction payloads, function/tool outputs, MCP tool results, and large exec output should be placeholder/expand-on-demand data.
  • Audit duplicated persistence/replay surfaces. Some data appears in both response items and event messages, e.g. image generation call/end and tool/function output/end pairs.

In short: #21154/#21211 title bounding fixes one major multiplier, but there is also a preview/list payload issue and a full-history initial-load issue.

clairernovotny · 2 months ago

I put the local workaround and perf notes here: https://github.com/clairernovotny/codex-perf

It currently includes:

  • CDP launcher wrappers for macOS and Windows
  • runtime title repair through Codex app actions (threads.list / threads.set_title)
  • click-time threads.read prefetch while native Codex thread navigation continues
  • README notes tying the title payload numbers in this issue to Electron renderer lag

The Windows launcher also looks for the MSIX install shape under C:\Program Files\WindowsApps\OpenAI.Codex_*\app.

baron · 2 months ago

Additional diagnostics from another affected local profile. I am intentionally omitting local paths, thread ids, cwd values, and prompt contents.

Environment/context:

  • macOS arm64
  • Codex Desktop 26.429.61741
  • SQLite state database was 999 MB before compaction
  • dbstat showed almost all space was in the threads table

Before title repair:

  • active thread rows: 6,073
  • active titles over 120 chars: 5,364
  • active rows where title == first_user_message: 6,026
  • total active title chars: 497,767,675
  • max active title length: 927,895 chars
  • title-list payload: 2,862,722 bytes
  • title-list SQLite median: 2.078 ms, p95: 5.148 ms
  • title-list JSON encode median: 17.012 ms, p95: 25.549 ms
  • full list item payload including first_user_message: 5,656,533 bytes
  • full list item JSON encode median: 31.474 ms, p95: 39.892 ms

After applying only title capping/repair to 120 chars:

  • active titles over 120 chars: 0
  • total active title chars: 462,934
  • max active title length: 120 chars
  • title-list payload: 97,799 bytes
  • title-list SQLite median: 0.268 ms, p95: 0.312 ms
  • title-list JSON encode median: 0.312 ms, p95: 0.340 ms
  • full list item payload including first_user_message: 2,891,610 bytes
  • full list item JSON encode median: 16.948 ms, p95: 19.274 ms
  • SQLite integrity check: ok

Post-repair storage shape:

  • sum(length(title)): 462,934
  • sum(length(first_user_message)): 497,772,414
  • max first_user_message: 927,895 chars
  • after checkpoint/VACUUM, state_5.sqlite was still about 506 MB, with threads about 525,783,040 bytes, now mostly due to retained first_user_message

Follow-up proof that the hot list path still carries the full first message as preview:

  • The packaged renderer code path for recent sidebar threads calls app-server method thread/list with limit, cursor, sortKey, archived: false, and local source kinds.
  • I then called the real codex app-server thread/list method against a copied state DB and measured only response field names and string lengths, not contents.
  • For thread/list with limit=20 after title repair, the response was 2,644,782 bytes.
  • Those 20 rows included a preview field totaling 2,445,255 chars.
  • Max single preview length in that response was 442,542 chars.
  • Correlating the same 20 rows back to SQLite by id, length(api.preview) == length(threads.first_user_message) for 20/20 rows.
  • The summed API preview length exactly matched the summed SQLite first_user_message length for those rows: 2,445,255 chars.

So title repair fixes the duplicated prompt-sized title field, but the thread/list/sidebar response still appears to transport unbounded first_user_message as preview. Even if the UI visually truncates it, the hot list path is still paying to read, serialize, transfer, parse, and retain those large previews.

Operational note: running the repair/compaction while a live Codex Desktop or CLI app-server is active is risky. In one attempt, the app-server respawned/held the DB during repair and the WAL temporarily grew to about 509 MB. The safe procedure was to run maintenance offline: fully quit/stop Codex app-server, run title repair, checkpoint/truncate WAL, then VACUUM.

This seems to support two separate fixes:

  1. Do not persist prompt-sized text into threads.title; generate/store bounded titles at write time and/or repair existing state.
  2. Do not return unbounded first_user_message as preview from thread/list / recent-thread sidebar APIs. Either omit it from the hot list path, return a bounded preview, or store a separate capped preview column. Even with titles fixed, large first messages still dominate the table and full-list serialization cost.
rdylina · 2 months ago

Adding another macOS repro. Codex 26.506.31421, one completed local thread with an 825MB rollout: opening/resuming caused thread/resume to take 252,892ms, then desktop dropped thread-stream-state-changed because payloadBytes=405234701 exceeded maxIpcFrameBytes=268435456. app-server RSS was ~4.5-5GB. This looks like eager full-thread hydration plus full-state IPC, not just thread-list/title bloat; the fix probably needs bounded resume/read plus paged turns/items and tail-only desktop opening.

rdylina · 2 months ago

Follow-up / PR offer: I read the contribution docs and understand external PRs are invitation-only. If the team thinks this direction fits, I’d be happy to take a stab at an invited PR for the open-source app-server side.

Conceptually, the fix I’d propose is:

  • in no case should opening a thread load the entire transcript into app-server + client state
  • open threads from metadata + the latest tail only, roughly the most recent 100-200 messages or a small byte cap, whichever comes first
  • lazy-load older chunks only when the user scrolls upward / explicitly asks for them
  • compact or summarize the old head for model context instead of treating the full durable transcript as active state
  • keep large tool outputs/artifacts behind references, not inline in normal thread state
  • add hard byte/count limits so thread/read, thread/resume, and stream-state snapshots cannot produce 100MB+ payloads
  • document that Desktop/IDE clients should use paged turns/items and never full-hydrate a completed thread just because it was clicked

The desktop side still needs closed-code changes, but the app-server can provide the safe bounded contract and fail gracefully when a client asks for full hydration of an oversized rollout.

SimpleZion · 2 months ago

Additional Windows evidence from a much larger local session store:

Environment:

  • Codex Desktop package: OpenAI.Codex_26.506.3741.0_x64
  • App/process metadata observed: 26.506.31421
  • Platform: Windows x64

Local session store summary:

%USERPROFILE%\.codex\sessions rollout files: 877
total session JSONL size: about 26.59 GB
files above 100 MB: 38
size of files above 100 MB: about 24.88 GB
recent 7 day session size: about 20.19 GB

The worst active thread in this profile:

title: Quant Research Platform
thread id: 019e0220-d159-72f3-a260-44918571cb12
rollout size: about 5.35 GB
rollout lines: about 121,307
state_5.sqlite tokens_used: 3,445,120,485
latest compacted checkpoints: ~25.6 MB each

Symptoms:

  • Opening this long thread is extremely slow.
  • After it opens, scrolling and conversation UI remain very laggy.
  • Sending or continuing the conversation in the opened long thread is also laggy.
  • This is not just disk-space pressure; the active codex.exe app-server process was observed around 1.5 GB private memory, and the renderer around 600 MB private memory while working with these large histories.

The local observation matches this issue's root direction: initial thread open appears to eagerly read/parse/hydrate/render far too much of a huge rollout-*.jsonl history. For a 5 GB JSONL, transparent filesystem compression or archiving can save disk space, but it does not solve the UI problem because the app still has to parse and render the giant logical history.

Expected behavior for this scale:

  • Opening a thread should load metadata plus the newest visible turns first.
  • Older turns should be paged or lazily hydrated in the background.
  • The UI should virtualize the transcript so rendering cost depends on visible turns, not total history size.
  • Full scrollback/search/copy should remain available, but not block the first usable render.
  • Very large compacted payloads and tool outputs should have placeholder or expandable representations rather than being fully materialized immediately.

Related storage issue:

  • #22593 reports that session forks duplicate parent history on disk. That storage amplification worsens this issue by creating more giant rollouts, but the open/render problem exists even for a single giant rollout.

I considered a local overlay workaround that points the thread at a slim active JSONL view and keeps the full rollout in an archive. It would make opening fast, but it would also hide old turns from the normal thread UI until restore, so it is not an acceptable substitute for an upstream lazy-loading/virtualized transcript fix.

zorkary · 2 months ago

Correction / clarification: the affected large threads do load on macOS Desktop for me. The client where I’m seeing the practical failure is mobile: the same very large threads appear to hang / never fully hydrate when opened there. Smaller threads load normally, so this still looks per-thread rather than a general account/app startup failure.

Environment / context:

  • Desktop source environment: Codex Desktop macOS release 26.513.20950
  • Bundled CLI: 0.131.0-alpha.9
  • Mobile symptom: large existing threads sit indefinitely during hydration/open; smaller threads open normally.

Read-only local metadata from the Desktop session store for the affected threads:

  • Active session store is ~3.3 GB.
  • 4 active rollout JSONL files are >100 MB, totaling ~3.0 GB.
  • Largest affected rollout is ~2.1 GB with ~280k JSONL records.
  • That largest file parses cleanly, but contains one ~85 MB function_call_output record and multiple ~40-49 MB compacted.replacement_history records.
  • Other affected threads are ~416 MB and ~374 MB, also with repeated multi-MB compacted.replacement_history records.

So the stronger claim is not “Desktop cannot load these.” It’s: Desktop has very large, valid histories; mobile appears unable to hydrate/render those large threads even though smaller threads work. That still seems consistent with whole-thread hydration / rendering pressure, and a metadata-first or paginated/cursor-based thread read path would likely help this class of failure.

Happy to provide more sanitized metadata if useful, but I’m avoiding sharing raw rollout contents because they contain private conversation data.

kangminlee-maker · 1 month ago

I opened a separate issue with additional local evidence that seems related to this thread-list/history path: #24510. It includes state_5.sqlite active metadata size, the observed query/index shape, logs_2.sqlite TRACE/log evidence, long-running goal-session rollout stats, and a temporary mitigation script gist.

ashjo42 · 1 month ago

Adding another Windows reproduction with current Codex Desktop data. This is the same failure family as the issue title describes: the app becomes sluggish at the UI level, including typing in the composer and general navigation, while the underlying conversations are static local history.

Environment:

  • Codex Desktop Windows package: OpenAI.Codex
  • Version: 26.608.1337.0
  • Platform: Windows x64
  • Date observed: 2026-06-12 local time

Observed user impact:

  • Typing in the composer becomes visibly delayed.
  • The desktop UI feels generally sluggish.
  • The user expectation is that static chat history should not make the active app slow, especially compared with terminal-first tools that do not continuously render a large thread list.

Read-only local diagnostics from the affected profile:

active threads: 87
thread title chars: 1,952,984
thread first_user_message chars: 1,955,396
max title chars: 57,533
max first_user_message chars: 57,533
titles over configured limit: 55
first_user_message over configured limit: 55
first_user_message over 10k chars: 54
metadata repair candidates: 55
session history size: 0.696 GB
archived session size: 1.573 GB
logs size: 650 MB

Process snapshot while the UI was lagging:

largest Codex Desktop process RSS: ~1.14 GB
another Codex process RSS: ~596 MB
codex app-server / CLI helper process RSS: ~160 MB
top node helper process RSS: ~895 MB

Disk space was not the immediate bottleneck:

C: total 476.28 GB
C: free 89.92 GB
free percent 18.9%

Important UX point:

The practical workaround should not be "archive old threads every time the app gets slow". Archiving can be a temporary escape hatch, but the product should keep thread-list and thread-open paths bounded even when many active conversations have large raw histories. In this reproduction, the direct suspect is not total disk pressure or one running model call; it is unbounded local display metadata (title / first_user_message) being large enough to affect the renderer/list/navigation path.

Expected behavior:

  • title and preview-like fields should be capped before persistence and before renderer/list transport.
  • thread/list should not send full prompt/history-sized first_user_message values as preview payloads.
  • Desktop should remain responsive while static local histories exist.
  • If local metadata is already bloated, Codex should offer a safe built-in repair/migration path instead of requiring unsupported manual SQLite edits or mass archiving.

I am intentionally not attaching raw DB rows or local thread IDs because they include private chat/project contents.

ashjo42 · 1 month ago

Severity escalation from the same Windows reproduction I added above.

After inspecting the local state more closely, this is not a case of the user simply having too many projects or too many manually maintained chats. The oversized active rows are overwhelmingly internal guardian/subagent rows:

active threads: 88
normal user threads: 12
guardian/subagent threads: 76
oversized active rows: 56
rows over 10k chars: 55
all oversized rows came from thread_source = subagent / source = guardian

The affected fields are also duplicated across display metadata:

title == first_user_message == preview
max per field: 57,533 chars

So the local profile is not just large history. Internal guardian/subagent threads are entering the active thread metadata surface, and full history-like text is being persisted into title/preview-like fields.

This makes the workaround especially unreasonable for users:

  • Users cannot be expected to know that internal guardian threads are accumulating.
  • Users should not have to archive chats repeatedly to keep the desktop app responsive.
  • Users should not have to run unsupported SQLite repair scripts to make typing in the composer usable.
  • Heavy local Codex usage should remain bounded by product invariants, not manual cleanup.

User-impact note: this is severe enough that the affected user explicitly said they will switch paid usage to Claude next month if Codex Desktop cannot handle this class of issue. I am including that because this is not a small polish bug; it directly affects whether the desktop app is usable for long-running Codex workflows.

Requested product fix remains:

  1. Hide or separately manage internal guardian/subagent rows so they do not bloat normal active thread navigation.
  2. Enforce hard caps on title, first_user_message, and preview at every write/reconciliation boundary.
  3. Ship an official local-state migration/repair path for existing affected profiles.
  4. Keep thread/list and renderer-facing thread metadata bounded regardless of raw transcript size.
najibninaba · 1 month ago

I hit a sharper version of this locally where thread metadata copied a very large first user message into SQLite and made Codex.app crash-prone.

The part I focused on is the metadata path, not the larger history hydration problem. The full transcript is already in the rollout, but title, preview, and first_user_message can still end up carrying prompt-sized text into state_5.sqlite. Once that happens, thread listing/discovery has to move a lot of text around before the user even opens the thread.

I put together a small local patch that caps derived thread metadata text at 4096 UTF-8-safe characters while leaving the rollout transcript and model history untouched:

  • rollout extraction caps derived title, preview, and first_user_message
  • live/resume metadata sync applies the same cap before building ThreadMetadataPatch
  • the local SQLite metadata write path also caps direct metadata patch text, including explicit thread names, so the same invariant cannot be bypassed by another metadata path
  • tests cover large user messages, UTF-8-safe truncation, goal objective previews, resume-history metadata sync, and direct metadata patch writes

Validation passed locally:

  • just fmt
  • just test -p codex-state
  • just test -p codex-thread-store
  • just fix -p codex-state
  • just fix -p codex-thread-store

Branch is here in case it is useful for review or for an internal patch:
https://github.com/najibninaba/codex/tree/najibninaba/thread-metadata-cap

If this is aligned with the intended fix direction, I am happy to open a PR or adjust the patch to match the shape maintainers want.

Necmttn · 29 days ago

Thread list metadata and replay payloads should be separate storage/read paths.

The list view needs bounded title/preview, timestamps, cwd/project key, model/provider, size, and source path. Full message bodies and large tool outputs can stay behind lazy hydration offsets until a thread opens. Reconciliation should preserve those preview bounds instead of repopulating unbounded first prompts.

---

_Generated with ax._

taoliuyohoho-rgb · 14 days ago

I hit a narrower version of the eager-history path described here: repeatedly opening an older local thread can spend time replaying/parsing the same rollout JSONL again within the same app-server process.

In the current local store path, thread/read with includeTurns=true eventually calls thread_store.read_thread(... include_history: true), and codex-rs/thread-store/src/local/read_thread.rs calls RolloutRecorder::load_rollout_items(path) each time history is attached. thread/turns/list also still reconstructs turns from loaded history. For larger local rollouts this means switching back to the same old thread can feel like it is loading from scratch each time.

I put together a small patch branch here:

https://github.com/taoliuyohoho-rgb/codex/tree/codex/cache-local-rollout-history

What it does:

  • adds a bounded process-local cache for parsed local rollout histories in LocalThreadStore
  • keys cache entries by canonical physical rollout path plus file size/mtime fingerprint
  • reloads when the rollout file changes
  • avoids inserting into the cache if the file changes while it is being read
  • caps the cache at 8 entries / 128 MiB of rollout-file size accounting

This is intentionally not a full fix for title/preview bloat or pathological multi-GB rollouts. It is just a small mitigation for repeated old-thread opens / repeated thread/turns/list calls against unchanged local rollout files in the same app-server process. A durable SQLite turn/item index or true paginated history backend would still be the more complete direction.

Local validation on the branch:

  • cargo test -p codex-thread-store load_history_items_ --lib
  • cargo test -p codex-thread-store
  • cargo clippy -p codex-thread-store --tests -- -D warnings
  • cargo test -p codex-app-server --test all suite::v2::thread_read::thread_read_can_include_turns -- --exact
  • cargo test -p codex-app-server --test all suite::v2::thread_read::thread_turns_list_can_page_backward_and_forward -- --exact

I tried opening this as a PR, but the repository currently limits PR creation to collaborators, so I am leaving the branch here in case it is useful for an internal patch or as a reference implementation.

laz-jamesfuller · 10 days ago

Adding a Windows data point that strongly links Guardian auto-review rows to the unbounded thread metadata described here.

Environment:

  • Windows Codex Desktop / app-server 0.144.0-alpha.4
  • Measurements were read-only against state_5.sqlite; no rows or rollout files were changed.

Active thread inventory:

  • 3,095 active threads and 192 archived threads
  • 1,712 active rows with source containing guardian
  • 0 archived Guardian rows
  • Guardian rows therefore account for about 55% of the active thread count

Guardian metadata:

  • 1,616 of 1,712 Guardian titles exceed 10,000 characters
  • average Guardian title: 34,586 characters
  • maximum Guardian title: 86,572 characters
  • Guardian title characters: 59,211,801
  • the same 59,211,801 characters are present independently in title, preview, and first_user_message
  • total duplicated text across those three fields: 177,635,403 characters
  • Guardian rows account for about 98.8% of all active title characters in this profile

Read-only timing:

  • counting threads grouped by archive state: 0.4 ms
  • fetching the 100 most recent visible rows with title metadata: about 290 ms
  • materializing all 3,095 active rows with SELECT *: about 2.93 s before Electron IPC or rendering
  • returned string data was about 182 million characters, approximately 348 MiB if represented as UTF-16 strings

During this diagnostic session, the Guardian row count increased from 1,710 to 1,712 across two additional escalated approval-review batches. That is temporal correlation rather than a controlled causality test, but it matches the rows' source.subagent.other = guardian classification.

This also appears closely related to #32035, which reports internal Guardian subagent threads exposed with full prompt-like titles, and #25570, which confirms that AutoReview is currently serialized as guardian_subagent.

This profile suggests a particularly concentrated form of the issue:

  1. an approval review creates a Guardian subagent thread;
  2. the complete review request/transcript becomes first_user_message;
  3. that text is copied into title and preview;
  4. completed review threads remain active indefinitely;
  5. normal thread-list/startup processing repeatedly pays for the accumulated payload.

Potential product-side mitigations:

  • keep internal Guardian review threads out of normal active conversation listings;
  • automatically archive or otherwise retire completed Guardian reviews while retaining any required audit record;
  • enforce bounded display titles for every thread source;
  • make list previews bounded projections rather than complete first messages;
  • avoid sending full review transcripts through thread-list metadata.

No prompt contents, local paths, task IDs, or account-specific data are included here.

Space-Boy166 · 7 days ago

Current Windows build still runs unbudgeted tail_history; closing one affected window is a decisive A/B

Environment:

  • Codex Desktop package: 26.707.8479.0
  • bundled app-server: 0.144.2
  • Windows 11 x64

Read-only inspection of the current production bundle still shows loadRemainingConversationTurns repeatedly calling thread/turns/list from a tight continuation loop. Requests are tagged:

priority: "background"
source: "tail_history"

but the loop has no delay, exponential backoff, page-count cap, time budget, byte budget, or resource-pressure gate between successful pages. “Background” is only a label here; it does not prevent the work from monopolizing the local backend.

Measured on one idle completed thread:

  • rollout JSONL: 1.258 GB
  • child/subthread rows: 181
  • thread/turns/list: one request roughly every 9.2–10.5 seconds
  • app-server: 98.1% of one CPU core
  • Git starts: 0

A/B control:

  • while that restored window remained open, the polling/replay sequence continued
  • closing only that one window stopped the thread/turns/list loop
  • app-server fell to roughly 1.0% CPU
  • app-server working set settled near 489 MiB

No repository, MCP, model workload, or system setting changed in that A/B. The affected thread was idle. This isolates the cost to automatic history completion/replay, not active agent work.

This behavior severely punishes exactly the long-running, tool-heavy workflows Codex is designed to support. Users should not have to close or abandon valuable threads to keep the Desktop responsive.

Required fix boundaries:

  1. Initial open must be metadata + recent visible tail, not full background hydration.
  2. Stop/cancel history loading when the thread/window becomes inactive or is superseded.
  3. Enforce real CPU/I/O/time/byte budgets for background work.
  4. Cache or index turns so every page does not replay the entire rollout.
  5. Keep large tool payloads behind references/placeholders until explicitly expanded.
  6. Add regression fixtures at hundreds of MiB and multi-GiB sizes.

The latest Windows build still ships the same unsafe loader shape; this should not be considered fixed.

Space-Boy166 · 6 days ago

Follow-up: preserving long tasks without paying the full-history cost on every reopen

A long Codex task is valuable because it holds decisions, failed attempts, images, and project context. On affected Windows Desktop builds, that same history can become a liability: reopening a large legacy task can rebuild a very large JSONL file and request old history before the user asks for it, producing blank or slow task views, a busy app-server core, desktop stalls, repeated resume cost, extra memory pressure, and crashes.

Starting a new task reduces that pressure by giving up convenient continuity. I published an unofficial reversible bridge that tests a different tradeoff: preserve every message, open recent turns first, and load exact older pages only when the user scrolls upward.

For one explicitly selected task, CLM SHA-256 archives the complete original, builds a transactional SQLite Turn index with the user-installed backend, keeps the exact native checkpoint plus recent suffix at the canonical rollout path, and serves older pages through the existing history API. Restore reconstructs the full original layout and preserves records appended after activation.

Current proof:

  • 236,869,210-byte real rollout -> 6,490,905-byte active tail;
  • the same native task reached its exact oldest message and materialized an old image;
  • 27 older-page requests completed in 5-11 ms with zero page errors;
  • a fresh renderer reopened from the bounded newest page;
  • an offline prepare/apply/restore transaction against the staged 26.707.9981.0 backend restored the exact source hash.

This is still an unofficial, Windows x64, version-sensitive alpha that requires an existing native replacement-history checkpoint. It addresses the long-history resume path only; it is not a universal fix for Electron, Git Review, MCP, GPU, or scheduling problems, and it is not a substitute for upstream bounded resume plus frontend row and attachment virtualization.

thunderfight127-svg · 6 days ago

Adding a Windows Desktop datapoint that appears narrower than the large-history cases already documented here: the thread-list backend remains fast, but the left conversation sidebar renderer becomes stuck after repeated scrolling.

Environment

  • Windows 11 Home, build 26200, x64
  • Microsoft Store package: OpenAI.Codex_26.707.9981.0_x64__2p2nqsd0c76g0
  • Desktop client annotation observed in logs: 26.707.72221
  • Local thread inventory: 215 threads

Reproduction

  1. Open ChatGPT Desktop in Codex mode.
  2. Scroll up and down through the left conversation sidebar several times.
  3. After a short period, the sidebar stops responding to scrolling or selection.
  4. Open Settings, then return to the main app.
  5. The sidebar works again temporarily.
  6. Scroll again; the sidebar eventually enters the same stuck state.

The Settings round-trip is highly repeatable as a temporary recovery. It looks like rebuilding/remounting the main route resets the failed sidebar state.

Read-only local evidence

Across retained Desktop logs:

  • approximately 11,950 successful thread/list responses were observed
  • median thread/list response time: about 1 ms
  • p95: about 14 ms
  • no thread/list error responses were observed
  • approximately 3,007 occurrences of:
  • ResizeObserver loop completed with undelivered notifications.

The local state DB contains only 215 threads. This does not look like a slow SQLite query or a very large thread-count problem on this machine.

Additional isolation:

  • fully quitting and restarting the app only helps temporarily
  • regenerating the UI, code, GPU, Dawn, and shader caches did not help
  • testing multiple Windows display-scaling settings did not help
  • the main renderer was not continuously CPU-bound while sampled
  • navigating to Settings and back consistently restores sidebar interaction for a while

Possible boundary

This looks consistent with a renderer-side sidebar virtualization / layout-observer lifecycle failure rather than the thread/list backend path. It may be adjacent to the renderer hydration/reconciliation work tracked here, but it is reproducible even when list RPC latency and local thread count are both low.

This comment is intentionally sanitized; no raw logs, account identifiers, local paths, thread IDs, conversation titles, prompts, or response content are included.

creynir · 5 days ago

macOS datapoint: Guardian metadata growth can prevent Desktop from opening at all

This appears to be the same unbounded-metadata failure, but with a complete startup failure rather than only slow navigation.

Environment:

  • Codex Desktop 26.707.72221 (build 5307)
  • bundled app-server 0.144.2
  • macOS 26.2 (25C56), arm64
  • reproduced twice on the same profile: 2026-07-11 and 2026-07-15

Observed startup behavior:

  • the native process starts and completes the app-server handshake
  • no renderer process/window is created
  • the main process can consume approximately one full CPU core
  • the application remains invisible/unusable

Read-only inspection of ~/.codex/state_5.sqlite immediately before the second repair:

  • 7,538 thread rows
  • maximum title / preview / first_user_message length: 81,034 characters
  • 544 titles over 200 characters
  • 491 previews over 1,000 characters
  • 352 first-user-message values over 2,000 characters
  • 515 active oversized rows were internal subagent rows
  • the pathological rows used source.subagent.other = guardian

The prior occurrence had fields up to roughly 93,757 characters. A broad metadata truncation restored startup then, but also damaged some visible conversation names, demonstrating that manual cleanup can create user-facing metadata loss.

Controlled A/B:

  1. Backed up the database.
  2. Tested against a cloned CODEX_HOME.
  3. Archived only active oversized rows explicitly identified as subagent/Guardian rows; no thread text or rollout files were deleted or rewritten.
  4. Codex immediately created two renderer processes and a visible, responsive window.
  5. Applied the same scoped transaction to the live profile; startup recovered, SQLite quick_check remained ok, and normal conversation titles were left intact.

Recurrence evidence:

  • while validating the recovered current build, three additional oversized Guardian rows appeared within minutes
  • automatically archiving only those new oversized Guardian rows kept startup and thread listing healthy
  • this indicates the current build is still producing the pathological records; a one-time SQLite repair only treats the accumulated state

The failure chain on this profile appears to be:

  1. Guardian review work creates internal subagent threads.
  2. Full prompt/review content is duplicated into title, preview, and first_user_message.
  3. Completed internal rows remain active and accumulate.
  4. Desktop startup/list hydration processes the accumulated payload before a usable renderer window appears.
  5. Eventually the app cannot visibly open.

Potential fix boundaries:

  • never expose completed Guardian threads as ordinary active conversations
  • retire/archive completed internal Guardian rows while preserving their audit/session data
  • enforce bounded title and preview invariants at every producer/write/reconciliation boundary
  • do not send prompt-sized first_user_message values through the thread-list startup path
  • add a startup regression fixture with hundreds or thousands of Guardian rows and tens of thousands of characters per metadata field

No prompt contents, thread IDs, account identifiers, or repository paths are included here.

abcjhby2871 · 1 day ago

Adding another observed symptom that may be related to the renderer hydration/repaint path described here.

On Codex Desktop, conversation rendering is occasionally very slow, and the currently open conversation can stop updating automatically. The model/backend appears to continue making progress, but the newest message or completed output is not reflected in the visible thread until some manual UI action triggers a refresh—for example, switching away and back to the thread, reactivating the window, or reopening the conversation.

This is intermittent rather than a deterministic per-thread delay, but it is especially disruptive in long-running conversations because it is unclear whether the turn is still running, stalled, or already complete but hidden by stale renderer state.

Expected behavior:

  • streamed and completed turn updates should repaint without requiring user interaction;
  • the newest visible messages and active-turn status should not wait behind older-history hydration;
  • if loading/rendering is delayed, the UI should expose a clear loading or reconnecting state instead of silently remaining stale;
  • renderer recovery/reconciliation should automatically catch up to the authoritative thread state.

This may overlap with the staged hydration work proposed here, but the manual-reactivation requirement suggests there may also be an invalidation/subscription or renderer-state reconciliation issue, rather than only raw loading performance.