[Bug] Paginated thread history projection stops at a token_count record

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• #40112
• #40178

Powered by Codex Action

Additional reproduction evidence from the same failure family:

• Codex Desktop build: 26.818.61809
• Codex CLI/app-server: 0.149.1
• Reproduces across Linux Desktop, laptop Remote, and mobile Remote.
• Affected long tasks show only the initial user message / early progress, or omit middle and later completed turns.
• The durable rollout JSONL still contains the omitted user messages, assistant messages, tool activity, and completed turns.
• Restarting the Desktop host and remote-control app-server does not restore the visible transcript.
• Archiving was not involved.

A local diagnostic mitigation forced unknown-conversation recovery to call full paginated hydration (hydrateBackgroundThreads([threadId], { includeTurns: true })) before replaying deferred live notifications. The patched bundle was installed, its marker verified, and the app-server restarted. Mobile still displayed the truncated transcript.

This narrows the remaining failure below a renderer-only unknown-conversation recovery path: likely the remote thread/read history projection/index or the data returned to Remote clients. In affected tasks, a read can report hasMore: false while returning only the first visible turn even though the durable rollout has many later completed turns.

This also overlaps the visible symptom in #39011 (completed messages disappearing after steered follow-ups), though the durable projection failure here persists across reconnect/restart and devices.

Confirmed local recovery and narrowed the invariant failure further.

For two affected tasks, thread_history_projection_state showed:

• next_rollout_ordinal = 18
• next_rollout_byte_offset exactly equaled the end byte offset of rollout ordinal 18
• the next durable line was ordinal 19

So the projector had consumed ordinal 18's bytes without advancing its ordinal checkpoint. Every subsequent append then failed with expected ordinal 18, got 19; 0 rejected rollout lines cannot cover that gap.

Transactionally advancing only next_rollout_ordinal by one, with the database backed up and rollout JSONL untouched, immediately restored an inactive task: its projection advanced from one visible turn/four items to the complete history.

The active task exposed a second restart-related invariant failure: duplicate ordinals were written around service restarts, initially token_count plus thread_settings_applied sharing ordinals 149 and 743. After backing up the rollout and removing only the duplicate token_count records, projection advanced to ordinal 1053 and restored 25 turns/339 items. A later restart produced another duplicate ordinal 1052. Advancing only the byte checkpoint past the already-accounted duplicate, while retaining next_rollout_ordinal = 1053, resumed projection normally; it advanced to ordinal 1087 with 26 turns/350 items.

This confirms two related correctness issues:

1. byte and ordinal checkpoints can diverge after a rejected or metadata-only rollout record;
2. resuming an active thread after app-server restart can reuse the previous ordinal.

Suggested regression coverage:

• checkpoint byte/ordinal consistency after an undecodable token_count line;
• successful projection of later valid records;
• ordinal monotonicity across active-thread restart/resume;
• safe handling of an already-accounted duplicate ordinal at the checkpoint boundary.

All repairs were performed against backed-up local state. No transcript content was deleted or reconstructed.

need fix

Independent sanitized confirmation from another Codex Desktop environment.

Environment

• Runtime version persisted in local thread state: 0.146.0-alpha.3.1
• Affected user-created threads use history_mode=paginated
• No explicit paginated-history override was found in user or workspace configuration

Observed behavior

For a completed affected thread:

• the Desktop/thread reader returns only one interrupted turn and reports no additional page;
• the canonical rollout JSONL parses successfully and contains later user messages, assistant final messages, and a final task_complete;
• the derived history store contains only one inProgress turn and four projected items;
• the projection byte checkpoint lands exactly after a token_count record with ordinal N, while next_rollout_ordinal remains N;
• the next durable rollout record is ordinal N+1, leaving later valid records unprojected;
• approximately 110 KB of valid rollout data remained beyond the stored projection checkpoint.

The recurring sanitized log signatures are:

invalid type: map, expected f64
thread history projection expected ordinal 17, got 18
failed to project durable rollout during shutdown

Restart and shutdown retries did not reconcile the projection.

Aggregate scope

A read-only scan of the 14 most recent user-created paginated threads in this environment found the same boundary in 14 of 14 cases:

• every projection checkpoint ended immediately after a token_count record;
• every stored projection contained a single inProgress turn;
• the next durable record was ordinal N+1;
• multiple canonical rollouts already ended with task_complete.

This confirms a repeatable projection/index failure rather than transcript deletion or renderer-only state.

A supported repair or automatic re-projection path for already-wedged threads would be valuable, in addition to preventing the byte and ordinal checkpoints from diverging.

Privacy note: this report intentionally excludes prompts, transcript text, screenshots, task or thread identifiers, project and filesystem names, host information, account details, credentials, raw logs, and database files.

I can confirm that I am experiencing the same issue in Codex Desktop
connected to a remote Ubuntu 24.04 host running codex-cli 0.145.0.

Additional evidence from read-only diagnostics on this host:

• 1,419 legacy-history threads
• 24 paginated-history threads
• 24/24 paginated-history threads have incomplete projections
• Raw history: 121 started turns, 110 completed turns
• Projected history: 24 turns, 0 completed turns
• 19,987 occurrences of invalid type: map, expected f64
• SQLite integrity checks pass
• The affected rollout files remain present and valid

This suggests that the issue affects paginated history materialization,
rather than being caused by transcript deletion or SQLite corruption.

Seeing this as well, pretty significant user experience issue

Independent reproduction on 2026-08-26 that closely matches this issue and extends it to ChatGPT iOS Remote:

• Originator: codex_chatgpt_ios_remote
• Host: Linux x86_64 / Ubuntu 24.04
• Runtime: codex-cli 0.145.0
• Subscription: ChatGPT Plus
• The iOS live view showed the complete assistant final response.
• After reopening the thread, iOS showed only the initial user message and the assistant's first commentary/progress update.
• The durable rollout contains a response_item assistant message with phase final_answer, immediately followed by task_complete.
• A fresh read_thread reconstruction reports that same turn as inProgress and stops after the commentary item; it omits the later final answer and completion.
• The first token_count record occurs at the cutoff boundary and contains structured rate_limits data (including primary, secondary, and credits objects), matching the projection-failure pattern described here.
• The symptom reproduced across a mobile reload/network change (LTE to Wi-Fi); restarting/reopening did not restore the projected history.

This is therefore not model-output loss: the completed response remains in the host rollout, while the reconstructed/mobile history is truncated before the token-count boundary.

The private thread ID, screenshots, and exact transcript timestamps are available through OpenAI Support if maintainers need them.

I can independently reproduce what appears to be the same paginated thread-history projection bug on two separate new threads.

2 independent new paginated threads affected

Thread 1:
next_rollout_ordinal = 15
next_rollout_byte_offset = 119292
record at byte offset = ordinal 16

Thread 2:
next_rollout_ordinal = 14
next_rollout_byte_offset = 125677
record at byte offset = ordinal 15

raw rollout = intact
state metadata = intact
SQLite integrity_check = ok
projection = permanently stalled
Codex CLI/app-server = 0.146.0

Both affected threads remain present in the raw rollout/state metadata, but disappear or become incomplete in the Codex App history UI after reload.

No database repair, reindex, cleanup, or manual modification has been performed.

I can provide additional timestamps or metadata if useful.

Additional Windows ChatGPT Work reproduction, cross-referenced as #40907:

• Surface: current ChatGPT/Codex Desktop app on Windows.
• A long-running Work thread had multiple later user/assistant turns beyond an older tool-activity checkpoint.
• After leaving and reopening the thread, the visible transcript repeatedly ends at the same older checkpoint; the later turns are no longer reachable by scrolling.
• The cutoff also retains a stale Thinking state, consistent with a completion/later-turn record not being projected.
• Reopening reproduces the identical cutoff rather than intermittently restoring the tail.

The user has not modified local rollout files or SQLite state. Exact runtime/build and private thread diagnostics are not yet available.

Based on the symptoms and the evidence already collected in this issue, the leading hypothesis is the same byte-offset / ordinal checkpoint divergence at a structured token_count record: the durable rollout may contain the later turns while the paginated projection remains permanently stalled.

Please correlate #40907 with this issue and check the affected Windows Work thread for history_mode=paginated, records after the visible cutoff, thread_history_projection_state consistency, and the known invalid type: map, expected f64 / ordinal-gap signatures. A privacy-scrubbed screenshot is available.

I can reproduce the same class of paginated-history failure on Windows Codex Desktop.

Environment:

• Windows 11 x64
• Codex Desktop: 26.820.7780.0
• Current standalone Codex CLI: 0.150.0-alpha.8
• Affected thread metadata reports codex-cli 0.149.0-alpha.4.1 and 0.149.0-alpha.4.3
• History mode: paginated

I have two affected local threads. I am omitting thread IDs, paths, and transcript contents for privacy.

Case A:

• The active, unarchived thread still exists in state_5.sqlite.
• Its rollout segment is newline-valid with 0 malformed JSONL records and continues through ordinal 5104.
• The thread-history projection contains only 589 items through ordinal 1658.
• The projection cursor is next_rollout_ordinal=1662.
• Later records include contact-form testing, email subject customization, sender display, and Reply-To behavior, but they are absent from the Desktop history view.

Case B:

• The rollout is newline-valid with 0 malformed JSONL records and continues through ordinal 7729.
• The thread-history projection contains only 194 items through ordinal 481.
• The projection cursor is next_rollout_ordinal=484.
• The visible freeze point is a completed tool-read item at ordinal 480.
• Ordinal 483 appears twice in the rollout, followed by many later turns and final progress messages about gameplay UI and visual-effect integration.

Both state_5.sqlite and thread_history_1.sqlite pass SQLite quick_check. The rollout files and authoritative thread rows are still present; this appears to be a stale or wedged thread-history projection rather than transcript deletion.

Expected behavior:

• Desktop should reconcile or rebuild the paginated projection from the durable rollout.
• Existing threads created by older runtimes should be repaired automatically after upgrading.
• Users should have a supported per-thread history repair command.

I have not modified the databases. Could you confirm whether these cases belong to the same projection bug tracked here, and whether there is a supported repair path for already-wedged projections?

I can confirm this issue on a remote Linux app-server used from Codex Desktop on macOS over SSH, with a larger blast radius and a successful recovery path.

Environment

• Client: Codex Desktop on macOS
• Remote host: Linux x86_64 over SSH
• Runtime that created/served the broken projections: codex-cli 0.145.0
• Upgraded runtime: codex-cli 0.150.1
• History mode: paginated

Symptoms

1. Affected threads reopened with only the first user message and the first assistant progress message.
2. The thread reader returned exactly one turn and reported no additional page.
3. Selecting text and opening a side chat also failed on these remote paginated threads with:

Forking is not available for threads using paginated history yet.

1. Canonical rollout JSONL files remained present, newline-valid, and contained all later turns and final answers.

Projection evidence

This was not isolated to one thread:

• 17/17 paginated-history threads on the remote host had incomplete projections.
• The canonical rollouts contained about 110 turns in aggregate.
• thread_history_1.sqlite contained only 17 turns total: exactly one projected turn per affected thread.
• Every projection cursor stopped very early (around rollout ordinal 14-18).
• One representative rollout was about 18 MiB / 1,199 valid JSONL records / 10 turns, while the projection contained only 1 turn and 3 items and stopped at roughly 0.9% of the file.
• Repeated server errors included:
• invalid type: map, expected f64
• followed by thread history projection expected ordinal N, got N+1

This matches the token-count/schema failure described in the issue: after rejecting the structured value, the cursor never advanced, and all subsequent records became unreachable through paginated history.

Version-skew detail

Upgrading the npm CLI alone did not replace the running server. Immediately after the upgrade:

{
  "status": "running",
  "cliVersion": "0.150.1",
  "appServerVersion": "0.145.0",
  "managedCodexVersion": null
}

The old 0.145.0 app-server was an unmanaged persistent process holding the control socket. codex app-server daemon restart correctly refused with:

app server is running but is not managed by codex app-server daemon

After terminating the old unmanaged instance and bootstrapping the managed daemon, both versions reported 0.150.1.

Recovery that restored all affected threads

1. Backed up the canonical rollout JSONLs and the SQLite projection.
2. Stopped the managed app-server daemon.
3. Moved thread_history_1.sqlite plus its WAL/SHM files aside (no rollout data was removed).
4. Started the managed 0.150.1 daemon.
5. Reopened each affected thread once. Projection rebuild is lazy: the new database did not exist immediately after daemon start, and each thread was rebuilt when first loaded.
6. All 17 affected threads recovered their complete visible histories. Side-chat creation also worked after the runtime upgrade.

Suggested product improvements

• Make the projection parser backward/forward compatible with structured token-count/rate-limit/duration values.
• Treat thread_history_*.sqlite explicitly as a rebuildable derived index and automatically retry/rebuild when the cursor is far behind a valid canonical rollout.
• On Desktop/SSH connection, detect cliVersion != appServerVersion and clearly warn or safely replace an unmanaged incompatible server.
• Provide a supported history doctor/rebuild command that scans all affected paginated threads, rather than requiring a manual database move and lazy reopening of every thread.
• Surface the real projection/fork error in the UI instead of a generic “failed to open side chat” message.

No private prompts, hostnames, repository names, absolute paths, credentials, or thread IDs are included here.

Additional Windows reproduction with a clearer interruption/power-off trigger (2026-08-28).

Environment

• Windows x64: Microsoft Windows NT 10.0.19045.0
• Current local Codex runtime reports codex-cli 0.150.0-alpha.8
• Desktop build number was not available from the read-only command-line diagnostic

Trigger pattern

This has now occurred in two separate long local chats:

1. A turn is running a local command/tool.
2. The user interrupts the running command/turn.
3. The computer is shut down before the chat visibly reaches a settled terminal state.
4. After reboot, Codex Desktop reopens the chat.
5. The conversation pane restores an old assistant progress message from much earlier in the task and presents that stale point as the visible/current task state.
6. Later completed turns remain missing from the pane after restarting the app.

In the latest occurrence, the stale message said that a proof-of-concept was only about to begin, while the repository's durable Git state already contained the later completed and validated implementation. This confirms that the project data was not rolled back; the Desktop history/task projection was stale.

Expected behavior

After an interrupted turn followed by an unclean host shutdown, startup recovery should:

• reconcile the in-flight turn to an explicit interrupted/failed terminal state;
• rebuild the visible history from the durable rollout;
• preserve all later completed turns;
• never surface an old intermediate progress message as the current source of truth.

This looks consistent with the projection/restart invariant failures already documented in this issue, but adds a repeatable user-level trigger: interrupt an active command, then shut down before the turn fully settles.

No session IDs, project names, paths, transcript text, screenshots, credentials, rollout files, or database contents are included. Local rollout/database state was not modified.