Bug: Rollout persistence errors silently discarded during session resume/fork — silent data loss on restart
What version of Codex CLI is running?
Reproduced with codex-cli 0.145.0. The same implementation is still present in main at e4fb5311d7468839def62eabda4b268f4a54cf11.
What platform is your computer?
Darwin 26.5.2 arm64 arm (macOS).
What issue are you seeing?
Session rollout persistence errors are silently discarded in three critical paths, causing in-memory state to diverge from on-disk state. If a crash or unclean shutdown occurs after the divergence, the session resumes from a stale or incomplete rollout, losing conversation history.
Three affected call sites:
1. flush_rollout() during session resume (core/src/session/mod.rs:1350)
// After InitialHistory::Resumed path
if !is_subagent {
let _ = self.flush_rollout().await; // io::Result discarded
}
2. flush_rollout() during session fork (core/src/session/mod.rs:1397)
// After InitialHistory::Forked path
if !is_subagent {
let _ = self.flush_rollout().await; // io::Result discarded
}
3. persist_rollout_items() throughout session lifecycle (core/src/session/mod.rs:3610-3616)
pub(crate) async fn persist_rollout_items(&self, items: &[RolloutItem]) {
if let Some(live_thread) = self.live_thread()
&& let Err(e) = live_thread.append_items(items).await
{
error!("failed to record rollout items: {e:#}"); // logged, never propagated
}
}
persist_rollout_items is called from 6+ critical paths:
send_event_raw_with_persistence(line 2076) — event messagespersist_rollout_response_items(line 3242) — response items- Compaction flow (line 3227) — history replacement
- Session fork seeding (line 1390)
Root cause: flush_rollout() returns std::io::Result<()> and is the durability barrier after reconstructing session history. The let _ = pattern discards any I/O error (disk full, permissions error, network storage timeout). persist_rollout_items calls live_thread.append_items() which also returns a Result, but the error is only logged at error! level and never propagated to callers.
Impact: After any of these failures:
- In-memory history contains the full session state
- On-disk rollout is missing the failed items
- Session appears to operate normally (no error shown to user)
- On crash/restart, session resumes from the incomplete on-disk rollout
- Conversation history is silently truncated — recent messages, tool calls, and compaction summaries are lost
Severity: Medium-High. The failure requires an I/O error during persistence (disk full, NFS timeout, permissions issue), but the consequence is silent data loss that the user cannot detect or recover from.
What steps can reproduce the bug?
- Fill the disk to near capacity (e.g.,
dd if=/dev/zero of=filler bs=1M count=900) - Start a Codex CLI session and have a conversation
- Resume or fork the session (the
flush_rolloutpath) - The
flush_rolloutcall fails withio::ErrorKind::Otherorio::ErrorKind::StorageFull - The error is discarded — session appears normal
- Kill the process (Ctrl+C or crash)
- Resume the session — history is truncated to the point before the failed flush
What is the expected behavior?
Rollout persistence errors should be surfaced to the user (at minimum a warning notification) and the session should indicate that durability was not achieved. In the case of flush_rollout, the error should be propagated so the caller can decide whether to retry or warn the user.
Suggested fix
Option A (minimal): Log at error! level and notify the user via AppEvent::Warning:
// mod.rs:1350 and 1397
if let Err(e) = self.flush_rollout().await {
error!("failed to flush rollout during resume: {e:#}");
self.send_event_raw(AppEvent::Warning(format!(
"Session history may not be fully saved: {e}"
)));
}
Option B (comprehensive): Change persist_rollout_items to return Result and propagate to callers:
pub(crate) async fn persist_rollout_items(&self, items: &[RolloutItem]) -> anyhow::Result<()> {
if let Some(live_thread) = self.live_thread() {
live_thread.append_items(items).await?;
}
Ok(())
}
Related
- #31074 — stale session_index entries resolving to missing rollout files (same persistence layer)
- #34282 — rollout trace reducer panics on non-ASCII truncation (same persistence layer)
- #34935 — orphan threads that disappear after restart (potential cascade from persistence failures)
Scope
Three call sites in one file (core/src/session/mod.rs). The fix is 5-15 lines depending on option chosen. No behavioral change to正常 operation — only error handling for the failure path.
1 Comment
The silent persistence-loss path described here is exactly what the snapshot and diagnosis check is meant to document.
Thanks for the detailed report. This is a good candidate for a bounded recovery check. Vetto 0.2.0-alpha.2 is available from npm and adds a read-only, copy-only Codex rescue adapter. It does not resume Codex, edit rollout files, or write vendor SQLite. On a disposable copy, try:
npm install --global @shleddy/vetto@next;vetto rescue --adapter codex --root <CODEX_HOME> --json scan; then use the exact returned key withdiagnoseandsnapshot ... --output ./vetto-recovery/session.jsonl. Please report OS, Codex/Vetto versions, sanitized JSON, and source SHA-256 before/after; never upload raw transcripts, auth/config, prompts, or tokens. An explicit unavailable/unsupported result is useful too.