[CLI] Active rollout writer keeps writing to an unlinked inode after the session path is replaced

Open 💬 3 comments Opened Aug 12, 2026 by ckarabulut
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

What version of Codex CLI is running?

Observed with codex-cli 0.143 and onwards

What subscription do you have?

N/a

Which model were you using?

Model-independent; this occurs in the local rollout persistence layer.

What platform is your computer?

Linux 6.16

What terminal emulator and version are you using?

Not terminal-specific.

Codex doctor report

N/A

What issue are you seeing?

Codex keeps the file descriptor for an active session rollout JSONL open across turns. If a synchronization, backup, or restore tool replaces the rollout pathname using normal Unix unlink-and-copy or rename-over semantics, the pathname starts referring to a new inode while Codex keeps writing to the old, unlinked inode.

Those later writes and flushes succeed, so Codex emits no error. The visible rollout path remains stale, and when the Codex process exits the unlinked inode—and every turn appended to it after replacement—disappears. codex resume <thread-id> then reloads the stale file and appears to go backward in time.

This is not specific to one synchronization product; any process that replaces $CODEX_HOME/sessions/...jsonl while a thread is live can trigger it.

What steps can reproduce the bug?

Use a disposable thread because this intentionally demonstrates loss of its post-replacement tail.

  1. On Linux, start a new interactive Codex thread and send FIRST_SENTINEL. Record the thread UUID from /status or its rollout filename.
  2. In a second terminal, replace that rollout path with a byte-identical file on a different inode:
THREAD_ID='<disposable-thread-uuid>'
CODEX_HOME_DIR="${CODEX_HOME:-$HOME/.codex}"
ROLLOUT="$(find "$CODEX_HOME_DIR/sessions" -type f -name "*${THREAD_ID}.jsonl" -print -quit)"
test -n "$ROLLOUT"
OLD_ID="$(stat -c '%d:%i' "$ROLLOUT")"
cp --preserve=all "$ROLLOUT" "$ROLLOUT.replacement"
mv -f "$ROLLOUT.replacement" "$ROLLOUT"
NEW_ID="$(stat -c '%d:%i' "$ROLLOUT")"
printf '%s -> %s\n' "$OLD_ID" "$NEW_ID"
test "$OLD_ID" != "$NEW_ID"
  1. In the still-running Codex thread, send SECOND_SENTINEL_AFTER_REPLACEMENT and wait for the response.
  2. Inspect the visible rollout path:
grep -F 'SECOND_SENTINEL_AFTER_REPLACEMENT' "$ROLLOUT"

The command finds no match even though the turn completed successfully. While Codex is still running, lsof +L1 can also show its writable descriptor for the deleted rollout inode.

  1. Exit Codex and run codex resume "$THREAD_ID". The resumed transcript contains the first turn but not the second.

What is the expected behavior?

Codex should not report a successful durable write to a rollout file that is no longer reachable through the configured pathname. After the second prompt, the canonical rollout path and a resumed thread should contain that turn.

If Codex cannot safely reconcile a replacement with the live branch, it should surface the conflict and preserve the local tail at a recoverable path rather than silently losing it. Reopening the current pathname—or comparing the open descriptor's (st_dev, st_ino) with the pathname and then safely reopening/forking—before each non-empty durable batch would detect this case.

Additional information

The current writer lifecycle in codex-rs/rollout/src/recorder.rs appears to explain the behavior:

  • RolloutWriterState caches Option<JsonlWriter>.
  • ensure_writer_open returns immediately when that option is already populated.
  • write_pending_once writes and flushes but does not close or reopen the file.
  • enter_recovery_mode drops the writer only after an I/O error. Writing an unlinked Unix inode remains valid, so pathname replacement does not trigger it.
  • The background rollout_writer task owns this state until shutdown.

The source comments explain the background task as a way to keep writes asynchronous and ordered, but I could not find a documented requirement to retain the same descriptor across successful write batches. Is that descriptor lifetime intentional? If so, what invariant requires it?

There is already a reopen-per-call precedent for unloaded threads: append_rollout_item_to_path opens the current path, derives its ordinal state, appends, and then drops the local writer.

A focused regression test could persist item A, replace the active rollout with byte-identical contents on a new inode, persist and flush item B, and assert that the currently named rollout contains B with valid ordinals.

For comparison, a controlled single-session probe of Claude Code 2.1.220 on Linux found no transcript JSONL descriptor held open between turns. After a byte-identical path replacement, its next turn appended to the replacement inode. That does not make Claude immune to a short write-time race, but it avoids this persistent active-writer split.

View original on GitHub ↗

3 Comments

github-actions[bot] contributor · 16 days ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #37419

Powered by Codex Action

jdcodes1 · 9 days ago

Confirmed structure on main @ 1f41cc5d92: the recorder opens the rollout for append once and holds the File across the writer task's lifetime (rollout/src/recorder.rsopen_rollout_for_append at #L896, plain OpenOptions append at #L1634-L1638). Nothing re-checks that the fd still corresponds to the pathname, so unlink-and-copy / rename-over by any sync tool silently forks the history exactly as your sentinel repro shows.

Cheap fix that keeps the long-lived-fd performance: on each append batch (or a periodic tick), fstat the held fd and compare (st_dev, st_ino) against stat of the path — or simpler, check st_nlink == 0 on the fd. On mismatch, either re-open the path and replay/append (if the new file is a prefix, your byte-identical case) or fail the turn loudly with "rollout file was replaced externally" — silent success into an unlinked inode is the only unacceptable outcome. One syscall pair per batch is noise next to a JSONL write.

Also worth a docs line: exclude $CODEX_HOME/sessions (and the state DBs) from live sync/backup tooling — the same guidance every database ships with. That plus your #38150 lock-configurability ask covers the shared-home story from both ends.

shleder · 6 days ago

This is a nasty one — the fd stays open across the unlink-and-replace, so writes silently go to the orphaned inode and vanish on exit, and resume reloads the stale pathname and appears to time-travel backward. No error anywhere, which is what makes it so hard to catch.

The detectable signature is a live writer whose open fd points at an inode that no longer matches the pathname on disk. On Linux you can see it via the writer's fd vs the path's current inode; the mismatch is the smoking gun.

Full disclosure: I maintain codex-rescue (https://github.com/shleder/codex-rescue). codex-rescue writer --latest inspects active writer locks and process ownership read-only, and the Alpha7 writer diagnostics specifically check for the stale-fd / unlinked-inode condition so you can confirm a live session is writing to an orphan before it's lost. It won't reattach the fd (that needs the fix on the Codex side — reopen-on-replace or lease the pathname), but it will tell you definitively whether your active session is currently orphaned.