Session forking writes full parent rollout into child JSONL (storage amplification despite forked_from_id)

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• #35458

Powered by Codex Action

Implementation update (2026-07-28):

• Full open-PR audit: 193 open PRs checked; no existing PR implements managed history_base references plus fork-local JSONL deltas. #30977 is related payload filtering but does not change child persistence.
• Fix branch: Yasei-no-otoko:session-jsonl-amplification-upstream
• Head: 61cb3d7565bf6409ea10bd55082675797086085b
• Updated onto upstream main at 49025589b0216b876b1a6a20977536c7d55cdb8b via merge commit 477a26d4d2.
• Validation: cargo +1.95.0 test -p codex-thread-store --lib => 192 passed, 0 failed; package fmt check passed.
• Added regression coverage for concurrent archive versus legacy fork mode dispatch and Paginated Latest cutoff after blank/rejected complete physical tail lines.
• The two reported local child rollouts are now delta-only references: 67,930 bytes / 16 records and 23,463 bytes / 2 records, each with a frozen parent history_base cutoff.

Compare URL: https://github.com/openai/codex/compare/main...Yasei-no-otoko:codex:session-jsonl-amplification-upstream?expand=1

I cannot open the upstream PR from the current account: GitHub returns Yasei-no-otoko does not have the correct permissions to execute CreatePullRequest because this repository's external contribution flow is invitation-only. The branch is pushed and ready for an invited maintainer/account to open the PR. The PR should use Fixes #35647 and Related to #22593.

Additional reproduction details for Issue 1 have been confirmed.

The affected child’s legacy history_base.end_byte_offset falls immediately after a newline in the parent JSONL, so it is physically at a complete record boundary. However, the preceding record’s token_count.rate_limits used an older or newer schema that differs from the current Rust type. As a result, legacy_fork::last_complete_jsonl_offset_at_or_before treated a nested RolloutLine deserialization failure as incomplete JSONL, causing resume to stop with:

invalid rollout history lineage for 019fa108-23a7-7921-ab39-0092e1227f2d: cutoff byte offset is not at a complete JSONL record

The fix restricts cutoff validation to the stable outer rollout envelope (timestamp: string, type: string, payload: object) and permits evolution of nested payload schemas. It requires the rollout-envelope shape rather than accepting arbitrary objects, and continues to reject malformed envelopes and payload: null or array values.

The regression tests and all 195 codex-thread-store tests pass.

Fix PR (submitted from a fork): https://github.com/Yasei-no-otoko/codex/pull/5

I reproduced this on Codex CLI 0.146.0 on Linux x86_64 and have a second, sanitized data point that narrows the high-frequency trigger.

Observed chain

A single long-lived process launched as plain codex created seven top-level user-thread children over 54 minutes. Their rollout sizes were:

62.68 MiB
63.56 MiB
63.80 MiB
63.87 MiB
63.98 MiB
64.02 MiB
65.08 MiB

All seven canonical session_meta records reported:

source = cli
originator = codex-tui
thread_source = user
history_mode = legacy
cli_version = 0.146.0

Each child had a fresh thread ID and forked_from_id pointing to the immediately preceding thread, producing one continuous lineage. This is about 447 MiB of child rollouts from one active conversation in under an hour.

Trigger classification

This chain is not byte-identical duplicate files and does not look like same-ID resume corruption. It is source-preserving prompt-edit/turn-replacement branching:

• Several source turns ended with the durable <turn_aborted> marker.
• For four corrected prompts, the discarded prompt occurred twice in the source rollout (response_item + event_msg) and zero times in the child's inherited initial batch.
• The replacement child retained the earlier conversation and excluded the interrupted prompt, which is consistent with a branch-before-turn operation.
• Model and effort remained gpt-5.5 / medium; I found no persisted model-safety-buffering event in the chain, so this does not match the faster-model safety retry path.

The branch behavior itself appears intentional. The storage amplification is not: prompt editing is a normal, potentially frequent TUI workflow, and every edit materialized nearly the complete legacy parent history again.

Persistence evidence

On each child creation, inherited rollout items were serialized again with timestamps clustered within roughly one second of the child's creation time. Historical session_meta records were inherited too, so the number of persisted session_meta lines increased by one at each generation. This explains why exact-file hashing finds no duplicates even though almost all logical history is copied.

The current implementation trace matches the analysis already posted here and in #22593:

• legacy root/user forks reach ForkPersistence::Copied;
• record_initial_history persists the reconstructed inherited items into the child rollout;
• paginated forks already have the reference-backed history_base path from #35220, but existing and newly continued legacy threads remain on copied persistence.

I am also aware of the external patch at Yasei-no-otoko/codex#5 and do not want to create a competing unsolicited implementation.

Contribution request

The repository contribution guide requires an explicit maintainer invitation before opening an external PR. If the team wants an external contribution here, I can prepare a focused, test-first PR against the preferred architecture rather than submitting another broad lineage rewrite.

A possible first reviewable stage would be a regression test for a legacy TUI prompt-edit fork showing that the child persists a bounded parent reference/delta rather than reserializing the complete parent rollout. Please confirm whether the intended direction is:

1. reference-backed legacy forks,
2. migration/new-thread defaulting to paginated history, or
3. another staged approach already planned internally.

I can provide additional aggregate-only measurements without exposing conversation contents.

Hi @Yasei-no-otoko, this subagent persistence/history issue aligns with some boundary anomalies observed in multi-agent rollouts. Codex Rescue Alpha5 provides read-only lifecycle and subagent boundary diagnostics, cleanly separating historical start markers from current live execution state without altering the source rollout.

If you have access to the local session, you can run a non-destructive check:

pip install codex-rescue==0.1.0a5
codex-rescue doctor --latest

No raw session data is required, and please redact private paths if you share any output.