Session forks duplicate parent history on disk; use delta/DAG storage to avoid large storage amplification

Open 💬 2 comments Opened May 14, 2026 by SimpleZion

What version of the Codex App are you using?

Codex Desktop on Windows:

  • Package observed locally: OpenAI.Codex_26.506.3741.0_x64
  • Session storage path shape: %USERPROFILE%\.codex\sessions\YYYY\MM\DD\rollout-*.jsonl

What platform is your computer?

Windows x64.

What issue are you seeing?

Forking / branching a long Codex conversation appears to create a new rollout-*.jsonl file that copies the parent session history into the child file, even though the child session_meta also records forked_from_id.

This makes branching from long conversations very storage-expensive. For users who frequently branch from long-running sessions, storage grows roughly with the full parent history size per branch instead of only with the new branch delta.

This is especially painful because branching from long conversations is an important workflow: users often want to explore alternatives while preserving the original conversation.

Local evidence

On one local machine, .codex/sessions had grown to:

jsonl files: 876
total size: about 26.57 GB
files above 100 MB: 38
size of files above 100 MB: about 24.86 GB

A concrete parent/child fork pair showed the behavior clearly:

parent session id: 019e1bc3-8787-7470-97b7-d35694bd1780
parent file size: about 2.858 MB
parent record count: 910
parent type counts: session_meta=1, turn_context=10, response_item=663, event_msg=234, compacted=2

child session id: 019e245b-687d-7f20-ac51-b8fd4a84d160
child forked_from_id: 019e1bc3-8787-7470-97b7-d35694bd1780
child file size: about 3.018 MB
child record count: 943
child type counts: session_meta=2, turn_context=11, response_item=684, event_msg=244, compacted=2

The child file starts with:

line 1: session_meta for the child, with forked_from_id=<parent id>
line 2: session_meta for the parent
line 3 onward: copied parent history records, then child branch records

So forked_from_id is present, but the physical JSONL storage is still mostly a copy of the parent prefix.

Expected behavior

A fork should preserve the full logical conversation history, but the child session should not need to physically duplicate the full parent history on disk.

A long-session fork should be closer to:

child_session = parent_session_id + fork_offset + child_delta_events

The cost of a fork should scale with the new branch content, not with the entire inherited history.

Suggested upstream design

A more storage-efficient session model could use a DAG / delta store:

  1. Keep each session's immutable event log or chunked event stream.
  2. Represent a fork as metadata containing:
  • parent_session_id
  • fork_event_index or parent byte/record offset
  • child branch event log path
  1. Reconstruct logical history by walking the parent chain plus the child delta.
  2. Add periodic snapshots for fast loading so deep fork chains do not require expensive replay.
  3. Use content-addressed chunks or shared immutable segments for large repeated prefixes.
  4. Add reference counting / garbage collection so unused parent chunks can be reclaimed safely.
  5. Preserve backward compatibility by treating current copy-style JSONL files as importable snapshots.

Why hard links are not a safe substitute

Hard-linking whole JSONL files is risky because child sessions continue to append independently. Whole-file hard links would couple parent and child writes and could corrupt session semantics unless the store is made append-segment-aware.

Local mitigations that help but do not solve the upstream issue

For current builds, users can reduce the impact by enabling transparent filesystem compression on %USERPROFILE%\.codex\sessions or by archiving cold sessions. On the affected machine, sampling large JSONL files showed text compression ratios around 1.7x to 2.17x. However, this is only mitigation; it does not fix the structural duplication caused by copy-style forks.

Related note

This storage amplification also increases the amount of session data other Codex components may need to watch, scan, index, or parse. In another local investigation, a very large .codex/sessions tree was a plausible contributor to native host memory pressure. Reducing duplicated session data would therefore help both disk usage and performance/memory risk.

View original on GitHub ↗

2 Comments

blain3white · 2 months ago

Clean My Agent (I'm the author) includes storage analysis for Codex sessions — it can surface the actual on-disk footprint per session before native deduplication ships: https://github.com/blain3white/clean-my-agent

Yasei-no-otoko · 1 month ago

I reproduced the same storage amplification on macOS and traced it to the current legacy app-server fork path.

Sanitized local inventory:

  • 934 rollout JSONL files inspected by metadata and size only
  • 209 files carry forked_from_id
  • direct fork children account for about 8.67 GB, roughly 62% of current rollout JSONL storage
  • 18 root or explicit fork children alone account for about 7.42 GB
  • the largest children are distinct thread IDs whose first records are child session_meta, inherited parent session_meta, then the copied parent prefix

Current implementation trace:

  • thread/resume reopens the existing rollout path with append semantics; it is not the path creating the copied child prefix.
  • For a legacy JSONL source, app-server thread/fork loads persisted history with include_history = true, then ThreadManager::fork_thread_from_history selects ForkPersistence::Copied.
  • Session initialization persists the inherited items into the new child rollout.
  • Paginated rollouts already use prepare_fork plus ForkPersistence::Referenced and a frozen history_base, so they do not need this physical full-prefix copy.

This issue is therefore the canonical report for explicit or root session forks. #34268 is the related automatic multi-agent full-history fork amplification path; the two paths should remain cross-linked.

I also audited all 194 open PRs visible at 2026-07-27 11:28 JST, including relevant persistence and fork candidates, and found no open PR that adds reference-backed lineage for legacy root thread/fork. The closest prior architecture proposal was #27249, which is closed and unmerged.

The expected invariant is exactly the one in this issue: preserve the full logical history while storing only a frozen parent reference plus child-local delta. A safe implementation also needs cold read/resume reconstruction, nested lineage, delete/archive protection, and a test proving parent appends after the fork are excluded from the child.