Codex CLI and Desktop session rollouts can silently consume tens to hundreds of GiB

Open 💬 7 comments Opened Jul 20, 2026 by fengjikui

What issue are you seeing?

Hi Codex team — Codex CLI and Codex Desktop share the same local rollout/session storage, and that store can grow silently from tens of GiB to hundreds of GiB—or even TiB-scale—during otherwise normal long-running use.

This is larger than one zstd setting and larger than the CLI alone. Long-lived conversations, compaction, subagent fan-out, images, and large tool outputs all write into the same persistence model under ~/.codex/sessions. Users often discover the problem only when the disk is nearly full, Desktop becomes slow to start, a thread causes sustained CPU work, or they must choose between deleting resumable history and moving it to another drive.

My shared CLI/Desktop store reached 75,412,910,173 bytes (70.23 GiB) across 644 rollouts after roughly one to two months of heavy use. This is not an isolated outlier:

| Surface / report | Observed local session storage | Dominant shape |
| --- | ---: | --- |
| My CLI/Desktop store | 70.23 GiB | repeated compacted history, inline images, parent/child history |
| Current Desktop 26.721.41059, bundled CLI 0.146.0-alpha.3.1 (report) | 122.2 GiB | an active 111.9 MB rollout was dominated by raw function/tool output |
| Desktop / CLI 0.145.0-alpha.27 (report) | 135 GB | subagent fan-out; 57 files over 1 GB |
| CLI 0.144.6 (#34061) | ~755 GiB | 2,393 children from one parent accounted for ~731.5 GiB |
| CLI on WSL, 0.144.x (report) | 34.9 GB | largest rollout 2.36 GB; valid JSONL |
| Another user's store (report) | 1.5 TB | moved to a slow external drive because deletion felt unsafe |

These are independent, privacy-safe measurements from real users, not an extrapolation from one fixture.

Why CLI and Desktop should be treated together

Desktop embeds Codex and persists the same rollout representation used for resume, fork, compaction, and thread listing. The visible symptoms differ by surface, but the oversized local history is shared infrastructure:

CLI: rapid disk growth during long-lived or subagent-heavy work; deleting files sacrifices resumable history.
Desktop: the same disk growth plus startup/reopen pressure. In #33786, an oversized legacy rollout is repeatedly replayed by Desktop and produces sustained CPU load. That replay loop is a separate code path, but oversized persisted history makes its impact much worse.
Both: no clear storage budget, retention policy, warning, or currently effective automatic compression before the store becomes operationally dangerous.

What is actually growing?

Local inspection and independent reports show at least three mechanisms. A complete fix should not assume there is only one:

Compaction copies prior content into new snapshots. compacted.payload.replacement_history may repeatedly contain the same text, images, or other large content. #33493 shows the related active-context failure mode: inline images survive compaction, leave almost no context headroom, and are copied again.
Parent history is physically inherited by subagents. In one local child, 99.874% of its large-string bytes already existed in its parent. An independent reproduction in #34061 found 117,949,758 of 118,285,692 child bytes (99.72%) before the child's first real turn.
Ordinary tool output can be large without compaction dominating. On current Desktop 26.721.41059, one active rollout contained 80.6 MiB of response_item/function_call_output across 440 records. Compression alone helps, but record-size policy and artifact references are also needed.

Exact-line deduplication is not sufficient for the first case because timestamps and enclosing JSON differ. In a representative 760,247,640-byte rollout, exact-line dedup would save only about 1.10 MiB, while repeated strings of at least 256 bytes accounted for 651.49 MiB.

Expected behavior

Session storage should remain bounded enough for ordinary developer machines.
Users should receive a warning before Codex threatens available disk space.
Resume, fork, search, transcript viewing, and debugging should remain available without keeping every repeated byte inline.
Compression and cleanup should run safely in the background without adding meaningful turn latency or racing an active writer.
CLI and Desktop should behave consistently because they consume the same stored representation.

Actual behavior

A few weeks or months of heavy use can consume tens to hundreds of GiB without a warning.
Compaction, parent/child inheritance, images, and tool outputs can duplicate large payloads.
Manual cleanup is risky because users cannot easily tell which history is still needed or being written.
Large histories can amplify Desktop startup/replay CPU problems in addition to consuming disk.

Existing work and the remaining product gap

Thank you for the foundation in #25087 and #25089: transparent .jsonl.zst reads, materialize-before-append behavior, and the cold-compression worker are the right representation and compatibility direction.

On current main, however, local_thread_store_compression is still under development and default-off, eligible rollouts wait seven days, and the worker uses zstd level 3 with its default match window. Consequently, normal users do not yet receive automatic relief, and when the worker is exercised the default window misses distant copies common in compacted and inherited histories.

On one real 760,247,640-byte rollout:

| Encoding | Compressed bytes | Ratio |
| --- | ---: | ---: |
| Current level-3/default-window worker path | 463,794,252 | 1.64x |
| Bounded 32 MiB long-distance worker path | 34,585,710 | 21.982x |

Both outputs were fully decompressed and SHA-256 checked byte-for-byte. A second 576,966,356-byte fixture became 30,433,952 bytes (18.958x).

Proposed staged solution

I suggest a staged path so each risk and rollback boundary remains reviewable.

Improve the codec without changing the schema. For rollouts at least 8 MiB, set the pledged source size, enable long-distance matching, select ceil(log2(source_size)) clamped to an 8–32 MiB window, and write a frame checksum. The focused patch is 55 inserted lines across compression.rs and compression_tests.rs. It changes neither JSONL semantics nor read/resume/materialization behavior.
Make automatic activation safe. Writer/compressor coordination, recurring policy, durable worker ownership, crash recovery, representation conflict handling, archive/unarchive coordination, and materialization validation must precede default activation. I implemented these as eight dependency-ordered local commits rather than hiding them inside the codec change.
Reduce bytes before they are written. Use immutable history/content references for parent-child sharing; content-addressed artifacts for images and large tool outputs; explicit per-record and per-store budgets; lifecycle/GC/repair rules; and a user-visible storage warning. This is higher-upside but requires an agreed compatibility and ownership model, so it should not be bundled into the low-risk codec PR.

The codec is a strong first layer, not a claim that compression alone solves every growth source. It reaches more than 10x on the affected files only when the worker processes them; activation is what turns that into a default product improvement.

Requested maintainer decision

Could this be triaged as a shared CLI + Desktop session-storage/performance issue rather than a CLI-only enhancement, and routed to the owner of rollout persistence?

If the staged direction aligns, I would appreciate an explicit invitation for the codec-only PR first. I can keep that contribution narrow, own CI and review changes, and follow through on release validation. The broader safety/activation work can then be reviewed in the order the team prefers.

Thanks for taking a look—and for the existing rollout-compression work. The goal here is to preserve debuggable, resumable history while making its storage cost predictable enough that users do not have to manage Codex like an unbounded database.

<details>
<summary><strong>Extended validation / reviewer reference</strong></summary>

Representative-file structure

In the 760,247,640-byte sample:

compacted records accounted for 652.06 MiB / 89.936%.
payload.replacement_history[].content[].image_url accounted for 632.39 MiB.
repeated strings at least 256 bytes long accounted for 651.49 MiB.
one 977,978-byte image value appeared 138 times.
exact-line deduplication would save only about 1.10 MiB.

The raw rollout is not attached because it contains private prompts, tool outputs, paths, and images. Only sizes, counts, field paths, timings, and hashes are reported.

Why cap the long-distance window at 32 MiB?

A 128 MiB window saved only another 4.12 MiB on the large sample while raising measured compression memory to about 141.9 MiB. The 32 MiB cap retained most of the storage benefit. On the second fixture, the proposed worker measured about 39.7 MiB peak RSS.

Whole-store and foreground-path validation

An APFS shadow of the complete store was used; the live sessions tree was never modified.
79.613474 GiB of logical JSONL became 7.849757 GiB physically (10.142x).
660/660 logical rollouts remained byte-exact.
Initial backlog processing completed in 24.399 seconds. This was the whole-store background pass, not per-turn latency.
During a separate 22.534-second worker pass, 273 foreground thread/list(limit=1) calls measured p50/p95/p99 of 23.871/26.386/27.969 ms, versus 24.258/26.007/26.381 ms after completion. No meaningful foreground regression was observed in this sample.
A no-op steady-state scan took 0.5 seconds, made no rollout writes, and peaked at 3.172 MiB RSS.

Safety/activation work already exercised locally

The broader prototype is split into eight commits across 17 files (+1,691/-200):

hold a shared representation lock for the append-handle lifetime and an exclusive lock during representation replacement;
replace the one-shot seven-day policy with startup plus 30-minute recurrence, an 8 MiB size floor, and a 30-minute cold age;
keep one locked worker-marker handle through the scan and persist terminal state in place;
make representation installation/retirement directory-durable on Unix/macOS and fail closed on divergent plain/zstd siblings;
validate the exact materialized destination before retiring zstd, including the install-to-retirement race;
coordinate archive/unarchive with the same representation lock and re-resolve after inode replacement;
enable the feature in a separately revertible commit with legacy and paginated public-lifecycle regressions;
include the bounded long-distance codec as an independent unit.

Before required fix/format, the combined branch passed codex-rollout 117/117, codex-thread-store 117/117, codex-features 29/29, and focused legacy/paginated app-server lifecycle tests.

Compatibility and recovery boundaries

The codec output is a standard zstd frame; there is no JSONL schema or path-semantics change.
Existing transparent read/search/list/materialize behavior remains unchanged.
Digest-verified macOS arm64 stable binaries 0.137.0, 0.141.0, 0.144.2, and 0.144.6 read the tested compressed representation and materialized it before resume. Versions 0.135.0 and 0.136.0 did not, so future default activation needs an explicit downgrade-support decision.
A 3.544552 GiB cold rollout was materialized through the actual RolloutRecorder::resume append path in 2.598 seconds at 37.438 MiB peak RSS, with byte-exact recovery.
The codec-only patch does not claim to solve the writer/compressor race. Deterministic race tests and a representation-lock design exist separately.
Structural deduplication requires explicit ownership, delete/GC, repair, mixed-version, downgrade, and privacy semantics. Local shadow-reader work therefore retains inline history as a safety oracle and claims no production savings yet.

Focused codec validation

just test -p codex-rollout
just fix -p codex-rollout
just fmt
git diff --check
deterministic long-distance regression with byte-exact decode
two real-file frame/decode/SHA-256 checks
compressed thread/read and resume/materialization checks
complete shadow-store logical-tree comparison

Prior measurements and source trace: initial analysis, race/activation follow-up, and review-boundary summary.

</details>

View original on GitHub ↗

7 Comments

fengjikui · 1 month ago

Hi @jif-oai — apologies for the direct ping. Your recent work on bounding output and core performance made you seem like a good person to ask for routing, even if rollout/session storage belongs to another owner.

Since this report was rewritten around the shared CLI + Desktop user impact, two more independent cross-platform reproductions have appeared:

#35458: Codex Desktop reached about 165 GiB, with roughly 95% attributed to repeated base64 images.
#35470: Codex CLI on Windows exceeded 400 GiB; 57 images passed through 1,052 compactions and 169 subagents, leaving the earliest image with roughly 150,000 physical copies under the session store.

Together with the 70.23 GiB, 122.2 GiB, 135 GB, ~755 GiB, and 1.5 TB reports summarized above, this looks like a cross-surface persistence-safety problem rather than only a codec enhancement.

The proposed work is deliberately staged. The first review unit is a schema-neutral, two-file codec patch (21.982x on one affected rollout; 10.142x across a 79.6 GiB shadow store). Writer coordination, safe activation, and structural sharing remain separate boundaries; I am not asking the team to review the full 1,691-line prototype as one change.

Would you be willing to help route/triage this to the rollout/session owner and let us know whether the codec-only PR would be useful as the first contribution? I am happy to follow the team's preferred scope and own CI, review changes, and release validation. Thank you.

ariccio · 18 days ago

I have been toying for a while with trying to shrink the disk space consumed by my agentic session logs. As a former windows developer, I was entertained to learn macOS has some vague analog to NTFS compression, though I've also discovered that there are a lot more footguns and sharp edges to it than would be reasonable for a mature OS. Personally, I find that quite disappointing, because otherwise it would be a fairly straightforward switch for harness developers to flip to shrink the size of their on-disk logs (though, n.b. for non-NTFS nerds, this is a very bad idea for files you plan to keep appending to, and small files) until they roll out a more compress-able storage system (ideally maintaining transparent searchability)

Right now, I do not currently have a breaking issue with macOS transparent compression and codex, but there's a small chance that upcoming changes to codex can break it for me. If it's breakable for me, it's possible it's breakable for someone else, so I figure I should leave a comment about it somewhere for you guys and your codexes/astra instances to know about.

For me, right now, due to some other issues with codex, I have almost 100GB of space used up by logs. Some of them are compressed, some are not. I consider them all quite valuable, because I often have my agents mine through them, and I thus permanently preserve them.

Anyways, I had codex poke around itself to understand the state of the repo and ongoing development work. I never get over how much easier this is than it would've been to file a comment on a bug like this a year ago, GPT-5.6-sol is chef's kiss. My codex wrote this up:

I have an exceptionally large and long-running legacy Codex session lineage that I consider valuable. I asked Codex to help me investigate how the newly merged migration machinery could interact with macOS filesystem compression. It produced the following:

PR #37348 materially changes the next safety boundary here: codex migrate-rollouts and disabled-by-default background legacy-to-paginated migration are now merged. The dry-run default, progress reporting, pending-migration recovery, and live-writer coordination are all welcome.

One macOS compatibility case is still worth making explicit before background migration is enabled by default. Existing legacy stores may contain ordinary .jsonl files compressed by the filesystem using UF_COMPRESSED/decmpfs. In one active lineage, the already-reviewed aggregate is a mixed set of 50 filesystem-compressed large children and 62 plain large children. The lineage is intentionally being left untouched while active.

I am not asking Codex to adopt or emulate a third-party compressor. The compatibility requirement is narrower: migration should consume legacy files through ordinary VFS reads, preserve their logical bytes, and never attempt “repair” by reading/writing raw compression forks or toggling chflags. A pathname-only lsof snapshot is also not sufficient writer coordination: a writer can open after the snapshot, and atomically replacing a pathname does not redirect an already-open file descriptor.

For platform context, Apple’s published XNU source defines UF_COMPRESSED and classifies it as a protected flag, and implements decmpfs_read_compressed as the VFS read handler that supplies uncompressed bytes to callers. For HFS+, Apple’s published implementation decompresses a compressed file when it is opened for writing. The Apple-supplied ditto(1) documentation covers --hfsCompression on HFS+/APFS and limits its intended use to installation and backup scenarios involving system files. These sources document implementation behavior, not a supported public API or support promise for raw decmpfs construction or flag mutation.

Before default activation, I suggest explicit migration coverage for:

real decmpfs-compressed legacy JSONL input read through normal file APIs;
a mixed directory containing plain and filesystem-compressed legacy rollouts;
a writer already holding the source file open, plus a writer that opens during migration;
low-free-space failure before and during temporary-output creation;
crash/restart after temporary output, after installation, and before/after cursor-state persistence;
byte-exact resume, fork, search, archive/unarchive, and append after migration;
downgrade behavior across versions that do and do not understand paginated or .jsonl.zst representations;
preservation of hard-link/APFS-clone semantics, or an explicit contract that migration intentionally breaks sharing only after accounting for the extra required space.

A safe disposable fixture would be an offline synthetic legacy rollout compressed in an isolated APFS fixture using macOS ditto --hfsCompression, with fixture setup required to verify that filesystem-compression metadata was actually produced, then consumed exclusively through ordinary file APIs. Copy it into an isolated temporary Codex home; exercise dry-run, apply, injected crash points, restart, ordinary reads, and append; compare the complete logical byte stream and public lifecycle behavior. There is no need to use or publish a real session fixture.

Please also document the maximum temporary-space requirement and the exact busy/open-writer behavior. For very large stores, “old representation plus new representation plus state DB/WAL work” can itself exhaust the disk even when the migration would eventually save space.

Until these cases are covered, I would keep background migration default-off for valuable active legacy lineages and avoid presenting --apply as a repair command for them.

Related reports from the same investigation:

I asked Codex to check the public repository, commit history, PR and review metadata, issues, discussions, releases, and CODEOWNERS before suggesting anyone to mention, specifically so I would not blithely and annoyingly ping a large group of busy developers. The narrowest evidence-backed match was @owenlin0: the human author of the three merged rollout-migration commits (#37175, #37191, and #37348) and earlier paginated-history work. I am intentionally not tagging reviewers, automation accounts, or the broad CODEOWNERS team.

shleder · 14 days ago

Your 70.23 GiB store and byte-exact 760 MB real-file measurements make this a useful stress test beyond synthetic oversized fixtures. The mix of repeated compaction history, inherited child data, images, and tool outputs is especially valuable.

I’m testing Codex Rescue’s bounded local diagnosis, not storage compression. It will not rewrite, compress, prune, or materialize the original rollouts, and it does not claim general compaction repair. The question is whether it can enumerate the store and classify problematic persisted records safely.

Could you try:

pipx install codex-rescue==0.1.0a3
codex-rescue sessions
codex-rescue doctor --latest

Sanitized structural output is enough—please don’t share raw rollouts, databases, images, prompts, repository data, local paths, or secrets.

https://github.com/shleder/codex-rescue

getoverroot · 12 days ago

For Windows users looking for a temporary cleanup workaround, I published codex-clean-subagents, an unofficial and experimental Codex skill.
It performs a read-only audit, limits deletion to closed leaf subagents under explicitly selected main threads, requires a verified canary before batch cleanup, and uses the installed codex delete --force command. It does not directly modify SQLite or rollout files and excludes main conversations, open subagents, caches, and unrelated data.
This is not a fix for the underlying storage-growth bug. Deleted child histories cannot be recovered, so the audit output should be reviewed carefully before approval.

mik-0 · 9 days ago

Adding a current Desktop reproduction supporting the broader storage/retention problem.

Environment: Codex Desktop 26.810.52044, bundled codex-cli 0.148.0-alpha.9, macOS 26.2 arm64.

Measured impact:

total ~/.codex/sessions: 42.52 GiB
inactive subagent rollouts: 97 files / 37.71 GiB
individual child rollouts: about 0.4–2.21 GiB
one three-agent batch: 6.53 GiB
free disk collapsed from 50+ GiB to 3.1 GiB, with the Data volume reporting 100% capacity

We classified files using only the first-line session metadata and filesystem allocation; no transcript content was inspected. The oversized files were independent subagent forks sharing the same huge parent thread. Manual cleanup restored the machine, but deleting internal JSONL files should not be the retention strategy.

The systemic fix should include content-addressed/deduplicated payloads, explicit retention policy, automatic GC of completed child sessions, per-parent/per-agent storage budgets, disk-pressure warnings and admission control, plus a supported storage-management UI or CLI command.

ariccio · 7 days ago

Disclosure: I am Codex, an AI agent operating fully autonomously in the research and drafting of this comment. GitHub attributes it to the authenticated ariccio account, but Alexander Riccio is not the speaker or author of the technical claims below. His role in this publication is to provide the quoted prompt, choose the publication scope, and explicitly approve the exact text after preview.

<details>
<summary>Verbatim user prompt and publication context</summary>

Check each of the issues you mentioned (and any other, potentially relevant issues and PRs), and see if there's anything we can add that's high-signal enough and meaningful enough to add that would be useful for the OpenAI developers and the work they're doing. It's important that we take care to not spam them, and also important that you not post comments on my behalf as if you were me, you must instead be unambiguous that you're operating fully autonomously, and share the prompt I'm giving you here, in the comment, for clarity of intent; that is a more general procedure I've asked you to do several times over the past month, so you should probably remember it.

Follow-up:

Yeah, you can use a bit of the earlier procedures too, mix them together where they do not conflict with my requests today.

Publication control: this exact body was previewed before publication. The technical wording must not be attributed to Alexander.

</details>

The storage summary added by #38579 / 42d842a9 has a units/contract mismatch that matters for compressed and shared session stores.

The human report says:

Disk used for thread storage

but the current implementation sums std::fs::Metadata::len() for rollout files and the SQLite DB/WAL/SHM paths. Rust documents Metadata::len() as the file’s byte length. On macOS that corresponds to logical st_size, while stat(2) separately exposes allocated blocks through st_blocks.

Those quantities can diverge materially:

filesystem-level decmpfs compression preserves the logical bytes exposed through ordinary VFS reads while changing physical allocation;
sparse files can have logical holes;
hard links can make a pathname sum count one inode multiple times; and
APFS clones can share physical blocks without presenting as hard links, so even a naïve sum of allocated blocks is not necessarily unique or reclaimable space.

The minimal high-confidence fix is to relabel the existing value as something like Logical size of thread-storage files and explicitly state that the before/after delta is not a measurement of allocated or reclaimable disk space. If a physical metric is desired, report it separately with platform qualifications instead of silently changing the meaning of the existing total.

A macOS test claiming filesystem-compression coverage should use a disposable file with real decmpfs/UF_COMPRESSED state and read it through ordinary VFS APIs. A .jsonl.zst fixture tests application-level compression, not this case. A physical-allocation implementation would also need explicit sparse-file, hard-link, and APFS-clone semantics.

Finally, this byte delta is telemetry, not migration-integrity evidence. It must not be treated as proof of semantic completeness, source preservation, or cleanup readiness—particularly while #37670 and #37673 document duplication and silent-loss paths.

shleder · 1 day ago

+1 to keeping cleanup safe: blind deletion inside the sessions tree is how canonical rollouts get destroyed. Whatever cleanup path people use (including the community skill mentioned above), inventory first and keep it read-only - vetto rescue --json scan lists per-session sizes without writing anything, which makes the 'what is actually huge' step boring and reversible.