Codex app-server reaches 27 GB footprint and 36 GB swap after compacted records grow one rollout JSONL to 10.2 GB with repeated inline PNG data URLs
Summary
A long-running, image-heavy Codex thread produced a 10.2 GB rollout JSONL under ~/.codex/sessions. The file is dominated by very large compacted records that retain PNG screenshots inline as data:image/png;base64,... strings.
While the thread was active in Codex Desktop, the codex app-server process reached a 27 GB physical footprint, with approximately 26.2 GB swapped out. System swap later reached 36.1 GB on an 18 GiB Mac. macOS generated a JetsamEvent that identified codex as the largest process.
One inspected compacted record was 50,382,022 bytes and contained 24 inline PNG data URLs totaling 50,276,604 bytes. More than 99.7% of that record consisted of image data. Other sampled compacted records were approximately 62 MB, 77 MB, and 79 MB.
This appears to combine two related problems:
- Compaction replacement history retains full inline image payloads and appends another large snapshot to the rollout.
- Resuming or continuing an oversized rollout causes
app-serverto allocate multiple times the file size in heap memory.
The affected rollout contains private project screenshots, so I cannot attach the original 10.2 GB file publicly. All evidence below was collected with streaming or fixed-size sampling, without loading the full file into an analysis process.
Environment
- Product: Codex Desktop inside ChatGPT for macOS
- ChatGPT app version:
26.715.72359 - Current bundled Codex version:
codex-cli 0.145.0-alpha.30 - Affected session metadata version:
0.142.4 - Affected session source:
vscode - macOS:
26.5.2 (25F84) - Architecture: Apple Silicon, arm64
- Physical memory: 18 GiB
- Affected thread age: created June 30, 2026 and still active on July 23, 2026
- Affected thread type: long-running local UI development task with repeated browser screenshots and automatic continuation/compaction
Timeline
- A UI development thread was created on June 30 and continued running across many turns.
- The thread repeatedly used browser and image tooling while developing and validating a local web application.
- On July 23, the Mac became extremely slow, disk usage was unexpectedly high, and memory usage appeared abnormal.
- The rollout file was found at 10.2 GB and was still being appended to by
codex app-server. footprintandvmmapshowed approximately 27 GB allocated to the Codex process.- macOS generated a JetsamEvent at 09:42 local time with
largestProcess: codex. - Fixed-size sampling of the rollout showed that most sampled regions fell inside giant single-line
compactedevents containing inline base64 PNGs.
Observed system impact
At the time of diagnosis:
Physical RAM: 18 GiB
Codex footprint: 27 GB
Codex peak footprint: 28 GB
Codex MALLOC_LARGE: 17 GB
Codex MALLOC_SMALL: 10 GB
Codex writable regions: 28.4 GB
Codex resident memory: about 944 MB
Codex swapped memory: about 26.2 GB
System compressed memory: about 7.4 GB
System swap used: 26.2 GB, later rising to 36.1 GB
Memory pressure level: 2
System free memory estimate: 26% to 28%
Load average peak observed: 78.53
Jetsam largest process: codexThe low RSS shown by ps was misleading because most of the Codex allocation had already been compressed or swapped. footprint and vmmap both reported the much larger allocation.
Relevant vmmap summary:
Physical footprint: 26.9G
Physical footprint (peak): 27.6G
Writable regions: Total=28.4G written=19.4M resident=742.4M swapped_out=26.2G
MALLOC_LARGE: 17.1G total, 16.3G swapped
MALLOC_SMALL: 10.2G total, 9.5G swapped
DefaultMallocZone: 26.8G allocatedThe app-server had the rollout open through both read and write descriptors while the problem was occurring.
Rollout evidence
File size and session state
Logical size: 10,213,146,014 bytes
Allocated size: 10,218,827,776 bytes
Session tokens_used field: 1,953,300,167The token value appears to be cumulative and includes repeatedly replayed context. It should not be interpreted as 1.95 billion unique generated text tokens.
The current unrelated diagnostic thread had a rollout of only about 2 MB, which isolated the problem to the older active thread.
Fixed-size sampling
I sampled sixteen evenly spaced 4 MiB regions across the rollout.
- Twelve of the sixteen samples contained zero newline characters, meaning each sample fell inside a single JSONL record larger than 4 MiB.
- Those zero-newline samples repeatedly contained
data:image,image_url, andbase64markers. - Normal event-dense regions contained hundreds of newlines within the same 4 MiB sample size.
Sampled compacted record sizes
Boundary scans around several random offsets found the following complete JSONL record sizes:
50,382,022 bytes type=compacted
61,676,555 bytes type=compacted
76,688,051 bytes type=compacted
79,544,935 bytes type=compacted
79,532,979 bytes type=compactedEach sampled record contained inline PNG data URLs.
Detailed breakdown of one record
One 50,382,022 byte compacted record contained:
Inline PNG data URLs: 24
Combined data URL bytes: 50,276,604
Share of record occupied by image data: approximately 99.79%The 24 individual data URLs ranged from approximately 382 KB to 3.18 MB. The decoded PNG bytes would be smaller, but base64 adds approximately 33% encoding overhead.
Reproduction outline
The original thread is too large and private to share. A smaller reproduction should be possible with the following pattern:
- Start a Codex Desktop or VS Code thread for a local UI project.
- Use browser or image tools repeatedly so the conversation contains multiple full-resolution PNG screenshots as inline image inputs or tool results.
- Continue the thread until automatic context compaction occurs.
- Continue adding screenshots and allow compaction to run repeatedly.
- Inspect
~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl. - Search for records with
"type":"compacted"and inspect their byte lengths. - Verify whether each compacted replacement retains previous
data:image/png;base64,...payloads. - Resume or continue the oversized thread in Codex Desktop.
- Inspect the app-server with
footprint -p <pid>andvmmap -summary <pid>.
Actual behavior
- Each compaction appends another snapshot containing large inline image payloads.
- Historical image bytes remain in earlier append-only records.
- New compacted records continue carrying image payloads forward.
- The rollout grows without an effective per-thread or per-record limit.
- An active oversized rollout can consume multiple times its on-disk size in app-server heap memory.
- The whole machine experiences heavy compression, swap growth, high load, and Jetsam pressure.
- An inactive or background thread can degrade the entire Codex Desktop process without clear UI attribution.
Expected behavior
- Compaction should remove, summarize, externalize, or deduplicate historical image bytes.
- A compacted replacement should not retain full base64 screenshots indefinitely.
- Rollout persistence should store large binary assets separately and reference them by content hash or stable asset ID.
- Opening or continuing one oversized thread should not require parsing all inline image bytes into heap memory.
- Codex should enforce limits for record size, thread size, image count, and retained historical image bytes.
- The app should warn when a thread is large enough to threaten disk space or memory stability.
- One problematic thread should not place all other Desktop threads and the operating system at risk.
Root-cause hypothesis
The rollout format appears to reuse the in-memory multimodal message representation for persistence. Image inputs are serialized as self-contained data URLs. During compaction, replacement history retains those image-bearing messages. The result is repeated persistence of the same class of heavyweight payloads in an append-only log.
When app-server resumes or continues the thread, JSON decoding, base64 strings, replacement-history structures, request construction, and intermediate buffers can coexist. This likely explains why a 10.2 GB file resulted in approximately 27 GB of malloc footprint.
The exact allocation call stack was not collected, so the final in-memory multiplication mechanism remains a hypothesis. The on-disk image amplification and the 27 GB process footprint are directly measured.
Suggested fixes
- Replace historical inline image items with a compact placeholder after compaction, for example
[Image omitted during compaction]plus lightweight metadata. - Store screenshots in a content-addressed blob store and keep only a SHA-256 reference, MIME type, dimensions, and asset ID in JSONL.
- Deduplicate identical image payloads across turns and compactions.
- Keep full-resolution bytes only for a small recent window. Use thumbnails or textual summaries for older images.
- Stream and lazily decode rollout records. Avoid materializing an entire large thread or all image strings at once.
- Add hard limits and graceful degradation for:
- Maximum JSONL record size
- Maximum retained image bytes per compacted record
- Maximum rollout size
- Maximum history bytes loaded during resume
- Add a repair or migration path that can externalize or replace historical inline images in existing oversized rollouts.
- Surface thread size and the identity of a background thread that is consuming significant CPU, memory, or disk.
- Add regression tests that run repeated compaction with multiple image inputs and assert bounded rollout growth.
Workaround
The current safe workaround is:
- Stop the affected background thread.
- Fully quit Codex Desktop so app-server releases the rollout and its heap allocation.
- Back up the rollout before changing it.
- Move the oversized rollout out of the active sessions directory, or stream-rewrite a copy that replaces historical inline image data with placeholders.
- Restart Codex and verify memory pressure and swap usage.
Deleting or rewriting a rollout while app-server still has it open for writing is unsafe.
Related issues
- #24388: Remote compaction deadlock when
input_imagepayloads remain in compacted replacement history - #24550: Responses WebSocket fallback when compacted
replacement_historycontains large inline images - #24676: Codex Desktop hangs when opening an image-heavy rollout with inline image records
- #24948: Session logs grow to hundreds of MB or multiple GB from repeated compaction history and raw tool output
- #21948: Large active session causes global freezes without UI attribution
- #22004: Desktop crash when rollout JSONL exceeds V8 maximum string length
This report adds an extreme macOS case with a 10.2 GB rollout, measured 27 GB app-server footprint, 36 GB swap usage, a JetsamEvent naming Codex as the largest process, and a byte-level breakdown showing that 99.79% of one 50 MB compacted record consists of 24 inline PNG data URLs.
Privacy and diagnostic availability
The original rollout contains private screenshots, prompts, tool outputs, and local project paths. It should not be uploaded publicly. I can provide:
- Redacted command output from
footprint,vmmap, and the JetsamEvent - A streaming analysis script that reports record sizes and image byte counts without exposing image contents
- Sanitized synthetic reproduction data
- Additional diagnostics through a private support channel if requested
10 Comments
Potential duplicates detected. Please review them and close your issue if it is a duplicate.
Powered by Codex Action
Corroborating data point from Windows, with no images involved — which may help isolate problem (2) in your summary from problem (1).
Environment
What I saw
A thread created 2026-07-18 and continued for nine days accumulated a compaction chain 119 windows deep. Each
compactedrecord containsreplacement_historypluswindow_number/first_window_id/previous_window_id, forming a linked list back to the original window.The records grow monotonically along the chain:
Every session that continues that thread rewrites the entire 119-window chain into its new rollout file. Measured on one session: 67.8 MB written in 1.358 seconds (timestamps 11:28:56.992 → 11:28:58.350). Total session file 102.8 MB, of which 66% was
compactedrecords — 119 records for 67.8 MB, against 25.9 MB for 32,298event_msgrecords.This reproduced identically across separate sessions started the same day:
| session started | compacted records | bytes | largest record |
|---|---|---|---|
| 10:40 | 118 | 66.8 MB | 1038 KB |
| 11:45 | 119 | 67.8 MB | 1038 KB |
| 12:19 | 119 | 67.8 MB | 1038 KB |
The controlled comparison
Sessions started clean on the same machine, same day, under the same load:
| inherited chain depth | session size |
|---|---|
| 119 windows | 100–103 MB |
| 31–33 windows | 13–21 MB |
| 0 (fresh thread) | 0.3–1.0 MB |
Size tracks chain depth linearly. No images were involved in any of these — the bulk is
replacement_historytext, so the replay/growth problem is not specific to inline data URLs.Different symptom to yours, same cause
On Windows this surfaced in the renderer, not app-server. Parsing ~68 MB of JSON into JS objects at session start put the renderer heap near 1 GB before any work began; ordinary use then pushed it to 1.7–1.8 GB, where a full GC pause runs for minutes. Measured during one: the V8 main thread pinned in
Runningat 90.7% of one core with 5,260 s accumulated CPU, then freed 721 MB and recovered.The UI stops repainting for the duration, so it presents as a crash — but Crashpad recorded zero dumps across nine days. It never crashes; it hangs and recovers. That distinction cost me a lot of diagnostic time and may be worth noting for others searching.
Workaround
Retiring long threads and starting fresh ones drops new sessions from ~100 MB to under 1 MB, which keeps renderer heap in the low hundreds of MB and eliminates the pauses entirely.
Suggestion
I notice
history_modealready supportslegacy/compact/fullinternally, and every thread on my install islegacy. Ifcompactbounds the replayed chain, exposing it as a user setting would resolve this without users needing to know any of the above. Related: #34963.Additional confirmed Windows case, substantially larger than the original report.
Environment
OpenAI.Codex 26.721.11231.0019f9b07-ee1e-7260-9e8d-b999f154c2fbMeasured impact
type: "compacted"JSONL record occupied 50,342,723,847 bytes (46.89 GiB)payload.replacement_historywindow_number: 158648,175,342The original screenshots were only approximately 2.70 MB, 2.70 MB, and 3.11 MB. They were persisted inline as
data:image/png;base64,...and repeatedly retained across later compactions.A fixed-size analysis of the final 256 MiB of the compacted record found:
The final remote compaction attempt failed after approximately 564,671 ms (9m25s):
A later attempt was also aborted after approximately 6m18s. Deleting the affected archived rollout, after preserving only privacy-safe structural metrics, immediately recovered 47.15 GiB of disk space. The raw rollout was not uploaded because it contained private project screenshots and conversation data.
This Windows case confirms that the issue is cross-platform and that repeated inline image retention can grow beyond 10 GiB into a single ~47 GiB compacted checkpoint.
Requested safeguards:
replacement_historyimage bytes.I can provide the privacy-safe diagnostic summary or additional structural measurements if useful, but the original 47 GiB rollout has been removed to restore the machine.
Additional macOS Desktop corroboration from a privacy-preserving structural audit across the full local session store.
Environment and scope
No raw rollout, screenshot, prompt, response, task title, thread ID, tool content, username, or local path is included here.
Aggregate byte attribution for the 27 oversized rollouts
| Stored record category | Size | Share of oversized set |
|---|---:|---:|
|
compactedrecords | 19.339 GiB | 82.16% || function-call outputs | 0.929 GiB | 3.95% |
| message records | 0.843 GiB | 3.58% |
| MCP completion records | 0.780 GiB | 3.31% |
| image-generation completion records | 0.532 GiB | 2.26% |
| custom-tool outputs | 0.421 GiB | 1.79% |
All 27 oversized rollouts were dominated by
compactedrecords. Together they retained 1,918 compaction records.Distribution
The 27 largest files account for roughly 88% of the total session-store bytes, so this is highly concentrated rather than ordinary uniform transcript growth.
Interpretation
This broader sample supports the issue's on-disk root-cause hypothesis: repeated append-only persistence of large replacement-history snapshots is the primary storage amplifier. Original image and tool-result records contribute measurable bytes, but the
compactedsnapshots are much larger in aggregate, consistent with heavyweight payloads being carried forward through repeated compactions.The analysis did not measure app-server heap multiplication, so I cannot independently corroborate the issue's memory-footprint figures. It directly corroborates the disk amplification across many independent rollouts rather than one extreme thread.
Operational mitigation used locally
Pending an upstream fix, I am treating rollout size as a low-cost warning signal:
Here “retire” means preserve the existing task as history and stop appending new work; it does not mean manually rewrite or delete the JSONL.
I can provide the privacy-safe streaming analyzer and additional aggregate histograms if useful.
Confirming the same failure mode on Linux with stable
codex-cli 0.146.0, including a kernel OOM cascade.Environment
codex-cli 0.146.0installed via the global npm packagecodex -c features.code_mode_host=true app-server --listen unix://Measured impact
At the kernel OOM snapshot, the Codex process had:
oom_score_adj=-900The strong inherited OOM protection caused Linux to kill unrelated application processes repeatedly instead of the largest consumer. After reboot, a fresh app-server went from 750 MiB to 2.03 GiB RSS in 119 seconds and later showed a 6.13 GiB RSS high-water mark.
Privacy-safe rollout metrics
The affected rollout contains private project data, so I cannot attach it. Structural measurements:
type=compactedlines: 360replacement_history: 360data:image: 461The rollout was created on July 10 and was still being modified by the restarted 0.146.0 app-server during the incident.
Additional observation
The same app-server tree also retained duplicate MCP/helper sets (Playwright, Serena/TypeScript LS, Graphify, Context7, Vibe Kanban MCP, and dev-manager MCP), consistent with #25015. One affected session scope had roughly 250 tasks and no memory limit. However, the single Codex process RSS and the 5.464 GiB compacted rollout make this incident a close match for this issue.
I can provide exact kernel task-table rows, timestamped RSS/cgroup samples, and privacy-safe structural analysis output. I will not publish the raw rollout because it contains prompts, tool results, screenshots, local paths, and session identifiers.
Additional macOS reproduction: the oversized-rollout failure is occurring in the Computer Use
cua_nodeworker, not only inapp-server. This also appears related to #26738.Environment
codex-cli 0.146.0-alpha.3.1codex-cli 0.145.0Directly measured behavior
cua_node/bin/nodeworker sustained approximately 107-160% CPU. Its RSS reached roughly 9.7 GB, andsamplereported a 14.7 GB physical footprint.lsof -oshowed the worker holding that same rollout through two read descriptors (12 and 13), both positioned at byte 3,568,314,812, which was EOF at measurement time.codex resume -> node_repl -> codex sandbox -> cua_node/bin/nodeThis looks like the Computer Use runtime eagerly or repeatedly reading an oversized active rollout, amplifying it in memory, and spending sustained CPU on buffer copying and garbage collection. It produces severe laptop heat even when overall system CPU is not saturated.
Useful safeguards would include streaming/lazy rollout processing, avoiding duplicate full-file readers, bounding retained history, and enforcing a CPU/memory watchdog for Computer Use workers.
No raw rollout, session ID, prompts, screenshots, or private project paths are included here.
Confirming a more severe reproduction of this exact failure mode in Codex Desktop on macOS.
Environment
26.727.51351 (build 6119)Measured impact
codexprocess to reach 20.22 GB and continue rising until the machine became unresponsive. An earlier run failed withmach_vm_allocate_kernel failed.lsofconfirmed that Codex opened this exact 75 GB rollout when the task was selected.tokio::fs->serde_json->realloc/memmove.tokens_usedmetadata had reached roughly 6.6 billion.Recovery confirmation
After archiving the task, deleting only the oversized rollout, and restarting the Codex backend:
No raw rollout, session ID, prompts, screenshots, usernames, or private project paths are included here.
Requested safeguards
Please consider:
stat()the rollout before opening it and reject/quarantine implausibly large files.The preflight size check is especially important here: a 75 GB JSONL should never enter the normal eager-load path.
Confirmed on Codex CLI 0.146.0-alpha.9.2 / Desktop 26.727.51351. One rollout reached 29.609 GiB; 99.276% was repeated compacted history containing 24,656 embedded images. Archiving the thread caused Codex to open and parse the archived 31.8 GB rollout, making macOS unresponsive and requiring a reboot. Replacing embedded images with text placeholders reduced the same valid JSONL to 0.154 GiB.
The 50–79 MB
compactedrecords carrying inline PNG data make this a direct oversized-persisted-payload case. Codex Rescue’s reader is bounded: it can identify oversized JSONL records without decoding or retaining the image bytes, and it never rewrites the source rollout.It does not externalize images, shrink the 10.2 GB file, or repair the app-server memory amplification. I’m looking for real cases that confirm whether the local diagnostic and fail-closed handoff remain useful at this scale.
If the affected rollout or a backup still exists, would you try:
Sanitized counts/classification are sufficient. Please don’t share the raw rollout, image data, prompts, project files, databases, credentials, or private paths.
https://github.com/shleder/codex-rescue
Additional confirmed Windows reproduction on the current Codex Desktop build. This repeatedly interrupted active paid work.
Environment
OpenAI.Codex 26.810.4967.0151.0.7922.13726.803.10989.0/ executable151.0.7922.76Oversized active rollouts
Two still-needed working threads had grown to approximately:
Fixed-size 64 MiB tail samples were inspected without loading or publishing the raw sessions:
Recent screenshot results appeared in paired records such as
mcp_tool_call_endandcustom_tool_call_output, with nearly identical 0.8–0.94 MiB payload sizes, suggesting the same image-bearing result is persisted in more than one protocol representation.Measured runtime impact
codex.exewas observed at about 9.1 GiB working set/private bytes.thread/turns/listcalls for the 5.38 GiB thread took 31–33 seconds. The previous day they took 16–22 seconds.host did not respond to sync request.Failure mode
Windows recorded three Application Hang events (Event ID 1002) on August 14–15. The most recent report classified the hang as:
HangType=Top level window is idleThe visible symptom is that the entire UI disappears and only the Codex logo remains. At the same timestamp, Desktop logs report:
killedkilledThis reproduces after the latest Store update, so the current build has not resolved it.
Privacy
The raw rollouts contain private screenshots, prompts, tool results, local paths, and credentials/configuration context. I am not uploading them publicly. Sanitized structural counts, timing data, and Windows event metadata can be provided through a private OpenAI support channel if needed.
Requested safeguards
thread/turns/list.