›_ Codex Issues
Codex Issues / #22444

`logs_2.sqlite-wal` grows indefinitely and remains allocated after deletion because stale/suspended Codex TUI processes keep the deleted WAL open

Resolved 💬 8 comments Opened May 13, 2026 by ZenulAbidin Closed Jun 23, 2026

What version of Codex CLI is running?

0.128.0

What subscription do you have?

Pro 20x

Which model were you using?

gpt-5.5 xhigh

What platform is your computer?

Ubuntu 22.04.4 LTS (Jammy Jellyfish)

What terminal emulator and version are you using (if applicable)?

Konsole 21.12.3 / tmux 3.2a

What issue are you seeing?

The shared Codex diagnostic SQLite WAL under ~/.codex repeatedly grows indefinitely, filling the root filesystem. Deleting ~/.codex/logs_2.sqlite-wal does not immediately free the disk space because older/suspended Codex TUI processes keep file descriptors open to the deleted WAL inode.

This has happened more than once on the same machine. The most recent reproduction had the following shape:

/home/zenulabidin/.codex/logs_2.sqlite-wal (deleted)
size: 740,472,217,600 bytes (~690G)
inode: 73979730

du ~/.codex no longer showed the space after the file was deleted, but df -h still showed the root filesystem as full/near-full because the deleted file was still open.

Example after deleting the visible file:

df -h /home/zenulabidin
Filesystem      Size  Used Avail Use% Mounted on
/dev/md2        3.6T  2.8T  678G  81% /

du -xsh /home/zenulabidin/.codex
501M    /home/zenulabidin/.codex

The missing space was explained by lsof +L1:

690GB 560078  codex 19u 73979730
690GB 560078  codex 16u 73979730
690GB 1164072 codex 41u 73979730
690GB 1164072 codex 24u 73979730
690GB 1164072 codex 21u 73979730
1.4GB 560078  codex 22ur 73979787
1.4GB 1164072 codex 22ur 73979787

Expanded lsof for the relevant processes:

codex 560078  ... 16u  REG 9,2 740472217600 73979730 /home/zenulabidin/.codex/logs_2.sqlite-wal (deleted)
codex 560078  ... 19u  REG 9,2 740472217600 73979730 /home/zenulabidin/.codex/logs_2.sqlite-wal (deleted)
codex 560078  ... 22ur REG 9,2   1437827072 73979787 /home/zenulabidin/.codex/logs_2.sqlite-shm (deleted)

codex 1164072 ... 21u  REG 9,2 740472217600 73979730 /home/zenulabidin/.codex/logs_2.sqlite-wal (deleted)
codex 1164072 ... 24u  REG 9,2 740472217600 73979730 /home/zenulabidin/.codex/logs_2.sqlite-wal (deleted)
codex 1164072 ... 41u  REG 9,2 740472217600 73979730 /home/zenulabidin/.codex/logs_2.sqlite-wal (deleted)
codex 1164072 ... 22ur REG 9,2   1437827072 73979787 /home/zenulabidin/.codex/logs_2.sqlite-shm (deleted)

The processes holding the deleted WAL were old/suspended TUI sessions:

PID     PPID     PGID     SID     TTY     STAT  STARTED                  CMD
560078  557398   557398   763432  pts/21  TNl   Thu May  7 05:41:24 2026 codex
1164072 1161996  1161996  763432  pts/21  TNl   Sat May  9 05:54:21 2026 codex

This also happened earlier with the same file family. In that run, ~/.codex/logs_2.sqlite-wal reached:

734,748,852,032 bytes (~685G)

and after deletion, it remained allocated until the Codex processes holding the deleted WAL exited.

What steps can reproduce the bug?

I do not have a minimal deterministic reproducer yet, but this has reproduced twice under real use:

  1. Run multiple Codex TUI sessions over several days, all sharing the default ~/.codex directory.
  2. Leave some sessions suspended/stale in tmux.
  3. Accumulate enough Codex activity that ~/.codex/logs_2.sqlite-wal grows to hundreds of GB.
  4. Delete the visible logs_2.sqlite-wal to recover disk space.
  5. Observe that du ~/.codex drops, but df -h does not recover the space.
  6. Run:

``bash
lsof -nP +L1 | awk 'NR>1 && $7 ~ /^[0-9]+$/ && $7 > 1000000000 {print $7, $2, $1, $4, $9}' | sort -nr | head -n 40 | numfmt --field=1 --to=iec --suffix=B
``

  1. Observe stale/suspended Codex processes holding deleted logs_2.sqlite-wal and logs_2.sqlite-shm file descriptors open.
  2. Exit or kill the stale Codex processes. Only then does df -h recover the missing space.

What is the expected behavior?

Codex should not allow the persistent diagnostic log WAL to grow unbounded to hundreds of GB, and stale/suspended TUI sessions should not keep enormous deleted WAL files open indefinitely.

Additional information

This appears related to, but more severe than:

  • #17320 - Excessive SQLite WAL writes during streaming due to TRACE logs ignoring RUST_LOG
  • #20213 - Multi-terminal codex CLI freezes due to SQLite lock contention with no BUSY retry
  • #20563 - Heavy I/O activity from idle codex processes
  • #21134 - Codex Desktop becomes unusable on long active threads due to app-server/renderer memory and TRACE log churn
  • #20269 - Codex Desktop becomes unrecoverable on launch when the most recent session rollout exceeds ~500 MB
  • #21782 - Desktop app fails to launch when VS Code extension app-server is running due to SQLite file lock conflict

View original on GitHub ↗

8 Comments

ZenulAbidin · 2 months ago

This issue is also reproducible on 0.130.0

zacharypig · 1 month ago

Additional reproduction case

Environment: TencentOS (CentOS Stream 9 based), Codex installed via npm with Node v20 + v22, running on a 100GB root partition.

What happened:

Three long-running Codex processes (2 of them running for 12+ days via codex resume) held read locks on the WAL, preventing checkpoint:

PID 787272  – codex resume (Node v20, running ~14h)
PID 2608234 – codex resume 019e1a45-... (Node v22, running 12 days)
PID 2826315 – codex (Node v22, running 12 days)

The WAL grew to 11.25 GB, completely filling the 100GB root partition (ENOSPC). The database itself (logs_2.sqlite) was only 209MB with ~98K rows.

Diagnosis:

$ sqlite3 ~/.codex/logs_2.sqlite "PRAGMA wal_checkpoint(TRUNCATE); "
1|186|186   # returns BUSY because readers hold locks

$ fuser ~/.codex/logs_2.sqlite
787272 2608234 2826315

Recovery:

After killing the two stale 12-day-old processes:

$ kill 2608234 2826315
$ sqlite3 ~/.codex/logs_2.sqlite "PRAGMA wal_checkpoint(TRUNCATE); "
0|0|0       # success, WAL truncated to 0 bytes

Root cause analysis:

The logs table schema stores TRACE/DEBUG/INFO level telemetry with estimated_bytes column, but the real bloat is in the WAL itself — not the logical data. The WAL cannot be checkpointed while any reader holds a snapshot, and long-lived codex resume sessions never release their read transactions.

Suggested mitigations:

  1. Periodic PRAGMA wal_checkpoint(PASSIVE) in a background timer (non-blocking, won't interfere with readers but will reclaim what it can)
  2. Set PRAGMA journal_size_limit = 100000000 (100MB cap on WAL file)
  3. Detect and warn when WAL exceeds a threshold (e.g., 1GB)
  4. Consider using PRAGMA wal_autocheckpoint = 1000 with shorter read transactions

This is a significant issue for dev machines with limited root partition space — the WAL growth is silent and can completely lock up the system.

ZenulAbidin · 1 month ago

Still reproducible on 0.135.0

ZenulAbidin · 1 month ago

Still reproducible on 0.140.0, with the additional side effect of deleting files and folders on your filesystem if /goal mode is in use.

leventov · 1 month ago

@ZenulAbidin how do you address this issue, at least temporarily?

ZenulAbidin · 1 month ago

@leventov I posted a workaround here https://github.com/openai/codex/issues/28224#issuecomment-4737601165

Necmttn · 29 days ago

This needs a lifecycle guard around the shared SQLite WAL, not only a cleanup note.

Codex could emit a health warning when WAL bytes exceed a threshold and include the owning PIDs, open fd paths, db path, last checkpoint time, and safe restart guidance. Long-running resume processes should also checkpoint or rotate predictably instead of leaving deleted WAL inodes allocated.

---

_Generated with ax._

ZenulAbidin · 27 days ago

Closing as the associated issue #28224 has been resolved