›_ Codex Issues
Codex Issues / #20563

Heavy I/O activity from idle `codex` processes.

Open 💬 9 comments Opened May 1, 2026 by emcd
💡 Likely answer: A maintainer (zanie-oai, contributor) responded on this thread — see the highlighted reply below.

What version of Codex CLI is running?

0.128.0

What subscription do you have?

Plus

Which model were you using?

gpt-5.5

What platform is your computer?

Linux 6.17.0-1014-nvidia aarch64 aarch64

What terminal emulator and version are you using (if applicable)?

_No response_

What issue are you seeing?

Completely idle codex processes dominating I/O on the host.

See attached screenshot, for example:

<img width="3732" height="450" alt="Image" src="https://github.com/user-attachments/assets/b2faf352-1ea5-4872-9a3d-1a6e024067bf" />

---

During diagnosis of the issue, GPT 5.5 noticed that all of the processes are writing a lot to a codex-tui.log file, but even its large size does not account for the amount of I/O. Does not seem to be network-related however. Diagnostics included in "Additional information" section below.

What steps can reproduce the bug?

Observe process I/O with htop or similar tools.

What is the expected behavior?

Idle codex processes should not be bombarding the system with writes at 100-200 KB/s.

Additional information

Diagnostics from GPT 5.5 via permissions escalation to me for calls needing special access to /proc or Linux capabilities not granted by the sandbox:

---

Codex CLI Idle Disk I/O Diagnostics

Additional Information

Observed on Codex CLI 0.128.0 on Linux/aarch64:

  • Package: @openai/codex@0.128.0
  • Native binary: @openai/codex-linux-arm64, aarch64-unknown-linux-musl
  • Node: v24.14.1
  • npm: 11.11.0
  • OS: Ubuntu/Linux 6.17.0-1014-nvidia, aarch64
  • Filesystem: ext4

I had seven long-running Codex TUI sessions open in a tmux-managed
infrastructure group. They were visually idle at prompts, with no active user
interaction and no known unfinished tasks. htop showed high disk write
activity from all Codex processes, so I gathered non-interactive diagnostics.

Per-process Disk I/O

Using pidstat -d -p <codex-pids> 1 10, idle Codex processes showed recurring
write activity, typically hundreds of KB/s per process. One sample average over
10 seconds:

PID 1066292: 149.20 kB_wr/s
PID 1066351: 221.20 kB_wr/s
PID 1066373: 144.80 kB_wr/s
PID 1066443: 150.00 kB_wr/s
PID 1066502: 146.40 kB_wr/s
PID 1066533: 1077.60 kB_wr/s
PID 1066534: 144.40 kB_wr/s

A direct /proc/<pid>/io before/after sample over 10 seconds showed idle
sibling processes writing roughly 1.4-3.2 MB each:

PID 1066292: +1,650,688 write_bytes
PID 1066351: +2,977,792 write_bytes
PID 1066373: +1,474,560 write_bytes
PID 1066443: +3,162,112 write_bytes
PID 1066502: +1,605,632 write_bytes
PID 1066534: +1,531,904 write_bytes

The currently active diagnostic session wrote more, as expected, but the
important observation is that the otherwise idle sessions also showed sustained
write activity.

Files Open by Idle Codex Processes

lsof and /proc/<pid>/fd showed each Codex process with open handles to
shared Codex state/log databases and WAL files, including:

~/.config/codex/state_5.sqlite
~/.config/codex/state_5.sqlite-wal
~/.config/codex/state_5.sqlite-shm
~/.config/codex/logs_2.sqlite
~/.config/codex/logs_2.sqlite-wal
~/.config/codex/logs_2.sqlite-shm
~/.config/codex/log/codex-tui.log
~/.config/codex/sessions/.../rollout-<session-id>.jsonl

Current file sizes during diagnosis:

codex-tui.log:        ~2.89 GB
logs_2.sqlite:        ~168.6 MB
logs_2.sqlite-wal:      ~7.3 MB
state_5.sqlite:        ~307 KB
state_5.sqlite-wal:     ~4.1 MB

File Growth Versus Write Counters

A 15-second file-size sample showed that idle session transcript files were not
growing:

Idle session JSONL files: no growth
state_5.sqlite-wal: no size growth
logs_2.sqlite-wal: no size growth
codex-tui.log: +83,379 bytes
active session JSONL: +2,334 bytes

This suggests much of the idle write activity is likely rewriting/flushing
existing SQLite/WAL pages or shared state/log database files rather than
appending to the per-session JSONL transcript.

Network Observations

lsof -iTCP showed established HTTPS connections for most Codex processes to
172.64.155.209:443.

However, at least one idle process in the sample did not show an established TCP
connection at that moment and still wrote about 2.98 MB over 10 seconds. So
the idle disk write activity does not appear to require an active model response
stream.

TUI Log Verbosity During Active Turns

During active work, codex-tui.log appears extremely verbose. The last 5000
lines contained:

websocket_event: 4393
log_only:        2249
trace_safe:      2235

Most common websocket event kinds:

2772 response.function_call_arguments.delta
1191 response.output_text.delta
76   response.output_item.done
74   response.output_item.added
52   response.completed

The log appears to record both codex_otel.log_only and
codex_otel.trace_safe entries for many websocket delta events. The log_only
entries also included account metadata such as account id and email address, so
raw log excerpts require redaction before sharing.

Working Hypothesis

There may be two related sources of disk I/O:

  1. Idle Codex TUI processes appear to perform periodic local persistence or

telemetry/state writes, likely involving the shared SQLite databases and WAL
files under ~/.config/codex.

  1. Active turns additionally generate very large codex-tui.log output due to

per-delta websocket telemetry logging.

The idle case is the surprising part: multiple Codex TUI sessions sitting at
prompts still produced sustained disk writes even though their per-session JSONL
transcript files did not grow.

View original on GitHub ↗

9 Comments

emcd · 2 months ago

Additional datapoint confirming the write hotspot.

I captured a syscall trace of an idle Codex process for ~28.3s using:

sudo strace -f -tt -yy -s 0 \
  -e trace=write,pwrite64,fdatasync,fsync,openat,close \
  -p <codex_pid> -o <trace_output>

In this sample (1052 syscall lines), the dominant writes were to:

  • ~/.config/codex/logs_2.sqlite-wal (by far the largest)
  • plus smaller writes to /var/tmp/etilqs_* temp files (likely SQLite temp/journal spill)

Quick aggregate from this trace window:

  • ~/.config/codex/logs_2.sqlite-wal: 1,664,480 bytes written
  • /var/tmp/etilqs_*: several files in ~70–98 KB range each
  • codex-tui.log: not the dominant writer in this sample

I also did not observe fsync/fdatasync calls in this short window; most activity was repeated pwrite64(...) to the WAL file in 4 KB + 24-byte patterns.

So this appears to be primarily SQLite WAL churn (logs_2.sqlite-wal) during idle, not just plain codex-tui.log append behavior.

---

Co-Authored-By: GPT-5.5 <noreply@openai.com>

clemisch · 2 months ago

I am also seeing this. I am not familiar with sqlite; Are we 100% sure this IO really leads to physical disk writes? Or is it an artefact of Codex holding the database filehandle open but the "writes" are actually transient?

I really hope it's the latter, because I tend to have the Codex window open all day (often idle) and it keeps hammering my SSD with (alledged) 5 MB/s constantly.

emcd · 2 months ago
I am also seeing this. I am not familiar with sqlite; Are we 100% sure this IO really leads to physical disk writes? Or is it an artefact of Codex holding the database filehandle open but the "writes" are actually transient?

@clemisch : Fair question. Here is what iostat reports for a quiet system (no codex processes) over a 30-second interval:

$ iostat -d -x -y -z -o JSON /dev/nvme0n1p2 1 30 | jq '.sysstat.hosts[0].statistics[].disk[0]."wkB/s"' | grep -v null
16.00
28.00
8.00
64.00
12.00

And here is what it reports for a system with 1 codex process running over another 30-second interval:

$ iostat -d -x -y -z -o JSON /dev/nvme0n1p2 1 30 | jq '.sysstat.hosts[0].statistics[].disk[0]."wkB/s"' | grep -v null
4.00
16.00
56.00
4.00
40.00
40.00
3568.00
72.00
2036.00
108.00

Sampling points with 0 KB/s in writes are excluded in both cases. The numbers do not line exactly with the process write counters, because the sampling frequency is not perfectly aligned with the Sqlite write frequency. But, I think it is quite clear that these are physical writes caused by Codex CLI. (I have repeated this sampling several times and the results are consistently like the above.)

danielkec · 1 month ago

Adding another reproduction from Linux/x86_64. This is still present on codex-cli 0.133.0.

Environment:

  • Codex CLI: codex-cli 0.133.0
  • OS: Ubuntu 26.04 LTS
  • Kernel: Linux 7.0.0-15-generic x86_64
  • Scenario: multiple long-lived Codex TUI sessions open in terminal tabs, several visually idle

What I saw:

  • Nine long-lived codex children were running, started between about 10:53 and 12:06 local time.
  • A 5-second pidstat -d sample showed sustained writes from most of them:
PID 15671:  1066.67 KiB/s
PID 17638:  1139.32 KiB/s
PID 32699:   215.57 KiB/s
PID 33625:  1475.45 KiB/s
PID 34440:   261.08 KiB/s
PID 39499:   931.74 KiB/s
PID 112142:  818.36 KiB/s
PID 113286: 2380.84 KiB/s
PID 114289: 1112.18 KiB/s

All of those samples had iodelay 0, so they were producing writes but were not visibly blocked on disk at that moment.

Some of the "idle-looking" sessions were not actually idle from the OS perspective: I saw systemd-inhibit --why Codex is running an active turn under a couple of the long-lived codex children. That explains part of the write rate, but not the whole user-visible problem: from the terminal, it is not obvious which tabs are still active/background-running, and several children still had steady writes.

Open FD evidence:

Each long-running Rust codex child had open handles to shared Codex state/log files such as:

~/.codex/logs_2.sqlite
~/.codex/logs_2.sqlite-wal
~/.codex/logs_2.sqlite-shm
~/.codex/state_5.sqlite
~/.codex/state_5.sqlite-wal
~/.codex/state_5.sqlite-shm
~/.codex/goals_1.sqlite
~/.codex/log/codex-tui.log
~/.codex/sessions/2026/05/26/rollout-*.jsonl

Some processes held more than one rollout JSONL file, which looked like background/sub-agent/nested transcript state rather than just one foreground prompt.

Relevant file sizes during the sample:

~/.codex/logs_2.sqlite       ~3.7G
~/.codex/logs_2.sqlite-wal   ~245M
~/.codex/log/codex-tui.log   ~846M
~/.codex/state_5.sqlite-wal  ~4.2M

One new clue: the TUI log was dominated by repeated plugin/skill loader warnings. In the last 200 lines of ~/.codex/log/codex-tui.log, 120 lines were this warning family:

codex_core_skills::loader: ignoring interface.icon_small: icon path must not contain '..'
codex_core_skills::loader: ignoring interface.icon_large: icon path must not contain '..'

The warning does not include the offending manifest path, but a local search found many agents/openai.yaml files under ~/.codex/.tmp/plugins using paths like ../../assets/logo.svg and ../teams/assets/teams.svg. If the loader is scanning temporary/plugin-cache metadata, this warning spam could be a major write amplifier. At minimum, the warning should probably be deduplicated and include the manifest path so users can identify the source.

Expected behavior:

  • A TUI session sitting at the prompt should not sustain hundreds of KiB/s to MiB/s of writes.
  • If a session is still running an active/background turn, the TUI should make that obvious enough that it does not look idle.
  • Repeated skill/plugin metadata warnings should be emitted once per offending manifest, not repeatedly during turns.
  • Shared SQLite log/state WALs should not be churned by idle sessions at this rate.

Possible related issues from the current search:

  • #22444, for unbounded logs_2.sqlite-wal growth / stale sessions keeping WAL FDs open.
  • #22411, for background app-server/session-list loading and state/log writes.

This comment is meant as additional evidence for the idle TUI process I/O side of the problem.

emcd · 1 month ago

Maybe someone is paying attention to this issue or a closely adjacent one: https://github.com/openai/codex/pull/26396

As long as you are looking into unnecessary Sqlite writes, @zanie-oai , maybe you want to look at this issue.

zanie-oai contributor · 1 month ago

It seems likely that my change should help with this issue.

emcd · 1 month ago
It seems likely that my change should help with this issue.

Excellent. I had some trepidation about OpenAI acquiring Astral, but this alleviates that some. Hopefully Codex isn't keeping you too much from all the other good work you do though.

zanie-oai contributor · 1 month ago

I'm excited to fix issues in the tools we use every day to work on the Astral tools :)

0xdevalias · 27 days ago

Potentially a duplicate of / related to:

Which should have been fixed by:

Which were released in:

But there still seem to be issues after that: