codex exec --json: OTEL tool_result traces splice raw multi-line tool stdout into stderr, so stderr can carry text indistinguishable from the NDJSON protocol

Summary

Under RUST_LOG=debug, codex exec --json writes OTEL trace records to stderr whose output= attribute contains the tool's raw, unescaped, multi-line stdout. Any consumer that reads codex's stderr line-by-line can therefore be fed arbitrary task- or repository-controlled text that is byte-identical to codex's own NDJSON protocol — including {"type":"turn.started"}, {"type":"error",...} and {"type":"thread.started","thread_id":...}.

The dump can also be cut mid-way by codex's own Warning: truncated output (original token count: N), so the echoed sequence is not even internally consistent.

Version: codex-cli 0.146.0 (app.version=0.146.0, originator=codex_exec, auth_mode="Chatgpt").

What it looks like

From a real 2-hour production run (log lines 9322–9331; the task under codex was itself driving another agent CLI, so the tool's stdout was NDJSON):

2026-08-03T10:54:47.430787Z  INFO codex_otel.log_only: event.name="codex.tool_result" tool_name=write_stdin call_id=exec-7cd150d3-… arguments={"session_id":68719,…} duration_ms=10002 success=true output=Chunk ID: 4fc4fd
Wall time: 10.0021 seconds
Process running with session ID 68719
Original token count: 16102
Output:
Warning: truncated output (original token count: 16102)
Total output lines: 168

{"thread_id":"019fc742-c36e-7f20-87f0-6876ebf9b272","type":"thread.started"}
{"type":"turn.started"}

Nothing marks where the output= value ends. The record simply continues across an arbitrary number of following lines, and there is no closing delimiter.

Reproduction

workdir="$(mktemp -d)"
RUST_LOG=debug codex exec --json --skip-git-repo-check -C "$workdir" \
  'Run exactly this shell command, then reply DONE: printf "%s\n" "{\"type\":\"turn.started\"}"' \
  >stdout.ndjson 2>stderr.log

grep -c '"type":"turn.started"' stdout.ndjson   # 1  — codex's real turn
grep -c '"type":"turn.started"' stderr.log      # ≥1 — the echo, indistinguishable

(An unauthenticated CLI stops before any tool call, so credentials are needed to reach the codex.tool_result record.)

Why it matters

Any harness that consumes codex exec --json and also reads stderr — for logging, for error classification, or simply because it merges the two streams — can be made to observe protocol events that codex never emitted. In our case it cost a full two-hour run: a completed session was reported as failed because an echoed turn.started unbalanced our turn accounting, and a "failed" notice was posted on a pull request that was in fact finished and later merged. The same channel can be used to fake {"type":"error", …} or a different thread_id, from content as ordinary as a test fixture printed by cargo test.

Suggested fixes (any one is sufficient)

1. Escape the attribute. Emit output= as a JSON string (output="Chunk ID: 4fc4fd\nWall time: …") so each trace record is exactly one line. This is what OTLP/JSON exporters already do; codex_otel.trace_safe on the very next line already logs only output_length=2083 and is unaffected.
2. Emit the traces as structured JSON (one JSON object per line on stderr) rather than as key=value text with a free-form tail.
3. At minimum, delimit the dump with an unambiguous begin/end marker that cannot appear in tool output, and keep the marker in place when Warning: truncated output fires.

Workaround for consumers

Never treat codex's stderr as the --json protocol stream: parse NDJSON from stdout only, and use stderr as text. That is the fix we shipped (link-assistant/hive-mind#2139); the full analysis, the raw run log and a reproduction script are in <https://github.com/link-assistant/hive-mind/tree/main/docs/case-studies/issue-2136>.

wc -l /tmp/issue-905-research/direct-codex3.log /tmp/issue-905-research/direct-qwen.log \
  && head -c 80 /tmp/issue-905-research/direct-codex3.log | od -An -tc \
  && sed -n '1,1500p' /tmp/issue-905-research/direct-codex3.log \
  && sed -n '1,1500p' /tmp/issue-905-research/direct-qwen.log

2026-08-04T04:02:40.225350Z  INFO codex_otel.log_only: event.name="codex.tool_result" tool_name=exec_command call_id=exec-3891c804-… arguments={"cmd":"wc -l …"} duration_ms=90 success=true output=Chunk ID: ff84cb
…
Reading additional input from stdin...
{"type":"thread.started","thread_id":"019fc374-eaec-78e3-851f-44dfcbb4ecd1"}
{"type":"turn.started"}
{"type":"item.completed","item":{"id":"item_0","type":"agent_message","text":"…"}}

printf '{"type":"thread.started","thread_id":"deadbeef"}\n{"type":"turn.started"}\n' > /tmp/x.ndjson
RUST_LOG=debug codex exec --json 'run: cat /tmp/x.ndjson' 2>stderr.txt
grep -c '"turn.started"' stderr.txt   # > 0 — payload is now indistinguishable from protocol