[Linux][CLI 0.147.0] Wi-Fi switch: HTTP 101 reconnects followed by incremental state mismatch and repeated EPIPE

Open 💬 3 comments Opened Aug 11, 2026 by YHDiamond
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

What version of Codex CLI is running?

codex-cli 0.147.0 (standalone Linux installation)

What subscription do you have?

ChatGPT authentication. The exact subscription tier is not exposed by codex login status, codex doctor, or the stored token claims inspected locally.

Which model were you using?

The incident reproduced with both:

  • gpt-5.6-luna, reasoning effort low
  • gpt-5.6-sol, reasoning effort xhigh

What platform is your computer?

Linux 7.0.0-28-generic x86_64 x86_64

Linux Mint 22.3 (Cinnamon 6.6.9, X11)

What terminal emulator and version are you using (if applicable)?

GNOME Terminal 3.52.0 / VTE 0.76.0. No terminal multiplexer was involved.

Codex doctor report

The original codex doctor --json output was reviewed locally. Local usernames and absolute paths are omitted below. All installation, auth, config, state-database, provider-reachability, and WebSocket checks passed. A later update-version probe independently encountered a 5-second DNS-resolution timeout while the WebSocket and ChatGPT reachability checks in the same run passed.

{
  "schemaVersion": 1,
  "generatedAt": "2026-08-11T10:44:34+03:00",
  "overallStatus": "warning",
  "codexVersion": "0.147.0",
  "checks": {
    "auth.credentials": {
      "status": "ok",
      "summary": "auth is configured",
      "details": {
        "auth storage mode": "File",
        "stored API key": "false",
        "stored ChatGPT tokens": "true",
        "stored auth mode": "chatgpt"
      }
    },
    "config.load": {
      "status": "ok",
      "summary": "config loaded",
      "details": {
        "config.toml parse": "ok",
        "model": "gpt-5.6-sol",
        "model provider": "openai",
        "feature flag overrides": "none"
      }
    },
    "installation": {
      "status": "ok",
      "summary": "installation looks consistent",
      "details": {
        "install method": "standalone",
        "platform": "linux-x86_64",
        "version": "0.147.0"
      }
    },
    "network.env": {
      "status": "ok",
      "summary": "network-related environment looks readable",
      "details": {
        "proxy env vars": "none"
      }
    },
    "network.provider_reachability": {
      "status": "ok",
      "summary": "active provider endpoints are reachable over HTTP",
      "details": {
        "ChatGPT base URL": "https://chatgpt.com/backend-api/ reachable (HTTP 404)",
        "reachability mode": "ChatGPT auth"
      }
    },
    "network.websocket_reachability": {
      "status": "ok",
      "summary": "Responses WebSocket handshake succeeded",
      "details": {
        "DNS": "2 IPv4, 2 IPv6, first IPv6",
        "auth mode": "chatgpt",
        "connect timeout": "15000 ms",
        "endpoint": "wss://chatgpt.com/backend-api/<redacted>",
        "handshake result": "HTTP 101 Switching Protocols",
        "model provider": "openai",
        "provider name": "OpenAI",
        "proxy env vars": "none",
        "supports websockets": "true",
        "wire API": "responses"
      }
    },
    "state.paths": {
      "status": "ok",
      "summary": "state paths and databases are inspectable",
      "details": {
        "log DB integrity": "ok",
        "state DB integrity": "ok",
        "thread history DB integrity": "ok",
        "queue DB integrity": "ok"
      }
    },
    "state.rollout_db_parity": {
      "status": "ok",
      "summary": "rollout files and state DB thread inventory agree",
      "details": {
        "rollout DB scan errors": "0",
        "rollout DB stale rows": "0",
        "rollout DB duplicate rollout thread ids": "0"
      }
    },
    "updates.status": {
      "status": "warning",
      "summary": "update configuration is locally consistent",
      "details": {
        "cached latest version": "0.147.0",
        "last successful check": "2026-08-11T07:14:04Z",
        "latest version probe": "curl: (28) Resolving timed out after 5001 milliseconds"
      }
    }
  }
}

An earlier codex doctor run during this investigation completed with 17 ok, 1 idle, 0 warn, and 0 fail, including successful HTTP and WebSocket reachability. No config or auth changes were made between the two runs.

What issue are you seeing?

After changing between Wi-Fi networks that remained usable for normal browsing, existing/forked Codex CLI sessions repeatedly failed to complete turns with:

Stream disconnected before completion: failed to send websocket request: IO error: Broken pipe (os error 32)

This was not a simple inability to connect:

  • Every recorded retry established a new Responses WebSocket successfully (HTTP 101 Switching Protocols).
  • On a representative turn, Codex then received codex.response.metadata, response.in_progress, and responsesapi.websocket_timing events.
  • Immediately before two independent failure clusters, codex_core::client logged:

``text
incremental request failed, items didn't match
``

  • Codex subsequently retried, but each newly established WebSocket failed about 34-35 seconds later with Broken pipe (os error 32).
  • The retry loop did not produce a completed assistant turn. Three affected turns were eventually interrupted manually.

Seven genuine EPIPE retries were recorded in a 9-minute interval on three turns and two Codex processes:

2026-08-10 18:30:18 +03:00  gpt-5.6-luna low    retry 1/5
2026-08-10 18:30:53 +03:00  gpt-5.6-luna low    retry 2/5
2026-08-10 18:36:31 +03:00  gpt-5.6-sol  xhigh  retry 1/5
2026-08-10 18:37:05 +03:00  gpt-5.6-sol  xhigh  retry 2/5
2026-08-10 18:38:24 +03:00  gpt-5.6-sol  xhigh  retry 1/5
2026-08-10 18:38:59 +03:00  gpt-5.6-sol  xhigh  retry 2/5
2026-08-10 18:39:34 +03:00  gpt-5.6-sol  xhigh  retry 3/5

The WebSocket handshakes around these failures were routed through both FRA and LHR Cloudflare edges, so the behavior was not confined to one edge. The affected machine also had ordinary browser traffic working during the incident.

What steps can reproduce the bug?

The incident was reconstructed from Codex's SQLite log database, rollout event files, NetworkManager, and systemd-resolved journals. It is not currently reproducing on demand, but the observed sequence was:

  1. Keep an existing local Codex CLI session open on Wi-Fi network A.
  2. Change Wi-Fi/network path while Codex sessions remain open. Both network A and network B remain functional for ordinary browsing.
  3. Submit a turn in an existing or forked Codex thread.
  4. Observe a successful WebSocket 101 handshake and, on at least one turn, response.in_progress.
  5. Observe incremental request failed, items didn't match in the trace.
  6. Observe repeated EPIPE failures despite reconnecting successfully before each retry.
  7. Interrupt the turn after it fails to recover.

The three affected turns lasted approximately 5m50s, 1m23s, and 5m57s before manual interruption.

Relevant network journal events occurred during the same interval:

18:34:20 NetworkManager: existing Wi-Fi deactivated for a new activation request
18:34:25 NetworkManager: replacement Wi-Fi activated with IPv4/IPv6
18:34:27 NetworkManager: CONNECTED_GLOBAL
18:39:54 NetworkManager: Wi-Fi deactivated for another activation request
18:39:55 NetworkManager: replacement Wi-Fi activated; CONNECTED_GLOBAL
18:39:56 systemd-resolved: DNS server downgraded from UDP+EDNS0 to UDP

Additional controls and observations:

  • No HTTP_PROXY, HTTPS_PROXY, ALL_PROXY, or related proxy variables were present.
  • ChatGPT authentication remained valid.
  • Codex installation and all local state databases passed integrity checks.
  • A live codex doctor WebSocket probe currently succeeds.
  • Firefox retained some TCP sockets associated with the prior private address while continuing to browse over fresh sockets on the new address. This explains why browsing could remain apparently healthy while a stateful client still had transition-related socket/state problems.
  • No genuine EPIPE retry was recorded on the following day after the incident window.

What is the expected behavior?

After a network transition closes or invalidates a persistent Responses WebSocket, Codex should:

  1. promptly discard the unusable socket;
  2. establish a new WebSocket;
  3. recover continuation state when valid; or
  4. if connection-local incremental state is unavailable/mismatched, start a new response chain and resend the full/compacted context;
  5. complete the turn or surface one clear, actionable terminal error.

It should not establish repeated successful WebSockets and then loop on EPIPE while the turn remains incomplete.

Additional information

The Responses WebSocket documentation states that continuation state is connection-local and that clients must reconnect and either continue with a valid persisted response or start a new chain with full/compacted input when the previous state cannot be continued:

https://developers.openai.com/api/docs/guides/websocket-mode#reconnect-and-recover

This makes the combination of incremental request failed, items didn't match followed by repeated EPIPE particularly relevant to the recovery path.

Historical context from the local log database (August 2-10, 2026): 43 codex_core::responses_retry warnings were recorded across several sessions:

15  failed to send websocket request: Connection closed normally
 8  WebSocket protocol error: Connection reset without closing handshake
 7  failed to send websocket request: IO error: Broken pipe (os error 32)
 6  idle timeout waiting for websocket
 2  IO error: Connection reset by peer (os error 104)
 2  request timed out
 2  Transport error: timeout
 1  Responses websocket connection limit reached (60 minutes)

Potentially related but not identical:

  • #17003: dead WebSocket detection after network changes; includes Linux stale-source-address evidence.
  • #30777: app/CLI reconnect trouble after Wi-Fi switching, possible DNS/Tailscale component.
  • #37661: CLI 0.147.0 interactive WebSocket/HTTPS failures despite a clean doctor report.
  • #37894: current-build Broken pipe leaves a Desktop task stuck.

Exact thread IDs, turn IDs, process UUIDs, rollout locations, and Cloudflare Ray IDs were retained locally in a support-only evidence file and can be supplied privately to an OpenAI maintainer. Prompts, tool outputs, cookies, credentials, SSIDs, local project names, and full transcripts are intentionally excluded from this public issue.

View original on GitHub ↗

3 Comments

github-actions[bot] contributor · 17 days ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #37894

Powered by Codex Action

YHDiamond · 17 days ago

Private diagnostic uploads completed through the Codex CLI /feedback flow for both affected session rollouts, including logs and doctor reports.

  • Parent/original affected thread: 019fec4b-6b7c-7ff3-93ac-b985841b0d82
  • Fork/representative affected thread: 019fec52-9a45-7e03-a3c0-5c30c0e7e346

The parent contains the first incremental request failed, items didn't match trace and four EPIPE retries across two turns. The fork contains the representative turn that received response.in_progress, then logged the same incremental mismatch, followed by three EPIPE retries.

jdcodes1 · 9 days ago

One clarification from the code that sharpens your analysis: incremental request failed, items didn't match is not itself an error — it's the trace marker for the incremental-reuse check declining, after which the client falls back to sending the full request payload over the (re)connected WebSocket (get_incremental_items, https://github.com/openai/codex/blob/1f41cc5d92/codex-rs/core/src/client.rs#L1221-L1259). After a network switch that's expected: the interrupted stream means last_response state diverged from what the server actually delivered, so the strict prefix comparison can't hold.

The consequential part is what happens next: every retry now pushes the entire context as one large send on a freshly switched network path, and each attempt dies mid-send with Broken pipe at a suspiciously consistent ~34–35 s — a half-open/LB-killed connection discovered only at write time. The retry loop keeps rebuilding the WS and repeating the identical large send, and never degrades to the HTTPS transport it demonstrably has (your HTTP 101 + fallback logs from other issues show the path exists).

Fix suggestions: (1) on Broken pipe/reset, clear websocket_session state (last_request/last_response) so the session doesn't keep straddling the pre-switch world; (2) after N consecutive WS send failures within one turn, fall back to HTTPS for the remainder of the turn instead of retrying WS indefinitely — that would have completed all three of your stuck turns.