MCP tool calls crash with aws-lc-rs jitter-entropy panic ("Transport closed") once they run past ~90 seconds — not specific to fan-out calls

Open 💬 4 comments Opened Aug 12, 2026 by ravindren-sm

UPDATE 3 (recovery timing is inconsistent, not a fixed window): The first update above described the poisoned window as temporary, self-clearing in roughly 30-75 minutes without a restart. A dedicated follow-up test running the full 90-minute poll (single MCP call every 5 minutes, no restart, same process the whole time) never recovered: 18/18 attempts over 1h 29m 19.5s all failed with the identical signature.

So across the two runs observed:

  • Run 1: poisoned window lasted roughly 30-75 minutes, then cleared on its own.
  • Run 2: poisoned window lasted at least 89 minutes with zero recovery (test ended before it cleared, if it ever did).

Conclusion: self-recovery is real (it has happened) but not reliable or time-bounded — don't rely on "just wait" as a fix. A full Codex process restart remains the only dependable way to clear the bad state; waiting can cost anywhere from nothing to well over 90 minutes with no guarantee of resolution.
UPDATE 2 (further correction, minimal repro found): The previous update above framed this as correlating with multi-provider fan-out tool calls specifically. A controlled follow-up test shows that framing was still not quite right — please read this update as superseding it.

Ran a set of isolated, single-purpose Codex CLI processes (codex exec), each making exactly one MCP tool call to a single-provider tool (no fan-out, no concurrency, one upstream model request per call), varying only how long the call took:

| Test | Tool shape | Elapsed | Result |
|---|---|---|---|
| A | single-provider chat, moderate prompt | 61.7s | succeeded |
| B | single-provider chat, deliberately large prompt/output | 106.9s | failed — Transport closed / aws_lc_0_39_0_jent_entropy_switch_notime_impl, identical to every prior occurrence |

Test B made no other MCP or fan-out calls whatsoever. This is the smallest reproduction so far: a single MCP tool call, to a single provider, with no concurrency of any kind, crashes the same way once it runs past roughly 90-100 seconds.

Combined with the earlier data, the actual pattern appears to be:

  • Any individual MCP tool call that completes in well under ~90s succeeds reliably (observed up to 61.7s).
  • Any individual MCP tool call that runs past ~90-107s crashes with this exact signature, regardless of whether it's a single-provider call or a multi-provider fan-out call.
  • Multi-provider fan-out calls aren't special in themselves — they just reliably run long enough (waiting on the slowest of N concurrent upstream requests) to cross whatever this threshold is, so they appeared correlated with the crash in earlier testing. A single slow call crosses the same threshold and crashes identically.

This points strongly at a fixed internal timeout somewhere around 90 seconds in Codex's own MCP client/tool-call path, whose expiry handling exercises aws-lc-rs's Jitter Entropy self-test, and that self-test failing takes down the shared MCP transport (see prior update for the temporary-not-permanent poisoned-window behavior) rather than just cleanly failing the one slow call.

If it's useful for tracking down the actual timeout constant: the four measured failure durations across this whole investigation were 91.77s, 92.33s, 92.88s, and 106.86s — consistent with a timeout in the ~90s neighborhood plus some retry/overhead variance, not a hard single fixed value observed so far.
UPDATE (correction): The original report below claimed this crash permanently disables MCP tool calls for the rest of the session, with the only fix being a full process restart. Further log analysis shows that claim was wrong in an important way, and I want to correct it rather than leave inaccurate info in the issue. Corrected understanding:

  • The crash consistently occurs on tool calls that internally fan out to multiple upstream model providers concurrently (a "council"-style tool issuing several parallel requests), and consistently after ~90-93 seconds of the call being in flight. Three independent occurrences in one session measured 92.33s, 92.88s, and 91.77s before failing — suspiciously close to a fixed internal timeout, not random jitter.
  • Single-provider tool calls on the same MCP server never triggered the crash directly, even when they ran just as long (up to 45s observed) or were issued minutes before/after a fan-out crash.
  • After a crash, there IS a window where all subsequent MCP tool calls — including calls to a different, unrelated stdio MCP server — fail instantly with the identical error. But this window is temporary, not permanent: in the session analyzed, it lasted roughly 30-75 minutes and then cleared on its own, with a call to the same server succeeding again with no process restart. A second and third fan-out call later re-triggered fresh ~90s-duration crashes, each followed by another temporary all-calls-fail window.

Revised timeline from one session (tool/server names generalized; "Server 1" = a custom stdio MCP server proxying an internal model gateway, "Server 2" = a second, unrelated custom stdio MCP server; "fan-out tool" = a tool that issues several concurrent upstream requests per call, "chat tool" = a tool that issues one upstream request per call):

t+0:00   Server 1 / list-models tool     completed   0.27s
t+0:47   Server 1 / fan-out tool         completed  24.28s
...(~10h gap)...
t+10:11  Server 1 / list-models tool     completed   0.14s
t+10:13  Server 2 / fan-out tool         FAILED     92.33s   <- crash #1
t+10:13  Server 1 / chat tool            completed  21.85s   <- recovers immediately
t+10:14  Server 1 / chat tool            completed  28.03s
t+10:14  Server 1 / chat tool            completed  19.66s
t+10:15  Server 1 / chat tool            completed   8.49s
t+10:15  Server 1 / chat tool            completed   6.66s
t+10:15  Server 1 / chat tool            completed   8.43s
t+10:16  Server 1 / chat tool            completed  39.30s
t+10:16  Server 1 / chat tool            completed   2.88s
t+10:34  Server 1 / fan-out tool         completed  31.14s
t+10:34  Server 1 / fan-out tool         completed  31.73s
t+10:35  Server 1 / fan-out tool         completed  27.15s
t+10:53  Server 1 / fan-out tool         completed  30.51s
t+11:35  Server 1 / fan-out tool         FAILED     92.88s   <- crash #2
t+11:35  Server 1 / fan-out tool         FAILED      0.00s   (instant, poisoned window)
t+11:35  Server 2 / fan-out tool         FAILED      0.00s
t+11:35  Server 1 / chat tool            FAILED      0.00s
t+12:06  Server 1 / list-models tool     FAILED      0.00s
t+12:47  Server 1 / chat tool            completed  45.26s   <- recovers again, unprompted
t+12:49  Server 1 / fan-out tool         FAILED     91.77s   <- crash #3
t+12:49  Server 1 / chat tool            FAILED      0.00s

So the more accurate framing is: a concurrent multi-provider MCP tool call reliably crashes Codex's MCP transport after ~90 seconds, which then produces a temporary (not permanent) window where all MCP calls across all configured servers fail instantly, before self-clearing. A full Codex restart is still a reliable way to clear the bad state immediately, but it isn't the only way it clears, and the failure isn't scoped to "the rest of the session" as originally stated.

I'd guess the ~90s constant points at some fixed internal timeout (tokio timeout, keepalive, or reconnect-attempt deadline) whose expiry path exercises aws-lc-rs's Jitter Entropy self-test, and that self-test failing takes down more than the one in-flight request. The multi-provider fan-out correlation suggests the trigger may specifically involve concurrent connection/TLS-session-ticket handling rather than any long-running call in general — that's the piece I'd focus investigation on given this data.

---

Original report

Title

MCP tool calls permanently fail with "Transport closed" (aws-lc-rs jitter-entropy panic) for the rest of the session after one crash

Codex CLI version

0.147.0 (npm install, @openai/codex)

Environment

  • OS: Windows 11 Enterprise (build 10.0.26200), x86_64
  • Install method: npm global install
  • model_provider uses a custom OpenAI-compatible endpoint with wire_api = "responses"
  • Multiple mcp_servers entries configured as local stdio servers (each spawns a local Python subprocess)
  • Tested with [windows] sandbox = "elevated" and also with --dangerously-bypass-approvals-and-sandbox (sandboxing fully disabled) — see notes below

What happened

During a long-running interactive session, an MCP tool call to one configured stdio MCP server failed with:

tool call error: tool call failed for `<server>/<tool>`

Caused by:
    Transport closed

Stack backtrace:
   0: <unknown>
   1: <unknown>
   2: aws_lc_0_39_0_jent_entropy_switch_notime_impl
   3: <unknown>
   4: <unknown>
   5: <unknown>
   6: <unknown>
   7: <unknown>
   8: <unknown>
   9: <unknown>
  10: <unknown>
  11: <unknown>
  12: <unknown>
  13: <unknown>
  14: <unknown>
  15: <unknown>
  16: <unknown>
  17: <unknown>
  18: <unknown>
  19: <unknown>
  20: <unknown>
  21: <unknown>
  22: <unknown>
  23: BaseThreadInitThunk
  24: RtlUserThreadStart

After this first failure, every subsequent MCP tool call for the remainder of the session — including calls to a completely different, unrelated MCP server — failed instantly (single-digit milliseconds, versus the ~90 seconds the original call had been in flight) with the byte-for-byte identical error. This persisted across dozens of tool calls and roughly 20 hours of wall-clock time until the Codex process was manually killed and a new session started, after which MCP tool calls worked normally again.

Diagnosis notes

  • The tool call that first crashed was not the one that failed on every later attempt — the failure is not scoped to one server or one tool. Once it happens, calls to any configured MCP server instantly fail identically.
  • Confirmed the MCP servers themselves stayed healthy the whole time: calling the same underlying tools directly via a separate, non-Codex MCP client succeeded normally while the Codex session was still reporting Transport closed for every attempt.
  • Reproduced the healthy path in a fresh Codex process (codex exec --dangerously-bypass-approvals-and-sandbox) immediately after killing the poisoned process — same MCP server, same tool, clean result. This isolates the fault to Codex's own process state, not the server or config.
  • Correlated the poisoned session's PID to its exact launch flags via Get-CimInstance Win32_Process: it was started with --dangerously-bypass-approvals-and-sandbox, i.e. sandboxing was already fully disabled when the crash occurred. So Windows sandbox mode (elevated vs unelevated) does not appear to be a causal factor, at least not exclusively.
  • The stack symbol aws_lc_0_39_0_jent_entropy_switch_notime_impl points at aws-lc-rs's Jitter Entropy health-test/self-test path, which is designed to abort hard on failure. Whatever thread/task that runs in appears to share fate with (or block) the MCP connection multiplexer for the entire process, since a single failure there poisons all subsequent MCP calls rather than just the one in flight.

Impact

  • A single transient crash (plausibly triggered by host timer-precision conditions — e.g. corporate endpoint-security software hooking high-resolution timer APIs, virtualization, etc.) takes down MCP tool access for the rest of the session, silently, with no recovery path short of fully restarting Codex.
  • There's no user-facing signal that this is a fatal, permanent, process-wide failure rather than a transient per-call error. The identical error repeats on every retry, which reads like a flaky single tool/server rather than "the process's MCP subsystem is dead."

Expected behavior

  • Ideally the aws-lc-rs entropy self-test failure shouldn't crash any part of the process — or if it must, it should only fail the one in-flight request rather than permanently poisoning the shared MCP transport.
  • Short of a fix, it would help if Codex detected a dead/corrupted MCP transport and attempted to reconnect/respawn per-server on the next call, or at minimum surfaced a distinct error ("MCP subsystem has crashed — restart Codex") instead of repeating the same per-call error indefinitely.

Suggested investigation

  • Why does a Jitter Entropy self-test failure (which reads as a local CPU/timing health check, unrelated to any actual network TLS handshake for a local stdio MCP server) get exercised on the MCP tool-call path at all?
  • Why does one failure there cascade to every other configured MCP server, rather than being isolated to the connection that hit it?

View original on GitHub ↗

4 Comments

ravindren-sm · 15 days ago

Correcting my own report: the original title/body claimed this crash permanently kills MCP tool access for the rest of the session, fixable only by a restart. New timeline data (edited into the issue body above) shows that's wrong — the failure window is temporary and self-clears without a restart, and the crash itself correlates specifically with multi-provider fan-out tool calls timing out around ~90-93s, not with long-running calls in general (single-provider calls up to 45s never triggered it directly). Editing the title/body to reflect this rather than leaving the earlier, less accurate framing up.

ravindren-sm · 15 days ago

Second correction, sorry for the churn: further isolated testing (single MCP call, single provider, no fan-out at all) shows this isn't specific to multi-provider fan-out calls either — it's duration. A lone single-provider call that happened to run 106.9s crashed with the identical signature, while ones under ~62s succeeded. Multi-provider fan-out calls just reliably run long enough to cross whatever the real threshold is (~90-107s observed), so they looked correlated with the bug when they weren't the actual cause. Updated title/body above with the minimal repro and the four measured crash durations (91.77s, 92.33s, 92.88s, 106.86s) in case that constant is useful for tracking down the actual timeout in the codebase.

ravindren-sm · 15 days ago

Third update: ran the full 90-minute retry poll (single lightweight MCP call every 5 min, same process throughout, no restart) after triggering the crash again. It never recovered — 18/18 attempts over 1h 29m 19.5s all failed identically. So the 'temporary poisoned window' from update 1 was real for that occurrence but isn't a reliable/bounded thing — this run stayed dead for at least 89 minutes with no sign of clearing. Restarting the Codex process remains the only dependable fix; waiting is a gamble, not a workaround.

jdcodes1 · 10 days ago

Build-system data point that likely settles this: the repo already treats this exact aws-lc code path as broken — but only disables it for Linux musl release builds. The release workflow sets AWS_LC_SYS_NO_JITTER_ENTROPY=1 with the comment "Avoid problematic aws-lc jitter entropy code path on musl builders", gated to *-unknown-linux-musl targets only:

https://github.com/openai/codex/blob/1f41cc5d92722748e45cae9cecc6d883a4e7cbb1/.github/workflows/rust-release.yml#L202-L212

The Windows release workflow (rust-release-windows.yml) sets no such flag, so your binary ships with jitter entropy enabled — and your panic frame (aws_lc_…_jent_entropy_switch_notime_impl) is precisely that path. The behavioral shape fits a jitter self-test failure too: aws-lc marks the entropy source bad process-wide (your poisoned window where unrelated MCP calls fail instantly), with unreliable recovery. Your ~90s correlation would then be whatever reconnect/TLS-reseed the long call triggers, rather than the timeout itself being at fault.

Suggested fix: extend the same env line to the Windows (and macOS, for consistency) build matrix — aws-lc then seeds from the OS RNG (BCryptGenRandom), which is the better source on Windows anyway. One-line workflow change, directly testable against your minimal 106s single-call repro.