Investigate active MCP transport closure after transient HTTP errors during tools/call

Summary

During an already-initialized Streamable HTTP MCP session, a transient upstream HTTP failure during tools/call has been observed alongside a later closed-channel failure that can make subsequent MCP requests unusable.

This report is limited to the post-initialization request path. Startup retries for initialize and initial tool discovery are handled separately by streamable_http_retry.rs.

Reported behavior

When the upstream HTTP client returns HTTP 429 or 503 for a tools/call POST, http_client_adapter.rs currently represents the non-success response as a string-formatted error similar to:

StreamableHttpError::UnexpectedServerResponse(
    format!("HTTP {}: ...", response.status).into(),
)

The affected tool call fails. In the reported failure sequence, subsequent MCP requests return a closed-channel error until the CLI or Desktop session is restarted.

Example symptom:

worker quit with fatal: Transport channel closed

Source-level cross-check

Codex currently pins rmcp 3.0.0. Reviewing the rmcp-v3.0.0 Streamable HTTP worker shows that a normal post-initialization post_message error is assigned to the current request's send_result, sent through the request responder, and does not directly break the main worker loop.

The response-stream completion path also logs an SSE stream error without directly terminating the main loop.

This means the HTTP 429/503 being converted to UnexpectedServerResponse does not, by itself, appear sufficient to explain a fatal worker shutdown in the active-request path. The closed-channel symptom may instead involve another concurrent or subsequent event.

Minimal reproduction needed

1. Start a Streamable HTTP MCP server and initialize the session successfully.
2. Complete initial tool discovery.
3. Configure the upstream HTTP client or mock server to return HTTP 429 or 503 for the next tools/call POST.
4. Invoke the tool and record the returned request error.
5. Send another request through the same RunningService, such as tools/list or another safe request.
6. Capture the full log sequence from the HTTP response through the first Transport channel closed error.

The reproduction should record:

• the negotiated MCP protocol mode,
• whether a session ID is present,
• the exact MCP method associated with each HTTP request,
• whether the response was buffered JSON, SSE, or an error before content-type handling,
• any cancellation, handler termination, session cleanup, or SSE stream event between the 429/503 and the closed-channel log,
• whether the second request reaches post_message or fails before being sent.

Expected behavior

A transient HTTP failure for one active request should remain a request-level failure when the underlying transport and session are still valid:

• The affected tools/call completes with an error.
• A subsequent request can continue through the same active service.
• The failed tools/call is not automatically replayed because tool calls may be non-idempotent.
• Actual transport loss, cancellation, shutdown, handler termination, or unrecoverable protocol errors continue to close the worker normally.

Current root-cause hypotheses

Based on the source-level cross-check, the highest-value areas to verify are:

• a concurrent handler or transport termination that occurs after the request-level HTTP error,
• Codex connection-manager logic treating the service as closed after the request fails,
• a pending-response or SSE lifecycle edge case associated with the failed tool call,
• the fatal log originating from a different request context than the active tools/call, such as an initialization notification or a reconnect attempt.

The RMCP active request loop itself is currently a lower-probability root cause because its generic post_message error path returns the error to the responder and continues.

Separate structured-error improvement

Independently of the closed-channel root cause, preserving HTTP status information as a typed adapter error may still be useful:

TransientHttp {
    status: StatusCode,
    retry_after: Option<Duration>,
}

This would avoid inferring status codes from formatted UnexpectedServerResponse strings and would let higher layers distinguish temporary backpressure from permanent failures. It should be treated as an error-reporting improvement unless a regression test demonstrates that it also changes the connection lifecycle.

Retry safety

tools/call requests should not be automatically replayed. A tool may have completed a side effect before the HTTP response failed, so replaying the request could duplicate writes or external actions.

When present, Retry-After can instead be propagated as metadata for user-facing messaging or caller-controlled retry behavior. Both standard forms should be considered:

• delay seconds, for example Retry-After: 30
• HTTP date, for example Retry-After: Wed, 05 Aug 2026 08:00:00 GMT

Malformed, missing, expired, or unreasonably large values should not cause a secondary failure.

Suggested validation

• A tools/call receiving HTTP 429 returns a request-level error.
• A tools/call receiving HTTP 503 returns a request-level error.
• A subsequent request is attempted through the same RunningService.
• The test identifies the exact event that closes the service if the second request fails.
• The failed request does not remain pending.
• tools/call is not automatically replayed.
• A real transport closure still terminates the worker.
• Cancellation and normal shutdown still clean up pending requests and tasks.

I am not submitting a code change at this stage. This issue is intended to provide reproduction details, source-level analysis, and a narrowed set of root-cause hypotheses for the Codex team.