Splunk MCP not working with Codex CLI - handshaking with MCP server failed

What version of Codex CLI is running?

codex-cli 0.128.0

What subscription do you have?

Business

Which model were you using?

gpt-5.5

What platform is your computer?

Darwin 25.2.0 arm64 arm

What terminal emulator and version are you using (if applicable)?

Warp

What issue are you seeing?

Human description:
Hi all, we are trying to use Splunk MCP in codex CLI.
I configured in the config.toml the following:

[mcp_servers.splunk-mcp-server]
url = "https://prd-p-8zn1c.splunkcloud.com/en-US/splunkd/__raw/services/mcp"
bearer_token_env_var = "SPLUNK_TOKEN"
startup_timeout_sec = 30
tool_timeout_sec = 60

When I start Codex CLI I get the following warning.

⚠ MCP client for `splunk-mcp-server` failed to start: MCP startup failed: handshaking with MCP server failed: Send message
  error Transport
  [rmcp::transport::worker::WorkerTransport<rmcp::transport::streamable_http_client::StreamableHttpClientWorker<codex_rmcp_
  client::http_client_adapter::StreamableHttpClientAdapter>>] error: Transport channel closed, when send initialized
  notification

When I configured Cursor to work with Splunk MCP it works fine (as it was configured like this per their documentation)

{
  "mcpServers": {
    "splunk-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote",
        "https://prd-p-8zn1c.splunkcloud.com:8089/services/mcp",
        "--header",
        "Authorization: Bearer <Your Token>"
      ]
    }
  }
}

And also when I configured it as
```
"splunk-mcp-server": {
"url": "https://prd-p-8zn1c.splunkcloud.com/en-US/splunkd/__raw/services/mcp",
"headers": {
"Authorization": "Bearer TOKEN"
}
}

it also worked fine.

However, in Codex when I configure it as I described above, I get the error mentioned.

When I configure codex cli with the following, it works

[mcp_servers.splunk-mcp-server]
command = "npx"
args = [
"-y",
"mcp-remote",
"https://prd-p-8zn1c.splunkcloud.com/en-US/splunkd/__raw/services/mcp",
"--header",
"Authorization: Bearer ${SPLUNK_TOKEN}",
]
env_vars = ["SPLUNK_TOKEN"]
startup_timeout_sec = 30
tool_timeout_sec = 60

### What steps can reproduce the bug?

1. You will need splunk account with token and install the Splunk MCP server
2. use the infromation from splunk (token and url) and configure the MCP in codex
3. start codex
4. try to use the MCP to get information from splunk



### What is the expected behavior?

not error from codex on splunk MCP
and the MCP is working as expected

### Additional information

LLM Generated description:

codex-rmcp-client registers a Streamable-HTTP MCP server in ~/.codex/config.toml, the server appears under codex mcp list and codex mcp get <name>, but at runtime the model never sees the server's tools. How the failure is presented to the user differs by entrypoint:

Under codex exec — silent. No error in user-facing output. The only signal is in RUST_LOG=codex_rmcp_client=debug,rmcp=debug stderr:

INFO  serve_inner: rmcp::service: Service initialized as client
       peer_info=Some(InitializeResult {
         protocol_version: ProtocolVersion("2025-06-18"),
         capabilities: ServerCapabilities { ... tools: Some(ToolsCapability { list_changed: Some(false) }) ... },
         server_info: Implementation { name: "<server-name>", version: "x.y.z", ... }
       })
ERROR rmcp::transport::worker: worker quit with fatal:
       Deserialize error: EOF while parsing a value at line 1 column 0,
       when send initialized notification
Under the interactive codex TUI — surfaced. Same ~/.codex/config.toml, same binary, displays a banner:

⚠ MCP client for `splunk-mcp-server` failed to start: MCP startup failed:
  handshaking with MCP server failed: Send message error
  Transport [rmcp::transport::worker::WorkerTransport<
    rmcp::transport::streamable_http_client::StreamableHttpClientWorker<
      codex_rmcp_client::http_client_adapter::StreamableHttpClientAdapter
    >
  >] error: Transport channel closed,
  when send initialized notification
⚠ MCP startup incomplete (failed: splunk-mcp-server)
So:

The server's response to initialize is well-formed JSON-RPC and parses fine.
The crash is on the client's response to its own notifications/initialized POST.
Once the worker quits, the server is removed from the active set. The TUI displays a banner; codex exec displays nothing.
The TUI banner pins the crate path: the failing component is codex_rmcp_client::http_client_adapter::StreamableHttpClientAdapter, which wraps rmcp::transport::streamable_http_client::StreamableHttpClientWorker. That's where the empty-body deserialize is happening.

Root cause (suspected, based on log + spec + behavior of other clients):

After initialize succeeds, codex-rmcp-client sends the mandatory notifications/initialized JSON-RPC notification as an HTTP POST. The server replies with:

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 0
(empty body)
This is normal/expected for notifications. Per JSON-RPC 2.0, notifications have no id and elicit no response, so an empty 2xx is the natural ack. The MCP Streamable-HTTP transport recommends 202 Accepted, but 200 OK with Content-Length: 0 is what real-world MCP servers commonly return, and other lenient clients (see below) accept it.

StreamableHttpClientAdapter (rmcp 0.15.0, wrapped by codex_rmcp_client::http_client_adapter) appears to unconditionally try to deserialize the HTTP response body as JSON. With Content-Length: 0, that fails with serde_json::Error: EOF while parsing a value at line 1 column 0. The transport worker treats this as fatal and exits — which is what the Transport channel closed, when send initialized notification line in the TUI banner is reporting downstream.

Cross-validation that the MCP server is fine:

The same MCP URL + bearer token works end-to-end with:

@modelcontextprotocol/sdk's StreamableHTTPClientTransport (the transport Cursor uses for native HTTP MCP).
npx -y mcp-remote https://... --header "Authorization: Bearer ${TOKEN}" (the stdio bridge).
Both lenient clients accept the empty-body 2xx ack to notifications/initialized and proceed to tools/list and tools/call normally. Only Codex's built-in rmcp-based transport crashes on it.

What steps can reproduce the bug?
Any spec-compliant Streamable-HTTP MCP server that returns 200 OK with Content-Length: 0 (or any empty body) for a notifications/initialized POST will reproduce this. The server I originally hit was Splunk Cloud's MCP at /__raw/services/mcp, but the failure mode is independent of Splunk — it's just one server that exhibits the empty-body ack pattern.

1. Confirm the server's behavior with curl (no Codex involved):

URL='https://<your-stack>.splunkcloud.com/en-US/splunkd/__raw/services/mcp'
curl -k -sS -i -X POST "$URL" \
  -H "Authorization: Bearer ${SPLUNK_TOKEN}" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","method":"notifications/initialized","params":{}}'
Expected response (this is the legal, "boring" case that breaks Codex):

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 0
(The same empty-body 2xx pattern can be reproduced with any minimal stateless MCP server that you control — just have the notifications endpoint return 200 OK with no body, or 202 Accepted with no body.)

2. Configure Codex to use it. In ~/.codex/config.toml:

[mcp_servers.repro]
url = "https://<your-stack>.splunkcloud.com/en-US/splunkd/__raw/services/mcp"
bearer_token_env_var = "SPLUNK_TOKEN"
startup_timeout_sec = 30
3a. Run Codex non-interactively and watch it fail silently:

RUST_LOG=codex_rmcp_client=debug,rmcp=debug \
  codex exec --skip-git-repo-check --json --full-auto \
  'List the MCP servers you can see and their tools.' </dev/null
Observe:

stderr contains:
ERROR rmcp::transport::worker: worker quit with fatal:
  Deserialize error: EOF while parsing a value at line 1 column 0,
  when send initialized notification
The configured MCP server is not in the list of MCP servers the model enumerates in its answer.
There is no error in the user-facing output — the only signal is in RUST_LOG=debug stderr.
3b. Run the same config under the interactive TUI and observe the banner:

codex
A banner appears at the top of the TUI:

⚠ MCP client for `repro` failed to start: MCP startup failed:
  handshaking with MCP server failed: Send message error
  Transport [rmcp::transport::worker::WorkerTransport<
    rmcp::transport::streamable_http_client::StreamableHttpClientWorker<
      codex_rmcp_client::http_client_adapter::StreamableHttpClientAdapter
    >
  >] error: Transport channel closed, when send initialized notification
⚠ MCP startup incomplete (failed: repro)
For comparison, the same url + same ${SPLUNK_TOKEN} works perfectly with npx -y mcp-remote "$URL" --header "Authorization: Bearer ${SPLUNK_TOKEN}" and with @modelcontextprotocol/sdk's StreamableHTTPClientTransport.

What is the expected behavior?
An empty-body 200 OK (or 202 Accepted) response to a JSON-RPC notification (no id) should be treated as a successful ack, not as a JSON deserialization failure. Notifications, by definition, have no response body to parse.
If the MCP transport handshake fails for any reason, the failure should be surfaced to the user regardless of entrypoint — currently the TUI shows a banner but codex exec is silent.
The configured MCP server should be reachable from the model and its tools should appear in tools/list.