app-server: dynamic tool request is broadcast to all thread subscribers and first response wins
What version of Codex are you using?
codex-cli 0.145.0- Source verified against tag
rust-v0.145.0, commit25af12f7e61572b0bc18ddb1008be543b91519b0 - App Server experimental API with
dynamicTools
What platform is your computer?
macOS, Apple Silicon.
What issue are you seeing?
When two app-server connections subscribe to the same thread, a dynamic tool request is sent to every subscriber with the same JSON-RPC request ID. The pending callback is keyed only by request ID, and response processing does not preserve or validate the responding connection ID. The first response or error therefore wins.
If one subscriber owns the dynamic-tool handler and another subscriber does not, the non-owner can return:
No handler registered for tool: <namespace>.<tool>
before the authoritative subscriber returns success. The model/caller sees a failure even though the tool executed successfully.
I observed this with a mutating sessions_create dynamic tool:
- the handler-owning connection executed successfully;
- the Desktop subscriber returned
No handler registeredfor the same call; - the caller saw the error;
- the created background session existed in durable state;
- retrying after the false failure created duplicate work.
This is not only a handler-registration problem. It is a request-routing and response-arbitration race between valid subscribers.
What steps can reproduce the bug?
- Initialize app-server connection A with the experimental API.
- Start a thread from A with a namespaced dynamic tool in
dynamicTools. - Initialize connection B without a handler for that dynamic tool.
- Resume the same thread from B so both connections are subscribers.
- Start a turn that invokes the dynamic tool.
- Observe that both A and B receive
item/tool/callwith the same request ID. - Have B immediately return a missing-handler error.
- Have A execute the tool and return a successful
DynamicToolCallResponse. - Observe that B's error resolves/removes the shared pending callback; A's later valid response is ignored, even though A may already have committed the side effect.
A mutating tool makes the impact especially clear: the operation exists despite the caller-visible failure, and a normal retry can duplicate it.
Source-level findings
In rust-v0.145.0:
- Thread start/resume adds each requesting connection as a subscriber.
- The thread listener gathers all subscribed connection IDs and constructs one
ThreadScopedOutgoingMessageSender. - Dynamic tool calls use the same generic thread-scoped request path, which sends one server request and request ID to every subscribed connection.
- Pending callbacks are stored by
RequestIdonly. MessageProcessor::process_responseandprocess_errordiscard the respondingConnectionIdbefore resolving the callback.DynamicToolCallParamsand initialization capabilities contain no handler-owner, connection-target, or dispatch-capability field.
Relevant areas:
codex-rs/app-server/src/request_processors/thread_lifecycle.rscodex-rs/app-server/src/outgoing_message.rscodex-rs/app-server/src/message_processor.rscodex-rs/app-server-protocol/src/protocol/v1.rs
What is the expected behavior?
A dynamic tool request should have exactly one authoritative responder.
One possible contract:
- A connection explicitly advertises dynamic-tool dispatch capability during initialization.
- For a tool-bearing thread, app-server selects exactly one subscribed capable connection. Zero or multiple capable subscribers fail closed.
item/tool/callis sent only to that connection; ordinary thread notifications can remain broadcast.- The pending request stores the expected connection ID.
- Responses/errors from non-target connections are ignored and cannot resolve the callback.
- Pending dynamic-tool requests are replayed after reconnect only when exactly one capable subscriber exists.
At minimum, a response from a connection that was not targeted for the request must not win the callback race.
Impact
- Successful mutations reported as failures.
- Duplicate sessions, approvals, messages, or other side effects after ordinary retries.
- Caller and durable state disagree.
- Handler-owning clients cannot guarantee their valid response is authoritative.
- Thread resume/reconnect makes the race more likely because dynamic tool definitions persist with the thread while executable handlers remain client-specific.
Application-side idempotency can contain duplicate side effects, but it cannot prevent app-server from delivering the request to a non-handler subscriber or accepting that subscriber's false error.
Related issues
These appear symptomatically related but do not identify this specific multi-subscriber routing cause:
- #29886 —
send_message_to_threadreports handler error even when delivery succeeds - #31119 — thread creation reports missing handler while child threads may still be created
- #28080 — Desktop thread tools intermittently lose handlers
- #24808 — dynamic tools cannot be updated after
thread/start
Additional information
I have a local source-reviewed draft that adds a dispatcher capability, targets one capable subscriber, carries connection identity through response/error processing, rejects non-owner responses, and adds focused routing tests. It has not been submitted or compiled yet. I can prepare it for review if this direction matches the intended app-server ownership model.
1 Comment
@etraut-openai @aibrahim-oai
can we has fix pls? is making agents argue.
i have PR if you'd like, but cause & fix is clear from description above. muchas gracias.