›_ Codex Issues
Codex Issues / #21503

tool_search can miss exactly named deferred MCP tools, causing false unavailable conclusions

Open 💬 4 comments Opened May 7, 2026 by dergachoff
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

tool_search can omit a deferred MCP tool even when the user/model query names that tool exactly.

In a real Codex session with codebase-memory-mcp configured and working, a broad discovery query like:

search_graph trace_path get_code_snippet query_graph search_code index_repository codebase-memory-mcp

did not surface search_graph in the first tool_search result set. A second exact-name query for:

search_graph

did surface the tool.

That creates a false negative: the agent may conclude an MCP tool is unavailable when it is actually installed, callable, and discoverable through a more specific query.

This matters because deferred tool discovery is now a core path for MCP-heavy workflows. Agents are commonly instructed to search for a family of tools first, then fall back only if expected tools are unavailable. A top-N ranking miss makes that workflow unreliable and pushes agents toward worse fallbacks like shell search.

Likely root cause from main: codex-rs/core/src/tools/handlers/tool_search.rs ranks deferred tools through BM25 over ToolSearchEntry.search_text. ToolSearchEntry includes canonical/callable/raw names in the search text, but the handler does not guarantee that exact tool-name matches survive the requested/default result limit.

What steps can reproduce the bug?

  1. Configure an MCP server whose tools are deferred behind tool_search.
  2. Include a tool with a specific callable name, for example search_graph.
  3. Have enough deferred tools with overlapping descriptions/server vocabulary that top-N BM25 ranking is competitive.
  4. Run tool_search with a broad discovery query that includes the exact tool name plus related tool names/server names, for example:
search_graph trace_path get_code_snippet query_graph search_code index_repository codebase-memory-mcp
  1. Observe that the exact named tool can be omitted from the first result set.
  2. Run tool_search again with only the exact name:
search_graph
  1. Observe that the tool is returned.

What is the expected behavior?

If a tool_search query contains an exact deferred tool name, callable name, or canonical MCP name, matching tools should be included ahead of lower-confidence BM25-only matches, while still respecting the requested/default result limit.

Exact-name matches should not require a second retry query to prove the tool exists.

Additional information

I investigated the current implementation and prototyped a small fix locally.

Prototype shape:

  • Add normalized exact-match terms to ToolSearchEntry.
  • For MCP tools, exact terms include canonical MCP name, callable name, and raw tool name.
  • For dynamic tools, exact terms include tool name and namespaced tool name.
  • In ToolSearchHandler, prepend exact-name matches before BM25 results, dedupe by entry id, then truncate to the requested/default limit.
  • Preserve the existing computer-use expanded default cap behavior.

Local validation:

cargo test -p codex-core tools::handlers::tool_search::tests
cargo clippy -p codex-core --lib

Results:

  • tool_search handler tests: 6 passed.
  • clippy: no issues found.

I noticed open PR #19881 overlaps the deferred MCP / tool_search area. If maintainers prefer, this could be handled as a follow-up to that direction rather than as a standalone change.

I am happy to share the patch or open a PR if invited.

View original on GitHub ↗

4 Comments

github-actions[bot] contributor · 3 months ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #21451

Powered by Codex Action

dergachoff · 3 months ago

Reviewed #21451. I think this is related but not a duplicate.

#21451 is about a skill/plugin dependency path where node_repl/js was deferred and the agent reported Browser Use blocked before running the right discovery query.

This issue is narrower and lower-level: tool_search was already called with the exact expected deferred MCP tool name (search_graph) included in the query, but the result set omitted that tool. A second exact-name-only query then returned it.

So the shared failure class is "deferred tool discovery can produce false unavailable conclusions", but the specific bug here is the tool_search ranking contract: exact tool-name matches should survive top-N result selection.

dergachoff · 3 months ago

Related cross-boundary data point.

With Safari MCP installed, a broad discovery query for live DOM/JS debugging surfaced Safari tools but missed mcp__safari__.javascript_tool. An exact follow-up query for javascript_tool surfaced it immediately, and the tool was callable.

This may be partly MCP tool metadata and partly tool_search ranking, but the practical failure mode matches this issue: an available deferred tool looked unavailable after broad discovery.

So exact-name guarantees still seem useful, and capability/alias matching may also need attention for terms like "evaluate JavaScript", "run JS", "DOM inspection", and "current tab script".

foltz · 1 month ago

Additional fresh reproduction/patch data point from a custom workflow integration.

We hit the same failure class with a deferred MCP coordination tool: a coordinator turn needed the MCP tool workflow_run_dispatch_round, but the tool was not directly attached and discovery depended on tool_search. The operational consequence was that the agent treated a real coordination capability as unavailable and fell back to worse paths.

I checked current code and the relevant mechanics still line up with this issue:

  • enabled_tools is only an MCP allowlist filter in codex-rs/codex-mcp/src/tools.rs; it does not force listed tools to attach directly.
  • Direct vs deferred exposure is decided later in codex-rs/core/src/mcp_tool_exposure.rs: when search_tool_enabled is true, direct MCP tools are empty and filtered MCP tools are deferred behind tool_search.
  • search_tool_enabled comes from codex-rs/core/src/tools/spec_plan.rs and is true when the model supports search tools and the provider supports namespace tools.

I prepared a small patch branch here:

https://github.com/foltz/openai-codex/commit/789d29e12029aef32b8a4900a8a24b122cd71e02

Branch:

https://github.com/foltz/openai-codex/tree/bugfix/mcp-tool-search-exact-match

Patch shape:

  • ToolSearchHandler::search first collects exact tool-name matches, then fills remaining result slots with BM25 results.
  • It dedupes by search entry id and still truncates to the caller-provided limit.
  • It handles namespace tool outputs, including suffix matches such as a query token dispatch_round matching workflow_run_dispatch_round.

Regression added:

tools::handlers::tool_search::tests::search_promotes_exact_tool_name_matches_before_bm25_results

The test builds three MCP tools in an ironclaw_workflow namespace, intentionally gives non-target tools stronger BM25 text for dispatch_round, searches with limit = 1, and asserts that workflow_run_dispatch_round is returned.

Validation on the branch:

cargo fmt -p codex-core
just fix -p codex-core
just test -p codex-core tools::handlers::tool_search::tests::search_promotes_exact_tool_name_matches_before_bm25_results

The targeted regression passes. I attempted to open a PR, but this repository currently limits PR creation to collaborators, so I am leaving the branch/commit here instead.