Ultra multi-agent / Responses multi_agent does not work with regular API-key model providers
What version of Codex CLI is running?
0.147.0 (also reproduces on recent Desktop / app-server builds)
What subscription do you have?
ChatGPT Pro (and separately OpenAI API key / custom model_providers for inference)
Which model were you using?
gpt-5.6-sol / gpt-5.6-terra / gpt-5.6-luna (and equivalent IDs via a custom Responses-compatible provider)
What platform is your computer?
macOS (also relevant to any platform using custom model_providers)
What issue are you seeing?
Full “Ultra” multi-agent / subagent orchestration does not work when inference goes through a regular API provider (custom model_providers.* with wire_api = "responses" + env_key API-key auth).
This is distinct from:
- setting
model_reasoning_effort = "ultra"(reasoning effort alone can work on API providers), and - ChatGPT-signed-in / first-party OpenAI provider sessions, where native Ultra multi-agent coordination works.
Observed failure modes with regular API providers:
- Responses API
multi_agentis rejected or non-functional
When a Responses request includes the multi-agent surface (e.g. multi_agent.enabled = true / multi-agent beta), OpenAI-compatible or non–ChatGPT-backend endpoints commonly respond with:
400/Unsupported parameter: multi_agent- or auth / routing errors that never produce a multi-agent tree
- Native Codex Ultra subagent product path is ChatGPT-backend-centric
With a custom API-key model_provider, Ultra-level parallel OpenAI multi-agent coordination (server-side subagent tree, multi_agent_v2 product behavior tied to first-party models) does not behave like a ChatGPT-auth session. Client-side spawn_agent / local multi-agent tooling may partially work, but that is not the same as first-party Ultra multi-agent.
- Related hardcoded ChatGPT-backend assumptions
GPT-5.6 Sol/Terra/Luna paths that assume ChatGPT-backend fields (e.g. responses-lite / multi_agent version headers) already cause 400s on non–ChatGPT backends — see related issues below. That further blocks “just use Ultra models + multi-agent on any Responses-compatible base_url”.
Net: users who correctly configure a regular API provider for Sol/Terra/Luna get coding + reasoning effort, but not the full Ultra multi-agent / subagent product experience.
What steps can reproduce the bug?
- Configure a working API-key custom provider (not ChatGPT OAuth), for example:
[model_providers.my_api]
name = "OpenAI-compatible Responses API"
base_url = "https://api.example.com/v1"
wire_api = "responses"
env_key = "MY_API_KEY"
requires_openai_auth = false
model_provider = "my_api"
model = "gpt-5.6-sol"
model_reasoning_effort = "ultra"
[features]
multi_agent_v2 = true
- Confirm plain inference works:
codex exec --skip-git-repo-check "Reply with exactly: API_OK"
- Attempt Ultra multi-agent / subagent work, e.g.:
- ask the agent to spawn multiple specialized subagents in parallel and synthesize, or
- send a Responses request through the same provider with multi-agent enabled.
- Compare with the same prompt on the default ChatGPT-authenticated OpenAI provider / profile.
What is the expected behavior?
Either:
A (preferred): Full Ultra multi-agent / native subagent orchestration works for regular API-key Responses providers when the upstream model supports it — including transparent multi-agent parameters and no ChatGPT-only internal headers required for Sol/Terra/Luna, or
B: Codex clearly documents and surfaces in product UI/CLI that Ultra multi-agent / server multi-agent is ChatGPT-backend-only, while API-key custom providers only support single-agent inference (and optionally client-local spawn), so users are not left debugging silent no-spawn / Unsupported parameter: multi_agent failures.
Today neither A nor B is complete: config accepts Ultra + multi_agent_v2 + custom providers, but the Ultra multi-agent experience is incomplete or broken on regular API providers.
Additional information
Works on API providers (in our testing):
- Single-agent Responses inference
- Reasoning effort including
ultra(as an effort level) - Many built-in tools that the upstream Responses API actually implements (e.g.
web_searchwhen the endpoint supports it)
Does not work / incomplete on API providers:
- Responses
multi_agent/ server-side multi-agent tree - Full native Ultra multi-agent product behavior comparable to ChatGPT-auth sessions
- Paths that inject ChatGPT-backend-only multi_agent / responses-lite assumptions against a plain API provider
Related issues:
- #17598 — native subagent orchestration broken / incomplete with non-OpenAI custom providers
- #31882 — gpt-5.6-sol/terra/luna hardcode use_responses_lite / multi_agent_version → 400s on non–ChatGPT backends
- #31870 — X-OpenAI-Internal-Codex-Responses-Lite failures on Azure / non-ChatGPT backends
- #13204 — subagent ignores active profile/provider
- #14039 — per-subagent model/provider/profile selection
Please treat this as a product + protocol gap for regular API providers, not an Azure-only issue: any model_providers.* entry that is “just” Responses + API key hits the same Ultra multi-agent wall.
Happy to provide redacted request/response pairs (no secrets) if useful.
4 Comments
Potential duplicates detected. Please review them and close your issue if it is a duplicate.
Powered by Codex Action
Data point for the client-side part: non-OpenAI parent -> non-OpenAI child V2 spawn / followup works reliably once the task is delivered as a plain user message (V1-style
UserInput); we verified this with deepseek-v4-flash. OpenAI parent -> third-party child remains broken because the assignment is real ciphertext (#36376 / #37197) and needs a parent-side plaintext delivery path. Patch and evidence: https://github.com/CCanxue/codex-deepseek-subagent-fixAdditional data point aligning with this issue for custom API-key providers (Windows, Codex Desktop).
Versions
codex-cli 0.147.0, Desktop MSIXOpenAI.Codex_26.803.10989.0_x64(internal26.803.81509), custommodel_provider = "litellm", auth modeApiKey(no ChatGPT/OpenID token).Headless path works; the Desktop app UI does not
codex execon the same custom provider works, and the CLI/rollout layer shows sub-agent activity (rollout source breakdown includessubagent:thread_spawn,subagent:other,subagent:reviewentries).namespace_flatten_callbackthat rewrites/routes the requests through the custom provider so plain inference and headless spawning succeed.Observed app behavior
ApiKeyprovider is configured and plain inference works (both headless and, partially, in the app).collaboration.spawn_agent/ MultiAgentV2tool_namespaceinterplay is exactly the edge the flatten callback has to compensate for here.Happy to provide redacted request/response pairs if useful. No credentials included.
Adding the client-side mechanism that makes API-key multi-agent structurally broken today, beyond server-side gating: inter-agent task payloads are shipped as
encrypted_contentblobs that only the originating ChatGPT-backed session context can decrypt server-side. The client never decrypts them —InterAgentCommunication::to_model_input_itememits a plaintext header plus anEncryptedContentitem and trusts the backend to expand it (protocol/src/protocol.rs#L813-L846). With a custommodel_providers.*+env_keysetup there is no such backend context, so even when spawn mechanics succeed, the sub-agent receives an empty payload and idles — the exact silent failure documented in #37822, and the cross-provider variant in #38656.So making Ultra multi-agent work on API providers needs two things, not one: (a) whatever server-side
multi_agentcapability gating applies, and (b) a plaintext inter-agent message path when the participants aren't on the encrypting backend — the constructor for it already exists (InterAgentCommunication::newvsnew_encrypted). Until then, the documentation/gating ask in #37859 seems right: fail at spawn time with a clear "multi-agent requires ChatGPT auth / first-party provider" instead of half-working.