Custom model_providers: /models discovery requires undocumented internal schema; support the standard OpenAI list as a fallback

Open 💬 2 comments Opened Aug 5, 2026 by sudeekshamurari
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

Summary

Codex's models-manager fetches <base_url>/models?client_version=… from custom model_providers and deserializes the response into the internal catalog schema (codex-rs/protocol/src/openai_models.rs). Custom OpenAI-compatible servers — vLLM, llama.cpp, LiteLLM, org gateways — universally serve the standard OpenAI model list ({"object":"list","data":[{"id":…}]}), which fails that deserialization. The failure is silent: the catalog comes back empty and the /model picker shows only builtin presets, so every custom provider appears to have no models even when routing works perfectly.

We (Docker's inference gateway) got our models into the picker by reverse-engineering ModelInfo from the tagged source and emitting it byte-compatibly — and can report the sharp edges of that contract from experience:

  1. Every field without a serde default is load-bearing. Missing any one of base_instructions, truncation_policy, support_verbosity, supports_parallel_tool_calls, default_verbosity, apply_patch_tool_type, experimental_supported_tools, … fails the entire catalog, silently. Providers tracking this schema will break, silently, on any release that adds a required field.
  2. base_instructions is required and replaces the builtin prompt with no inheritance option. A provider must either invent a system prompt (silently degrading Codex's agent behavior for its users) or vendor prompt.md from this repository and re-serve it — which is what we ended up doing.

Requests

  1. Fallback-parse the standard OpenAI model list when the native schema doesn't match, mapping data[].id → minimal ModelInfo with the same defaults model_info_from_slug uses (builtin prompt included). One lenient branch makes model discovery work for the entire OpenAI-compatible ecosystem with zero server-side changes.
  2. Let catalog entries inherit client defaults — e.g. treat empty/absent base_instructions (and other behavioral fields) as "use the builtin," so providers that do speak the native schema don't have to vendor and track prompt.md.

Reproduction

  • Point a model_providers entry at any standard OpenAI-compatible server (e.g. llama.cpp's llama-server), launch the TUI, open /model: only builtin presets appear; no error is surfaced anywhere.
  • Serve the native schema instead (all required fields present): models appear correctly (verified on codex-cli 0.146.0).

🤖 Generated with Claude Code

View original on GitHub ↗

2 Comments

github-actions[bot] contributor · 22 days ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #36532

Powered by Codex Action

sudeekshamurari · 17 days ago

Not a duplicate of #36532, though they share a root cause (custom providers can't populate the picker without producing the internal ModelInfo schema). The two requests are complementary and target different layers:

  • #36532 stays within the static-catalog workflow: it asks for a codex models export command that generates a model_catalog_json file from the models visible to the active credential. The catalog remains a file the user manages; the ask is tooling to build it safely (credential-scoped, atomic writes, no cross-key leakage).
  • This issue asks for runtime changes to model discovery itself: (1) fallback-parse the standard OpenAI {"object":"list","data":[…]} response when the native schema doesn't match, and (2) let native-schema catalog entries inherit client defaults (notably base_instructions) instead of every provider vendoring prompt.md.

They compose rather than overlap: if fallback parsing lands, most servers in #36532's scenario need no export step at all (live discovery already returns only what the credential can see, eliminating the drift #36532 works around); the export command remains useful for the cases that want a pinned, reviewable catalog file. And the default-inheritance request here applies equally to hand-authored or exported catalogs — #36532's exporter would otherwise have to embed a copy of the builtin prompt into every generated entry.