`--oss` mode fails to connect to vLLM endpoint due to unconditional "pull" attempt (HTTP 404)

What version of Codex is running?

codex-cli 0.22.0

Which model were you using?

gpt-oss-120b

What platform is your computer?

Linux 5.15.167.4-microsoft-standard-WSL2 x86_64 x86_64

What steps can reproduce the bug?

Describe the bug

When attempting to use CodexCLI with a vLLM OpenAI-compatible endpoint via the --oss flag, the operation fails. The client appears to be hardcoded to follow an Ollama-style workflow by making requests to /api/tags and /api/pull after an initial check to /v1/models. Since these endpoints do not exist on a standard OpenAI-compatible server like vLLM, the process fails with an HTTP 404 Not Found error.

Environment:

• Codex Version: codex-cli 0.22.0
• Client Platform: WSL (Ubuntu 24.04.1 LTS)
• Backend LLM Server: vLLM
• Backend Server Setup:
• Host: Remote Ubuntu 22.04
• Containerization: Docker
• Image: vllm/vllm-openai:v0.10.1
• Key Command Arguments: --model /models/gpt-oss-120b --served-model-name gpt-oss-120b (Note: model name contains no colons)

Steps to Reproduce:

1. Start the vLLM server using Docker with an OpenAI-compatible endpoint exposed on http://192.168.8.80:8888. The server is configured to serve the model under the name gpt-oss-120b.

1. Verify the endpoint is working and the model is listed correctly via curl.

``bash
$ curl http://192.168.8.80:8888/v1/models
# This returns a 200 OK with the model "gpt-oss-120b" listed.
``

1. Set the environment variable and attempt to send a prompt using codex.

``bash
CODEX_OSS_BASE_URL="http://192.168.8.80:8888/v1" codex --oss -m "gpt-oss-120b" "Hi"
``

Expected Behavior:

I expect CodexCLI to use the /v1/models endpoint to verify the model exists and then proceed directly to the /v1/chat/completions endpoint to get a response.

Actual Behavior:

The client displays a "pulling model" message and fails with a 404 error.

Pulling model gpt-oss-120b...Error: OSS setup failed: failed to start pull: HTTP 404 Not Found

Server-side Log Confirmation

The vLLM Docker container logs provide definitive proof of CodexCLI's behavior. Upon running the codex command, the server receives the following sequence of requests from the client IP (192.168.6.29):

vllm-gpt-oss-120b  | (APIServer pid=1) INFO:     192.168.6.29:13918 - "GET /v1/models HTTP/1.1" 200 OK
vllm-gpt-oss-120b  | (APIServer pid=1) INFO:     192.168.6.29:13918 - "GET /api/tags HTTP/1.1" 404 Not Found
vllm-gpt-oss-120b  | (APIServer pid=1) INFO:     192.168.6.29:13918 - "POST /api/pull HTTP/1.1" 404 Not Found

Analysis:

This log sequence clearly shows that after a successful standard check at /v1/models, CodexCLI immediately attempts to use non-standard, Ollama-specific endpoints (/api/tags, /api/pull). These requests inevitably fail on any generic OpenAI-compatible API server.

This confirms the tool's current incompatibility is not a configuration error but a hardcoded workflow assumption in its --oss mode implementation.

Suggested Solution:

To support the broader ecosystem of OpenAI-compatible servers (vLLM, TGI, etc.), the --oss mode should not assume an Ollama-like API. A potential fix could be:

• Add a flag or a configuration option (e.g., oss_mode="openai_compatible") to bypass the /api/tags and /api/pull checks and interact directly with the standard /v1/chat/completions endpoint.

---

Update & Further Investigation: Bypassing the Connection Issue Reveals a Deeper Streaming Error

Following the initial analysis, I attempted an alternative connection method to bypass the --oss flag's Ollama-specific behavior. I configured a custom model_provider and profile in ~/.codex/config.toml to point directly to my vLLM server, as documented.

Configuration (~/.codex/config.toml):

# Step 1: Define a new model provider pointing to the vLLM endpoint.
[model_providers.vllm]
name = "My vLLM Server"
base_url = "http://192.168.8.80:8888/v1"

# Step 2: Create a profile to bind the provider and model.
[profiles.my_vllm]
model_provider = "vllm"
model = "gpt-oss-120b"

Launch Command:

codex --profile my_vllm

Results:

This approach was partially successful. It successfully bypassed the initial 404 Not Found error, and the interactive Codex TUI started correctly.

However, upon submitting a prompt (e.g., "explain this project's code structure"), the TUI hangs and displays no output. The vLLM server logs reveal a critical error during the response streaming process:

New Server-side Log:

vllm-gpt-oss-120b  | (APIServer pid=1) WARNING 08-20 02:23:57 [protocol.py:81] The following fields were present in the request but ignored: {'strict'}
vllm-gpt-oss-120b  | (APIServer pid=1) INFO:     192.168.6.29:13956 - "POST /v1/chat/completions HTTP/1.1" 200 OK
vllm-gpt-oss-120b  | (APIServer pid=1) ERROR 08-20 02:23:58 [serving_chat.py:1051] Error in chat completion stream generator.
vllm-gpt-oss-120b  | (APIServer pid=1) ERROR 08-20 02:23:58 [serving_chat.py:1051] Traceback (most recent call last):
vllm-gpt-oss-120b  | (APIServer pid=1) ERROR 08-20 02:23:58 [serving_chat.py:1051]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/serving_chat.py", line 646, in chat_completion_stream_generator
vllm-gpt-oss-120b  | (APIServer pid=1) ERROR 08-20 02:23:58 [serving_chat.py:1051]     harmony_parser.process(token_id)
vllm-gpt-oss-120b  | (APIServer pid=1) ERROR 08-20 02:23:58 [serving_chat.py:1051]   File "/usr/local/lib/python3.12/dist-packages/openai_harmony/__init__.py", line 627, in process
vllm-gpt-oss-120b  | (APIServer pid=1) ERROR 08-20 02:23:58 [serving_chat.py:1051]     self._inner.process(token)
vllm-gpt-oss-120b  | (APIServer pid=1) ERROR 08-20 02:23:58 [serving_chat.py:1051] openai_harmony.HarmonyError: Unexpected token 200003 while expecting start token 200006

Conclusion from New Findings

This reveals two distinct, layered issues preventing CodexCLI from working with a vLLM-hosted model:

1. Connection Layer Issue: The --oss flag is fundamentally incompatible with non-Ollama servers due to its hardcoded pull logic, as established previously.
2. Response Parsing Layer Issue: Even when a connection is established using a custom provider, a stream parsing failure occurs on the server side (openai_harmony.HarmonyError). This suggests an incompatibility between how CodexCLI formats its request and/or how vLLM + gpt-oss-120b generate response tokens for streaming. The server-side error breaks the stream, resulting in a silent failure on the client.

Thank you for your work on this great tool.

What is the expected behavior?

_No response_

What do you see instead?

_No response_

Additional information

_No response_