Codex not working with OSS running on VLLM

Resolved 💬 4 comments Opened Aug 13, 2025 by jayteaftw Closed Sep 22, 2025

💡 Likely answer: A maintainer (seratch, member) responded on this thread — see the highlighted reply below.

What version of Codex is running?

codex-cli 0.21.0

Which model were you using?

oss-gpt-120B

What platform is your computer?

Darwin 24.5.0 arm64 arm

What steps can reproduce the bug?

Setup config.toml with

[model_providers.oss]
base_url = "https://vllm.example.com/v1"
env_key = "OPENAI_API_KEY"

[profiles.genai]
model_provider="oss"
model = "openai/gpt-oss-120b"
name = "Vllm"

What is the expected behavior?

I expect prompt "2+2" should respond with "4"

What do you see instead?

I get the output "🖐 error sending request for url (http://localhost:11434/v1/chat/completions)"; however, I set the base_url to not be localhost?

Additional information

Whats the proper way to integrate with VLLM?

View original on GitHub ↗

4 Comments

seratch member · 1 year ago

I've confirmed the base_url in config.toml is not used for oss. As a workaround for now, you can go with the env variable "CODEX_OSS_BASE_URL" and it works without any issues: CODEX_OSS_BASE_URL="http://{your-host}/v1" codex

BlackFrameAI · 1 year ago

A good workaround I have found for now is just start a local TCP forwarder: 127.0.0.1:11434 > xxx.xxx.xxx.xxx:PORT

bartlettroscoe · 11 months ago

I think this is still an issue. Tool calling with codex 0.36.0 does not work with the gpt-oss-20b model served with vllm using their container:

docker run --runtime nvidia --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e "HUGGING_FACE_HUB_TOKEN=$(cat ~/huggingface.token.txt)" \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
  -p 8000:8000 \
  --ipc=host \
  --name vllm_gpt-oss-20b_$(date +%Y-%m-%d_%H-%M-%S) \
  vllm/vllm-openai:latest \
    --model openai/gpt-oss-20b \
    --max-model-len 10k \
    --gpu-memory-utilization 0.95

I can get codex to call the model with "Hello" and other prompts, but as soon as the prompt triggers the usage of tools, codex just does nothing.

But when I serve the gpt-oss-20b model with updated llama.cpp, everything works great.

So is the problem codex or vLLM?

jayteaftw · 11 months ago

@bartlettroscoe Its still and issue, just a separate one found here.

I think it is VLLM. In VLLM's guide for deploying GPT OSS it shows that Response API with Streaming does not support function calling. So if Codex, is calling that api path, then I think that would be the problem