Support for local / other LLMs

The current logic supports the openai sdk, but we are excited to review contributions that would decouple this if done without introducing significant complexity to the project.

Some parts about the code patch format that the model is expected to generate are quite specific to the openai models as we have seen this improve general accuracy and model performance, and so at the moment this isn't a trivial refactor and change.

@tibo-openai I’d be very happy to find a way to introduce this without making the project too complex.

Will there be support for Azure OpenAI? I don’t see an option to set a custom endpoint. This would be a great addition for enterprise use

i knew this would be the very first question people will be asking 😁

There doesn't seem to be any as far as I can see

https://github.com/openai/codex/issues/14#issuecomment-2810871348

This project is currently using the /responses endpoint from OpenAI's API. Ollama has compatibility with some OpenAI API endpoints, but the /responses endpoint is not included yet. If Ollama supported this endpoint, we could simply update the OPENAI_BASE_URL variable (defined here) to point directly to the Ollama URL.

I've opened an issue in the Ollama repository about this

Hi all,

Would the maintainers be open to adding a lightweight LLMProvider abstraction in the CLI? The idea is to decouple transport from prompt/patch logic, so you could plug in OpenAI, Ollama (or any other backend).

E.g., an interface:

interface LLMProvider {
complete(prompt: string, opts: any): Promise<string>;
}

and two simple adapters (OpenAIProvider for /responses, OllamaProvider for /completions + wrap). Defaults remain on OpenAI.

This keeps the CLI flexible for local/offline models while preserving existing behavior. Curious if this aligns with the repo’s roadmap - thanks!

I just found this fork which uses the completions api to work with gemini! It could probably work with ollama as well https://github.com/ymichael/codex

The best current way to do this is to use LiteLLM Proxy. You can quickly spin one up locally.

https://docs.litellm.ai/docs/simple_proxy

All you have to do is set OPENAI_BASE_URL to your local LiteLLM proxy and then you can set OPENAI_API_KEY to your LiteLLM proxy key.

Voila, you can configure any model you want from any provider that supports Responses API (currently only directly to OpenAI, but should be available for other providers soon) and codex works!

Is that possible to set BASE_URL in https://github.com/openai/codex/blob/main/codex-cli/src/utils/agent/agent-loop.ts#L255

The best current way to do this is to use LiteLLM Proxy. You can quickly spin one up locally. https://docs.litellm.ai/docs/simple_proxy All you have to do is set OPENAI_BASE_URL to your local LiteLLM proxy and then you can set OPENAI_API_KEY to your LiteLLM proxy key. Voila, you can configure any model you want from any provider that supports Responses API (currently only directly to OpenAI, but should be available for other providers soon) and codex works!

I do not believe that litellm would build responses api upon chat completions (which is supported by most third party provider) because of lack of ability.

@tcztzy They have implemented Responses API already, it is just not supported for all providers yet. They are adding Azure support this week:

https://github.com/BerriAI/litellm/issues/9653#issuecomment-2811760812

Appreciate the downvote though.

https://github.com/mudler/LocalAI lets you host local LLMs with /v1/responses
Can't wait to try it out!

@marty-sullivan
I know, but azure's responses api is not implemented by litellm, litellm would not and could not do such a tough task. I prefer write by myself.

What about https://github.com/open-responses/open-responses?

After reviewing more of the request pipeline (especially around the /responses endpoint and tool support), I realize this implementation heavily relies on OpenAI-specific features.

Particularly:
The tools field enables structured function calls (like shell) which Ollama doesn’t support yet.
The /responses endpoint returns streaming patches and reasoning metadata, which aren’t part of most third-party APIs (including Ollama’s /generate or /completions).
Features like reasoning hints and Codex-style diff handling also seem tightly coupled to OpenAI models.

https://github.com/openai/codex/blob/b0ccca555685b1534a0028cb7bfdcad8fe2e477a/codex-cli/src/utils/agent/agent-loop.ts#L512

So at this point, it looks like supporting third-party models would require a separate, more limited execution path - or a pluggable provider setup that degrades gracefully when features like tools aren’t available. Please correct me if I'm wrong.

Appreciate everyone’s thoughts - mostly leaving this here in case others are exploring similar directions!

What about https://github.com/open-responses/open-responses?

Great, even currently not support DeepSeek official api, but we can access through openrouter.

I just set up open-responses and it works. Tool Use is currently disabled because it is only partially implemented in the library, but seems like it is on the roadmap. https://github.com/lolrazh/codex

https://github.com/user-attachments/assets/b1bdc00d-f971-4fc2-84eb-a9cb8269b81c

@marty-sullivan I know, but azure's responses api is not implemented by litellm, litellm would not and could not do such a tough task. I prefer write by myself.

@tcztzy I have no idea what you mean by this... Did you not go to the issue that I linked? They are literally implementing it for Azure right now. They already have it implemented for OpenAI. Their entire business is creating parity with the OpenAI API

Re-purposing this thread for general support of other LLMs and model providers. There are a series of refactors needed to support this and I would be excited to help here in reviewing a solid plan that allows us to keep excellent support for the current set of models, without ending up in a worst of all worlds scenario and where certain features can be better supported or toggled based on whether the underlying models / api support it well.

FYI I got Codex to work locally with OpenAI on Azure but it required this hack to pass the api-version query parameter: https://github.com/s4v4g3/codex/blob/feature/support-azure-hack/codex-cli/src/utils/agent/agent-loop.ts#L541

For those plugin codex with litellm, this issue should make it work https://github.com/BerriAI/litellm/issues/9653 , related merge request https://github.com/BerriAI/litellm/pull/10116

@tcztzy I have no idea what you mean by this... Did you not go to the issue that I linked? They are literally implementing it for Azure right now. They already have it implemented for OpenAI. Their entire business is creating parity with the OpenAI API

thx, I will do more research on it. Previous comments have been marked as off-topic and hidden for others new to this issue thread.

FYI I got Codex to work locally with OpenAI on Azure but it required this hack to pass the api-version query parameter: https://github.com/s4v4g3/codex/blob/feature/support-azure-hack/codex-cli/src/utils/agent/agent-loop.ts#L541

@s4v4g3 are you able to get it to work with o3 and o4-mini on Azure? I'm finding that when I deploy these models in Azure they are just returning empty responses when using the Responses API.

> FYI I got Codex to work locally with OpenAI on Azure but it required this hack to pass the api-version query parameter: https://github.com/s4v4g3/codex/blob/feature/support-azure-hack/codex-cli/src/utils/agent/agent-loop.ts#L541 @s4v4g3 are you able to get it to work with o3 and o4-mini on Azure? I'm finding that when I deploy these models in Azure they are just returning empty responses when using the Responses API.

Same for me. For /openai/responses?api-version=2025-03-01-preview endpoint with o3 or o4-mini, I get a 200 status and a response content length of 0.

Edit: per your comments in the LiteLLM issue discussion, sounds like you're seeing the exact same behavior.

Microsoft employee here - I checked with the Azure OpenAI team, and they confirmed that the responses API does not work yet for those models. Here's the list of supported models:
https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/responses?tabs=python-secure#model-support

Maybe we could find a way to make the model or provider selection process a bit more general.

Let me explain. Say I just downloaded Codex for the first time. When I launch it, the first thing it asks me is whether I want to use a local model or run something on my own server (and if so, I just give it the endpoint), or which provider I want to use, and then which model.

Only after that, if I pick a provider, it asks me for the API key.

I get that this could feel a bit tedious, but it would basically let me use whatever I want with the tool, so it would make it super flexible in that sense.

For parsing the responses, unless we can find some sort of middle ground, we’d probably need to introduce different parsing logic depending on the type of response. That might get a little annoying, to be honest.

I tried to use Codex with the Deepseek API, I set the OPENAI_BASE_URL environment variable to https://api.deepseek.com and added the DeepSeek API key. It loads the available model properly, but when I send a message it says

OpenAI rejected the request. Error details: Status: 404, Code: unknown, Type: unknown, Message: 404 status code (no body). Please verify your settings and try again.

Why doesn't this work? Doesn't DeepSeek API work with the Responses API?

I tried to use Codex with the Deepseek API, I set the OPENAI_BASE_URL environment variable to https://api.deepseek.com and added the DeepSeek API key. It loads the available model properly, but when I send a message it says > OpenAI rejected the request. Error details: Status: 404, Code: unknown, Type: unknown, Message: 404 status code (no body). Please verify your settings and try again. Why doesn't this work? Doesn't DeepSeek API work with the Responses API?

DeepSeek don’t have responses api entry point.

Thanks for mentioning LiteLLM @marty-sullivan, we now support using Anthropic, Bedrock, Vertex, Azure OpenAI, Google, Deepseek, Ollama with OpenAI Codex

Tutorial using Codex with LiteLLM: https://docs.litellm.ai/docs/tutorials/openai_codex

!Image

Steps to Use

1. Install OpenAI Codex CLI:

npm i -g @openai/codex

2. Configure model routing in litellm_config.yaml:

model_list:
 - model_name: o3-mini
   litellm_params:
     model: openai/o3-mini
     api_key: os.environ/OPENAI_API_KEY
 - model_name: gemini-2.0-flash
   litellm_params:
     model: gemini/gemini-2.0-flash
     api_key: os.environ/GEMINI_API_KEY

3. Start LiteLLM Proxy:

litellm --config /path/to/config.yaml

4. Point Codex to LiteLLM Proxy (via Env Variables)

We point codex to litellm as a base url and set the api key to the litellm virtual key

export OPENAI_BASE_URL=http://0.0.0.0:4000
export OPENAI_API_KEY="sk-1234"

5. Run Codex with your preferred model:

codex --model gemini-2.0-flash --full-auto

Thanks for mentioning LiteLLM @marty-sullivan, we now support using Anthropic, Bedrock, Vertex, Azure OpenAI, Google, Deepseek, Ollama with OpenAI Codex Tutorial using Codex with LiteLLM: https://docs.litellm.ai/docs/tutorials/openai_codex !Image  # Steps to Use ## 1. Install OpenAI Codex CLI: npm i -g @openai/codex ## 2. Configure model routing in litellm_config.yaml: model_list: - model_name: o3-mini litellm_params: model: openai/o3-mini api_key: os.environ/OPENAI_API_KEY - model_name: gemini-2.0-flash litellm_params: model: gemini/gemini-2.0-flash api_key: os.environ/GEMINI_API_KEY ## 3. Start LiteLLM Proxy: litellm --config /path/to/config.yaml ## 4. Point Codex to LiteLLM Proxy (via Env Variables) We point codex to litellm as a base url and set the api key to the litellm virtual key `` export OPENAI_BASE_URL=http://0.0.0.0:4000 export OPENAI_API_KEY="sk-1234" `` ## 5. Run Codex with your preferred model: codex --model gemini-2.0-flash --full-auto

How can I get:

system
⚠️ OpenAI rejected the request. Error details: Status: 400, Code: 400, Type: unknown, Message: 400
litellm.BadRequestError: Responses API not available for custom_llm_provider=gemini, model: gemini-2.0-flash. Please
verify your settings and try again.

Thanks @ishaan-jaff for the instructions, but I still get this error in my proxy terminal. Shouldn't it work with DeepSeek?

`litellm.exceptions.BadRequestError: litellm.BadRequestError: Responses API not available for custom_llm_provider=deepseek, model: deepseek-chat

07:03:07 - LiteLLM Proxy:DEBUG: common_request_processing.py:341 - An error occurred: litellm.BadRequestError: Responses API not available for custom_llm_provider=deepseek, model: deepseek-chat None`

Thanks for mentioning LiteLLM @marty-sullivan, we now support using Anthropic, Bedrock, Vertex, Azure OpenAI, Google, Deepseek, Ollama with OpenAI Codex Tutorial using Codex with LiteLLM: https://docs.litellm.ai/docs/tutorials/openai_codex !Image  # Steps to Use ## 1. Install OpenAI Codex CLI: npm i -g @openai/codex ## 2. Configure model routing in litellm_config.yaml: model_list: - model_name: o3-mini litellm_params: model: openai/o3-mini api_key: os.environ/OPENAI_API_KEY - model_name: gemini-2.0-flash litellm_params: model: gemini/gemini-2.0-flash api_key: os.environ/GEMINI_API_KEY ## 3. Start LiteLLM Proxy: litellm --config /path/to/config.yaml ## 4. Point Codex to LiteLLM Proxy (via Env Variables) We point codex to litellm as a base url and set the api key to the litellm virtual key `` export OPENAI_BASE_URL=http://0.0.0.0:4000 export OPENAI_API_KEY="sk-1234" `` ## 5. Run Codex with your preferred model: codex --model gemini-2.0-flash --full-auto

You should update doc. I success run codex with deepseek with install litellm from github but not pypi. uv tool install "git+https://github.com/BerriAI/litellm[proxy]"

Thanks @ishaan-jaff for the instructions, but I still get this error in my proxy terminal. Shouldn't it work with DeepSeek? > litellm.exceptions.BadRequestError: litellm.BadRequestError: Responses API not available for custom_llm_provider=deepseek, model: deepseek-chat > 07:03:07 - LiteLLM Proxy:DEBUG: common_request_processing.py:341 - An error occurred: litellm.BadRequestError: Responses API not available for custom_llm_provider=deepseek, model: deepseek-chat None

!Image

1. uv tool install "git+https://github.com/BerriAI/litellm[proxy]"
2. prepare a minimal config yaml only contain deepseek like following
3. litellm --config config.yaml
4. export OPENAI_BASE_URL=http://127.0.0.1:4000 and export DEEPSEEK_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxx
5. codex -m deepseek-chat

model_list:
  - model_name: deepseek-chat
    litellm_params:
      model: deepseek/deepseek-chat
      api_key: "os.environ/DEEPSEEK_API_KEY"

> Thanks for mentioning LiteLLM @marty-sullivan, we now support using Anthropic, Bedrock, Vertex, Azure OpenAI, Google, Deepseek, Ollama with OpenAI Codex > Tutorial using Codex with LiteLLM: https://docs.litellm.ai/docs/tutorials/openai_codex > !Image > > > > >  > > > > > > > > > > > > > > > > > >  > > > > >  > > > > > > > > > > > > > > > > > > > # Steps to Use > ## 1. Install OpenAI Codex CLI: > npm i -g @openai/codex > ## 2. Configure model routing in litellm_config.yaml: > model_list: > > model_name: o3-mini > litellm_params: > model: openai/o3-mini > api_key: os.environ/OPENAI_API_KEY > model_name: gemini-2.0-flash > litellm_params: > model: gemini/gemini-2.0-flash > api_key: os.environ/GEMINI_API_KEY > > ## 3. Start LiteLLM Proxy: > litellm --config /path/to/config.yaml > ## 4. Point Codex to LiteLLM Proxy (via Env Variables) > We point codex to litellm as a base url and set the api key to the litellm virtual key > `` > export OPENAI_BASE_URL=http://0.0.0.0:4000 > export OPENAI_API_KEY="sk-1234" > ` > > > > > > > > > > > > ## 5. Run Codex with your preferred model: > codex --model gemini-2.0-flash --full-auto How can I get: system ⚠️ OpenAI rejected the request. Error details: Status: 400, Code: 400, Type: unknown, Message: 400 litellm.BadRequestError: Responses API not available for custom_llm_provider=gemini, model: gemini-2.0-flash. Please verify your settings and try again.`

You might need upgrade litellm greater than v1.66.3.dev5, currently (1745057457.941345) not available from pypi. You can install via uv tool install "git+https://github.com/BerriAI/litellm[proxy]"

Hi @tcztzy @nivzx @nikkoxgonzales send your litellm related questions here: https://github.com/BerriAI/litellm/discussions/10156

Let's not add litellm related questions on this issue

Hey @gdjohn4s, I am very curious about the need behind your (second-most-upvoted) Codex request. Is it mere privacy concerns, or is there something more to running a much weaker LLMs locally instead of the trillion/sub-trillion parameter models? I'd love to understand your need behind your request.

Hey @gdjohn4s, I am very curious about the need behind your (second-most-upvoted) Codex request. Is it mere privacy concerns, or is there something more to running a much weaker LLMs locally instead of the trillion/sub-trillion parameter models? I'd love to understand your need behind your request.

Just to add—your question comes off a bit confrontational.

There could be a number of valid reasons for such a request. I can't speak for the OP or others in the thread, but here are my personal reasons:

1. I'm interested in seeing how OpenAI stacks up against other models—both top-tier ones (via other APIs) and smaller/local models.

1. I work at a company that provides its employees with an internal LLM API, which routes requests through OpenRouter to a mix of models, including OpenAI's. Since we already have a budget set aside for OpenRouter, management prefers to stick with that for now rather than adding a separate OpenAI-specific expense.

I strongly recommend you make the base URL configurable, which would immediately open up this application to being something people can use with any open model, including DeepSeek, Qwen, your own open model running in a LAN that you plan on releasing this year.

I plan on creating a Python or Lua fork of this Codex CLI with the Apache licence attached so equivalent functionality can be used from Vim and Neovim directly without a web browser or any JS required.

This is an awesome project and I look forward to submitting PRs in future as I explore producing a fork. Thank you all for making it easier to automate away boilerplate code and making working on software more fun than ever.

The best current way to do this is to use LiteLLM Proxy. You can quickly spin one up locally. https://docs.litellm.ai/docs/simple_proxy All you have to do is set OPENAI_BASE_URL to your local LiteLLM proxy and then you can set OPENAI_API_KEY to your LiteLLM proxy key. Voila, you can configure any model you want from any provider that supports Responses API (currently only directly to OpenAI, but should be available for other providers soon) and codex works!

i running my litellm server in my local
but i am getting 404 error

!Image

@rlaneth Thank you for providing your personal reasons for this feature.
That said, I regard your and @nivzx thumbs-down as disrespectful. A thumbs-down is appropriate only in case of a voting or saying something wrong or technically inaccurate, etc. Simply voting down a sincere on-topic question is plain rude.

It seesm like codex has supported DeepSeek already?

It works for me like below:

codex -p deepseek

I didn't set OPENAI_API_KEY, on the contrary, I set DEEPSEEK_API_KEY, and it indeed works properly.

<img width="1433" alt="Image" src="https://github.com/user-attachments/assets/273d7932-fe45-41a5-93d9-1a12eb8b9edd" />

@MSDimos how did you change the model name for the codex, I have set the api for DEEPSEEK_API_KEY yet the model hasn't changed

!Image

@nb0309

codex -m deepseek-chat -p deepseek

If it doesn't work, try to update to the latest version. There is still a lot of work for improvement in the code support for deepseek.

Hey everyone, I am currently trying to use a self hosted deepseek model within codex, so I have set myself up with a devcontainer containing everything i need (node, npm, litellm, openai, ... ). If this issue is to litellm related for this thread I am sorry.

i have configured litellm like that:

model_list:
  - model_name: deepseek-70b
    litellm_params:
      provider: openai
      model: openai/deepseek-ai/DeepSeek-R1-Distill-Llama-70B
      api_base: https://our_inference_ip:30000/v1    
      api_key: os.environ/OPENAI_API_KEY

litellm_settings:
  ssl_verify: "/etc/ssl/certs/ca-certificates.crt"
  drop_params: true

in devcontainer.json:

  "containerEnv": {
    "OPENAI_BASE_URL": "http://0.0.0.0:4000",
    "OPENAI_API_KEY": "sk-test"
  }

our model can only be reached with a certificate. I have added the cert file within the container, so that it is possible for me to curl for a response from litellm like this:

curl -v -H "Authorization: Bearer sk-test" -H "Content-Type: application/json" -d '{"model":"deepseek-70b","messages":[{"role":"user","content":"2+2"}]}' http://localhost:4000/v1/chat/completions

I am able to receive answers form this endpoint. Here comes the codex related problem:
remember the base_url and api_key are already set. I run the following command:
codex --provider openai --model deepseek-70b -q "what is 2+2?"

i receive:

codex --provider openai --model deepseek-70b -q "what is 2+2?"
{"role":"user","content":[{"type":"input_text","text":"what is 2+2?"}],"type":"message"}
{"id":"error-1745578428541","type":"message","role":"system","content":[{"type":"input_text","text":"⚠️ OpenAI rejected the request. Error details: Status: 404, Code: 404, Type: unknown, Message: 404 litellm.NotFoundError: NotFoundError: OpenAIException - {\"detail\":\"Not Found\"}. Please verify your settings and try again."}]}

on my litellm side of things i receive this error:
httpx.HTTPStatusError: Client error '404 Not Found' for url 'https://10.244.192.53:30000/v1/responses'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404
litellm.llms.base_llm.chat.transformation.BaseLLMException: {"detail":"Not Found"}
litellm.exceptions.NotFoundError: litellm.NotFoundError: NotFoundError: OpenAIException - {"detail":"Not Found"}

the question is "why is codex looking for /responses endpoint?" can i somehow provide that?

looks like litellm would have to allow for the new /responses endpoint. We may have to wait for an update or fix it ourselves....In the meantime, I'll look at the endpoint call functions and see how difficult that would be to add another endpoint

There is experimental support for /responses endpoint in a litellm release on this issue

There is also discussion pertaining to responses api in the vllm issue here

I would like pointers on how I can retroactively patch either or both layers to support codex, in the event this proves counterintuitive and the recommended way forward is to try and use /v1/responses I am very hesitant that effort spent there is basically useless because even if support lands in vllm I don't think it'll be trivial to finetune any B parameter models to work easily with /v1/responses because I am naively suspecting that the interfaces would have to be trained significantly by some big player with GPUs before it becomes effective again accross open source's definition of "open ai compatible api endpoint" (this is probably very naive, and potentially wrong, maybe vllm internally does all the logit stuff at a more granular level and the differences in /v1/responses and /v1/chat/completions is only cosmetic and purely app layer)

I am very interested and keen on seeing this through and would like any pointers to get me started on the right angle of approach

Here's the set up for custom/local providers you need that support the (regular) OpenAI Completion API (not responses):

1. Define your provider in ~/.codex/config.json
2. export the provider_API_KEY env
3. Change the OPENAI_BASE_URL env

<img width="453" alt="Image" src="https://github.com/user-attachments/assets/08ee8e00-5f56-4aae-bef9-6f8365c7f761" />

<img width="947" alt="Image" src="https://github.com/user-attachments/assets/a91d00ce-58d3-40ee-a84f-8c6811ae56fa" />

@dnakov how is state managed?
(And what about responses features, like web search, and upcoming MCP support?)

@dnakov how is state managed? (And what about responses features, like web search, and upcoming MCP support?)

Just a pretty basic map:
https://github.com/openai/codex/blob/main/codex-cli/src/utils/responses.ts#L165

(And what about responses features, like web search, and upcoming MCP support?)

I haven't tried web search, but it should work as its just defined as a tool call

It's worth noting that I wrote a middleware with state management. (So you work with responses and it operates any Chat Completions AI Provider)

https://github.com/teabranch/open-responses-server

I'm about to finish adding MCPs support.