Performance of newer versions of codex with gpt-oss has fallen off a cliff

The overwhelming majority of codex users are using the latest frontier models, so that's where almost all of the team's attention is focused. We do little or no benchmarking or testing with older models.

I'm somewhat surprised to hear that you're seeing such a large degradation. You hypothesized that it might be related to changes to the fallback system prompt. That's an interesting theory, and I'd be very interested to hear the result of the experiment you proposed. Depending on the results of that experiment, there are various ways we might mitigate the degradation.

One other idea that comes to mind. Codex has historically supported two different wire protocols: "chat/completions" and "responses". We recently announced that we will be discontinuing support for that older chat wire protocol. It wasn't designed for tool calls or agentic workflows, and we've struggled to keep it working as we've introduced new features. Some of the hacks we've added to keep it working could have resulted in performance degradations. If you haven't already switched to the "responses" API, give that a try.

Hello @etraut-openai, thanks for the quick response 👍

You hypothesized that it might be related to changes to the fallback system prompt. That's an interesting theory, and I'd be very interested to hear the result of the experiment you proposed. Depending on the results of that experiment, there are various ways we might mitigate the degradation.

It is reasonable to just try copying the system prompt from codex 0.55.0 to 0.73.0 and see if that restores the performance? Are there other issues I may need to look out for? (I have not built codex from source yet but hopefully that is not too hard.)

If you haven't already switched to the "responses" API, give that a try.

Unfortunately, codex 0.73.0 + llama.cpp (as of 'master' a few days old) with the "responses" API seems to be broken. I had to explicitly set wire_api = "chat" in the config.toml file to get it codex-cli 0.73.0 + llama.cpp + gpt-oss-20b working at all.

The broader question: Is OpenAI going to be supporting the usability and performance of the open-source stack codex-cli + llama.cpp + gpt-oss? (Note that codex-cli+ vllm + gpt-oss still does not work.) I mean, could OpenAI run a benchmark suite with codex-cli + llama.cpp + gpt-oss-20b (or 120b) and make sure there are no major regressions before putting out a new codex-cli release? Otherwise, what was the point of putting out the gpt-oss models if you can't actually use them in an open-source stack locally? If there is no such testing with codex-cli + llama.cpp + gpt-oss for codex releases, should we start looking at other open-source tools to effectively use the gpt-oss models and better control the system prompts and other tooling?

NOTE: The gpt-oss models are pretty impressive given their smaller size and great speed, so it would be a shame to not be able to use them for agentic coding tasks. We just don't have the option of using external models. We have to run models locally for the time being. And we can't run Chinees open-weights models either, so gpt-oss is really the only viable open-weights models to be doing anything useful right now with local models.

It is reasonable to just try copying the system prompt from codex 0.55.0 to 0.73.0 and see if that restores the performance?

Yes, that should work for experimentation purposes. You can find instructions for building codex here.

This is a rapidly evolving space, and we're focused on expanding the frontiers of agentic coding. Our priorities are guided by input from our user base, and the overwhelming majority are using the frontier models.

The gpt-oss models were released way back in August. Since then, we've released half a dozen major updates to the frontier models. Perhaps you haven't used these lately, but their coding abilities are far beyond those of just a few months ago — and significantly beyond what the gpt-oss models are capable of.

If the gpt-oss models are working well for you, here are some options:

1. You can file bugs (like this one) if you run into problems. We will try to keep gpt-oss running well, but as you can see, we have a lot of open bug reports and feature requests. We generally prioritize bug fixes based on user impact. We also accept PRs for bug fixes.
2. You can maintain your own fork of codex. We've intentionally made the code open source to allow users to adapt it for their own use cases.
3. You can pin to an older version of codex. I wouldn't generally recommend this if you were using the cloud-based models, but for local models, it's an option.

The gpt-oss models were released way back in August. Since then, we've released half a dozen major updates to the frontier models. Perhaps you haven't used these lately, but their coding abilities are far beyond those of just a few months ago — and significantly beyond what the gpt-oss models are capable of.

No question, it is just that we can't use any of those models and are stuck with open-weights models for the foreseeable future.

If the gpt-oss models are working well for you, here are some options:

I will do some more investigating and comment back here

@bartlettroscoe additional information here would be really helpful for debugging the specific failures you're seeing.

1. What config are you running with, exactly? model value, any specific features enabled/disabled?
2. Is there anything unusual about your setup? Are you just running standard ollama?
3. Do you have any diagnostics on the specific failure modes? e.g. are specific function calls failing more frequently? is the model getting stuck on a particular behavior?

additional information here would be really helpful for debugging the specific failures you're seeing.

@dylan-hurd-oai, I am going to do some more debugging today after upgrading llama-swap/llama.cpp again and report back here with more details. I can likely provide access to a pre-built container image that will reproduce what we are seeing. Stay tuned.

gpt-oss models are frontier OpenAI OSS models, since there are no updates for them. So, why they would be neglected?

Update: We have not had time to dig into debugging this issue. We are able to keep making progress for now on our short-term goals using an older version of Codex CLI (0.55.0). At some point soon, we will get back to this and try the most recent Codex CLI release and/or we may look at other open-source coding agents (like OpenCode or OpenHands) to see if they work correctly with the gpt-oss models served with llama-swap/llama.cpp.

@bartlettroscoe Note that llama.cpp recently started to partially (good enough for codex) support the completions API: https://github.com/ggml-org/llama.cpp/pull/18486

After disabling web_search (web_search = "disabled" in ~/.codex/config.toml), switching to the responses API (wire_api = "responses", and setting model_reasoning_effort to "high" everything seems to work rather well for me (small to medium sized python coding tasks).

(On the chat completions API I had to rename the "reasoning" field in codex to "reasoning_content" to get good performance with codex and llama.cpp. The naming mismatch made tool calls unreliable/looping because the model wasn't fed back its reasoning trace correctly.)

In case your performance is still degraded compared to 0.55.0 you can set a custom system/developer prompt via model_instructions_file = "~/.codex/custom_prompt.md" in ~/.codex/config.toml; Just to try different prompts than the ones that had been optimized for the commercial/closed frontier models.
Finally for debugging it might be helpful to see the model reasoning text in the transcript which can be enabled via show_raw_agent_reasoning = true in ~/.codex/config.toml.

@bfroemel, thanks for the tips.

you can set a custom system/developer prompt via model_instructions_file = "~/.codex/custom_prompt.md" in ~/.codex/config.toml; Just to try different prompts than the ones that had been optimized for the commercial/closed frontier models.

According to:

• https://developers.openai.com/codex/config-reference

it says:

| model_instructions_file | string (path) | Replacement for built-in instructions instead of AGENTS.md. | | --- | --- | --- |

That would suggest that this just replaces AGENTS.md file, not the hard-coded system prompt baked into codex?

However, then looking at:

https://github.com/openai/codex/blob/81a17bb2c1041ad1d89527911318386ce036afa9/codex-rs/core/src/config/mod.rs#L802-L806

it seems to suggest this would replace the codex system prompt. This is confusing.

I guess I can experiment and examine the actual JSON passed to the model to see what system prompt is actually used.

@bartlettroscoe I take it that you are still seeing degraded performance with latest llama.cpp & codex?

About model_instructions_file: Currently the documentation seems to be outdated/wrong. And sure, you can start llama.cpp's server with the flags --verbose --verbose-prompt and can easily verify the complete prompt/session.

@bartlettroscoe I take it that you are still seeing degraded performance with latest llama.cpp & codex?

@bfroemel, yes, I recently tried with codex-0.87.0 and seem to have similar problems. Looking at some of the traces, it seems like updated codex is getting confused with tool calling formatting returned from the llama-server endpoint. The configuration I am using for my endpoint for codex/config.toml is:

model_provider = "endpoint"

[model_providers.endpoint]
name = "endpoint"
base_url = "http://<host>:<port>/v1"
wire_api = "chat"

After disabling web_search (web_search = "disabled" in ~/.codex/config.toml), switching to the responses API (wire_api = "responses", and setting model_reasoning_effort to "high" everything seems to work rather well for me (small to medium sized python coding tasks).

Setting model_reasoning_effort to "high" was just needed for your use case, not for correct tool handling, correct?

(On the chat completions API I had to rename the "reasoning" field in codex to "reasoning_content" to get good performance with codex and llama.cpp. The naming mismatch made tool calls unreliable/looping because the model wasn't fed back its reasoning trace correctly.)

Did you have to fork the codex repo and build/install codex from your fork to fix this? I don't see a fork 'codex' under:

https://github.com/bfroemel?tab=repositories

(unless you made it private).

@bartlettroscoe I would suggest to first try the latest codex (either from source, or 0.92.0 should be fine; 0.87.0 is from 2 weeks ago ;) meaning it could work, but I didn't test it) and llama.cpp (also preferably from source, or b7870+) with the responses api (wire_api = "responses").

Only if this doesn't work (shows degraded performance), I would look into debugging (prompt replacing and looking at raw session data). As a last resort you could try patching the deprecated codex chat completions api (field "reasoning"). If you really need to patch, I'd just clone the codex repo locally, and replace the strings "reasoning" with "reasoning_content" like this:

diff --git a/codex-rs/codex-api/src/requests/chat.rs b/codex-rs/codex-api/src/requests/chat.rs
index 5c16a5fb5..a53f9af74 100644
--- a/codex-rs/codex-api/src/requests/chat.rs
+++ b/codex-rs/codex-api/src/requests/chat.rs
@@ -194,7 +194,7 @@ impl<'a> ChatRequestBuilder<'a> {
                         && let Some(reasoning) = reasoning_by_anchor_index.get(&idx)
                         && let Some(obj) = msg.as_object_mut()
                     {
-                        obj.insert("reasoning".to_string(), json!(reasoning));
+                        obj.insert("reasoning_content".to_string(), json!(reasoning));
                     }
                     messages.push(msg);
                 }
@@ -320,14 +327,14 @@ fn push_tool_call_message(messages: &mut Vec<Value>, tool_call: Value, reasoning
     {
         tool_calls.push(tool_call);
         if let Some(reasoning) = reasoning {
-            if let Some(Value::String(existing)) = obj.get_mut("reasoning") {
+            if let Some(Value::String(existing)) = obj.get_mut("reasoning_content") {
                 if !existing.is_empty() {
                     existing.push('\n');
                 }
                 existing.push_str(reasoning);
             } else {
                 obj.insert(
-                    "reasoning".to_string(),
+                    "reasoning_content".to_string(),
                     Value::String(reasoning.to_string()),
                 );
             }
@@ -343,7 +350,7 @@ fn push_tool_call_message(messages: &mut Vec<Value>, tool_call: Value, reasoning
     if let Some(reasoning) = reasoning
         && let Some(obj) = msg.as_object_mut()
     {
-        obj.insert("reasoning".to_string(), json!(reasoning));
+        obj.insert("reasoning_content".to_string(), json!(reasoning));
     }
     messages.push(msg);
 }
diff --git a/codex-rs/codex-api/src/sse/chat.rs b/codex-rs/codex-api/src/sse/chat.rs
index a3effce2e..b68ff9706 100644
--- a/codex-rs/codex-api/src/sse/chat.rs
+++ b/codex-rs/codex-api/src/sse/chat.rs
@@ -156,9 +179,36 @@ pub async fn process_chat_sse<S>(
 
         for choice in choices {
             if let Some(delta) = choice.get("delta") {
-                if let Some(reasoning) = delta.get("reasoning") {
+                if let Some(reasoning) = delta.get("reasoning_content") {
                     if let Some(text) = reasoning.as_str() {
                         append_reasoning_text(&tx_event, &mut reasoning_item, text.to_string())
                             .await;
@@ -245,7 +295,7 @@ pub async fn process_chat_sse<S>(
             }
 
             if let Some(message) = choice.get("message")
-                && let Some(reasoning) = message.get("reasoning")
+                && let Some(reasoning) = message.get("reasoning_content")
             {
                 if let Some(text) = reasoning.as_str() {
                     append_reasoning_text(&tx_event, &mut reasoning_item, text.to_string()).await;
@@ -650,7 +693,7 @@ mod tests {
             "choices": [{
                 "delta": {
                     "content": [{"text": "hi"}],
-                    "reasoning": "because",
+                    "reasoning_content": "because",
                     "tool_calls": [{
                         "id": "call_a",
                         "function": { "name": "do_a", "arguments": "{}" }

Setting model_reasoning_effort to "high" was just needed for your use case, not for correct tool handling, correct?

Yes; tool calls work also with low and medium (at least on gpt-oss-120b). btw: the responses API appears to be also more reliable than the chat completions API where I sometimes encountered up to 3 failed tool calls per 100 tool calls.

Did you have to fork the codex repo and build/install codex from your fork to fix this?

No; everything moves so fast, I didn't see a benefit to put this on github.

I figured I would give it a college try to upgrade Codex and llama.cpp and see if things worked with the updated versions of this software. I tried the most current versions codex-0.116.0 and llama-swap (which contains updated version of llama.cpp) version v198-cuda-b8496.

With that, we get the error from llama-server:

srv    operator(): got exception: {"error":{"code":400,"message":"'type' of tool must be 'function'","type":"invalid_request_error"}}
srv  log_server_r: done request: POST /v1/responses 127.0.0.1 400

which is reported back to codex and displayed as:

{"error":{"code":400,"message":"'type' of tool must be 'function'","type":"invalid_request_error"}}

It seems these are known incompatibilities:

• https://github.com/ggml-org/llama.cpp/issues/20156: Eval bug: Codex, 'type' of tool must be 'function'
• Opened 3/6/2026
• https://github.com/ggml-org/llama.cpp/issues/19343: Misc. bug: Codex broken with llamacpp backend
• Opened 2/4/2026

The only hope would be for everyone to implement the Open Responses API:

• https://www.openresponses.org/

There does seem to be a desire to make llama.cpp adopt this standard:

• https://github.com/ggml-org/llama.cpp/issues/19138: Feature Request: Support OpenAI Responses API (/v1/responses) in llama.cpp server
• https://github.com/ggml-org/llama.cpp/pull/19720: server: add OpenAI Responses API compliance

but the latter PR has not been merged yet.

NOTE: It seems that LM Studio and Ollama are being constantly patched to fix compatibility issues with llama.cpp and various coding agents, including Codex CLI.

Otherwise, you might be able to use LiteLLM to patch things to work. See:

• https://chatgpt.com/s/t_69c44e4ec040819197942ad0f612902c

But it seems that waiting for the Open Responses standard to stabilize and for llama.cpp to adopt that standard is the way to go. Anything else is likely to be a fragile hack.

I will wait to see if llama.cpp implements the Open Responses API and try this again. As long as Codex CLI sticks to that API, there is a hope for a better future where AI codes walk hand in hands and fill the world with love:

https://www.youtube.com/watch?v=1VM2eLhvsSM

👍

Until then, we are stuck with codex-0.55.0 (until we can switch over to OpenCode and/or OpenHands CLIs).

@bartlettroscoe jfyi, llama.cpp already implemented responses api, started since January with more improvements later. But codex support of non-openai providers and non-flagship models become much worse since then, so there is just no point to complain anymore.

@bartlettroscoe jfyi, llama.cpp already implemented responses api, started https://github.com/ggml-org/llama.cpp/pull/18486 with more improvements later.

@vt-alt, note that:

• https://github.com/ggml-org/llama.cpp/pull/18486

is just a partial implementation of the /v1/responses API.

Given the open issue and PR:

• https://github.com/ggml-org/llama.cpp/issues/19138
• https://github.com/ggml-org/llama.cpp/pull/19720

that suggests that llama.cpp has not yet fully implemented the Open Responses API (until that PR is merged)?

But codex support of non-openai providers and non-flagship models become much worse since then, so there is just no point to complain anymore.

Is that not the purpose of the Open Responses standard? Other than specifics about the system prompt (which is tuned for proprietary OpenAI models), if the AI coding agent applies the tool calls correctly as sent by why the model as specified by the Open Responses API, why should we expect Codex to fall off a cliff?

Are you suggesting that Codex CLI is going to go past the Open Responses API specification defined at https://www.openresponses.org/ and use non-standard API features of the OpenAI cloud API? That is their right, but it is worth asking.

Are you suggesting that Codex CLI is going to go past the Open Responses API specification defined at https://www.openresponses.org/ and use non-standard API features of the OpenAI cloud API? That is their right, but it is worth asking.

Hello OpenAI developers, is this the case? Will Codex CLI not stick with the Open Reponses API and use non-standard API features? That is fine, but it would be good to know if this is the plan. (But it does seem ironic that OpenAI's flagship open-weights models don't work with OpenAI's own AI coding agent CLI.)

NOTE: It is fine for a company to customize its CLI for its proprietary models and cloud model API. The best way to use OpenAI's models is to use Codex, that is fine and makes perfect sense. We just need clarification on if Codex CLI will stick with the Open Responses standard or not.

Please let us know.