GPT-5.5 Codex reasoning-token clustering at 516/1034/1552 may be leading to degraded performance on complex tasks

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• #29353

Powered by Codex Action

I've been facing the same issue and am very frustrated as well

meet same problem, desire openAI to feedback!

Potential duplicates detected. Please review them and close your issue if it is a duplicate. * gpt-5.5 xhigh sometimes short-circuits with reasoning_output_tokens=516 and wrong final_answer in Codex Desktop #29353 _Powered by Codex Action_

This is a much more data driven report than the previous one

> Potential duplicates detected. Please review them and close your issue if it is a duplicate. > > * gpt-5.5 xhigh sometimes short-circuits with reasoning_output_tokens=516 and wrong final_answer in Codex Desktop #29353 > > _Powered by Codex Action_ This is a much more data driven report than the previous one

I agree, the previous one was closed for no reason. I hope this takes their attention

Exactly — this matches what I found, and it's clearly not an isolated case. I shared my initial finding on Reddit earlier (Half of Your High-Stakes Codex Requests May Be Silently Downgraded by Truncated Reasoning), and it's great that you've now dug deeper with model-specific and monthly data.

I've added a link to this GitHub issue in that Reddit post, so readers can cross-reference and upvote here.

Thanks for the thorough testing!

Same issue

Same issue

same problem

Same issue

same issue

> > Potential duplicates detected. Please review them and close your issue if it is a duplicate. > > > > * gpt-5.5 xhigh sometimes short-circuits with reasoning_output_tokens=516 and wrong final_answer in Codex Desktop #29353 > > > > _Powered by Codex Action_ > > > This is a much more data driven report than the previous one I agree, the previous one was closed for no reason. I hope this takes their attention

You should post this on the official OpenAI forums too:

https://forum.openai.com/
https://community.openai.com/

Those are more likely than Github to be seen by people who actually have the authority to investigate or escalate model-quality issues internally.

same issue

same issue

Adding this instruction to AGENTS.md can alleviate this issue for 5.5xhigh:
Spend time on thinking; you do not need to use the commentary channel to report progress to me.

yes, same issue

same issue

Adding this instruction to AGENTS.md can alleviate this issue for 5.5xhigh: Spend time on thinking; you do not need to use the commentary channel to report progress to me.

The effect is very limited.
Looking forward to an official fix for this issue.

same issue

Same Issue

same issue

I'm experiencing the exact same reasoning truncation issue, which severely degrades the model's performance on my complex coding tasks.

Its completely unusable! It cannot be that I pay 200$ a month and select XHigh but it gets outperformed by a free-tier chinese model...

I dont want any resets. They are collecting dust because of this.
Stop handing them out if you cant handle the capacity.

Id rather have -50% less usage if it meant the model actually spent any time thinking at all.

Same issue

Same issue.

an effective mitigation measure is to modify the system prompt(Share intermediary updates in commentary channel.) to

• Do not send optional commentary.
• You do not need to use the commentary channel to report progress to me.
• Use tools normally.
• Put user-facing text in final only.

or directly use the gpt-5.2-codex prompt

an effective mitigation measure is to modify the system prompt(Share intermediary updates in commentary channel.) to Do not send optional commentary. You do not need to use the commentary channel to report progress to me. Use tools normally. Put user-facing text in final only. or directly use the gpt-5.2-codex prompt

system prompt does not manage juice value

Same issue, it's for $200?

Same issue, edit system prompt can relieve a bit

Because of this issue, I haven't been able to use Codex for any real work for quite a while now.
If this doesn't get fixed, I don't see any reason to keep paying for the subscription.
It's pretty disappointing that GPT-5.5 xhigh is now delivering a worse experience than some budget open-source models.

This is outright fraud!!!

Because of this issue, I haven't been able to use Codex for any real work for quite a while now. If this doesn't get fixed, I don't see any reason to keep paying for the subscription. It's pretty disappointing that GPT-5.5 xhigh is now delivering a worse experience than some budget open-source models.

YES LITERALLY THIS. Gpt-5.5xhigh thinking for only 30 seconds regularly is abysmal. Im holding for gpt5.6 and see if this issue is resolved, otherwise ill shift my team too.

The same test question, when asked in Opencode, achieves a 100% accuracy rate, but in Codex, the accuracy drops to nearly 0%. I don't know what the OpenAI team is doing. This is clearly not an issue of model intelligence—a single system prompt can ruin an entire model.

Same issue.

Same issue.

Related data point for this issue: I observed a path-dependent high rate of telemetry-reported reasoning_output_tokens=0 in Codex Desktop GUI sessions under gpt-5.5 / xhigh, while comparable CLI exec replays did not show the same zero-rate.

Environment

• Product: Codex Desktop app
• Platform: Windows
• Codex version shown in session metadata: 0.142.5
• Displayed model in turn context: gpt-5.5
• Displayed reasoning effort in turn context: xhigh
• GUI source: vscode
• CLI replay source: exec
• Observation date: 2026-07-03

I understand the GUI and CLI paths are not identical execution environments. The point is that the anomaly appears path-dependent: prominent in Desktop GUI / vscode, absent or much weaker in CLI / exec replays using the same displayed model and effort.

Broader local aggregate

Across local Codex session JSONL files, without printing prompts, assistant messages, session IDs, local paths, or filenames:

• Total token_count records scanned: 45,499
• Desktop GUI gpt-5.5 / xhigh, pre-compaction:
• Records: 6,562
• reasoning_output_tokens=0: 1,297
• Zero rate: 19.77%
• Desktop GUI gpt-5.5 / xhigh, post-compaction:
• Records: 36,066
• reasoning_output_tokens=0: 7,680
• Zero rate: 21.29%
• Desktop GUI gpt-5.4 / xhigh, pre-compaction:
• Records: 1,762
• reasoning_output_tokens=0: 120
• Zero rate: 6.81%
• CLI exec gpt-5.5 / xhigh, pre-compaction:
• Records: 46
• reasoning_output_tokens=0: 0
• Zero rate: 0%

One GUI thread before first compaction

I focused on the segment before first compaction to reduce confounding from context summarization or replay effects.

• token_count records: 139
• reasoning_output_tokens=0: 60
• Zero rate: 43.2%

Completed-turn grouping:

• Completed turns: 115
• reasoning_output_tokens=0: 56
• Zero rate: 48.7%

CLI replay comparison:

• CLI 4-turn replay: 4 records, zero-rate 0%, values 516, 1034, 2070, 1552
• CLI 30-turn replay: 36 main token_count records, zero-rate 0%, median reasoning tokens 516, mean 885.9, max 2890
• Auxiliary completed-turn extraction found 1 zero-reasoning CLI turn, still far below the GUI zero-rate

Why this is not just short acknowledgements

For the 56 telemetry-reported zero-reasoning completed GUI turns before compaction:

• Simple acknowledgement-like turns: 0
• Short responses: 3
• Judgment / strategy / expectation-setting turns: 22
• Live situation handling turns: 8
• Investigation / meta-analysis turns: 2
• Other structural discussion turns: 21

Additional pattern:

| GUI completed turns before compaction | Count |
| --- | ---: |
| telemetry-reported reasoning_output_tokens=0 | 56 |
| zero-reasoning turns with a visible reasoning response item | 1 / 56 |
| zero-reasoning turns with tool calls | 0 / 56 |

Timing / output-size comparison:

• GUI zero-reasoning group: median input tokens 74,578; median output tokens 388.5; median assistant text length 550 chars; median duration 10.7s; median TTFT 2.6s
• GUI nonzero-reasoning group: median input tokens 83,247; median output tokens 956; median assistant text length 590 chars; median duration 22.4s; median TTFT 12.7s

The assistant text length medians are close, while timing differs sharply. Output length alone does not explain the high zero-rate.

I am not claiming that reasoning_output_tokens=0 proves no internal reasoning occurred. The issue is the large path-dependent discrepancy: GUI completed turns show a high telemetry-reported zero-rate under xhigh, while comparable CLI replays do not.

Potential areas to investigate:

• GUI-specific routing or fallback behavior
• reasoning-effort propagation failure
• response-stream / phase handling differences
• token_count telemetry under-reporting
• a valid but undocumented direct-final path for xhigh

I am not attaching raw session JSONL because it contains private conversation content, local paths, session IDs, and other user-specific data. The numbers above are aggregated from local Codex session telemetry.

I reproduced this issue using https://github.com/haowang02/codex-candy-eval.

Environment

• Plan: ChatGPT Pro (Japan)
• Client: Codex CLI
• Version:

$ codex --version
codex-cli 0.142.5

• Model: gpt-5.5
• Reasoning: xhigh
• Command:

python3 codex_candy_eval.py -m gpt-5.5 -r xhigh -n 5

Results

Run set 1:

• Correct: 2/5
• Incorrect runs:
• reasoning_output_tokens = 516, answer = 30
• reasoning_output_tokens = 516, answer = 26
• reasoning_output_tokens = 516, answer = 35
• Correct runs:
• reasoning_output_tokens = 6732, answer = 21
• reasoning_output_tokens = 3624, answer = 21

Run set 2:

• Visible completed runs: 4
• Correct: 1/4
• Incorrect runs:
• reasoning_output_tokens = 516, answer = 30
• reasoning_output_tokens = 1034, answer = 29
• reasoning_output_tokens = 516, answer = 29
• Correct run:
• reasoning_output_tokens = 4142, answer = 21

Aggregate

• Total visible runs: 9
• Correct: 3/9
• Accuracy: 33.3%
• Incorrect runs: 6/6 clustered at 516 or 1034 reasoning tokens
• Correct runs: all used substantially more reasoning tokens, from 3624 to 6732

This looks strongly correlated: low fixed reasoning-token buckets, especially 516, produce confident but incorrect answers, while successful runs consistently use much larger reasoning budgets.

This is especially concerning because the command explicitly requests gpt-5.5 with xhigh reasoning, yet many complex reasoning attempts appear to finalize after exactly 516 reasoning tokens.

Could OpenAI clarify whether 516 and 1034 are intentional reasoning budget tiers, fallback behavior, load-shedding, or a truncation/streaming bug?

Same issue. I routinely use Codex for complex bioinformatics data analysis and tool dev and get really disappointed because gpt-5.5 xhigh reasoning effort has been deteriorating for the past couple of months. Already downgraded from $200 to $100 Pro plan and now seriously considering canceling subscription.

This is really serious. Especially if this is not a bug but a feature.

I am also affected, this is not cool and will not continue the pro sub.

I am also affected, Max 20x subscription is very expensive for me, I am really disappointed if this is not a bug.

<img width="1600" height="900" alt="Image" src="https://github.com/user-attachments/assets/8f873805-3bca-4d4c-97e2-720c854505e9" />

Independent reproduction from two machines (different workloads, same ChatGPT-plan account family). I parsed local Codex session logs (~/.codex/sessions/**/*.jsonl), taking last_token_usage.reasoning_output_tokens from each token_count event with non-null info, attributed to the session's current model.

Machine A (54 sessions, Apr–Jun 2026, gpt-5.5 only):

| Model | Records | Mean reasoning tokens | exact-516 / ≥516 |
| --- | ---: | ---: | ---: |
| gpt-5.5 | 5,964 | 180 | 50.1% |

Machine B (430 sessions, May–Jul 2026, mixed models):

| Model | Records | Mean reasoning tokens | exact-516 / ≥516 |
| --- | ---: | ---: | ---: |
| gpt-5.5 | 20,219 | 170 | 47.9% |
| gpt-5.4 | 3,921 | 463 | 13.1% |
| gpt-5.4-mini | 33 | 1,659 | 7.7% (n=13, noise) |

Both machines land close to the 44.0% reported here for gpt-5.5, with gpt-5.4 elevated but ~3–4× lower — same ordering as the OP's data.

One additional observation not in the OP: the spikes form an arithmetic ladder with step 518, not just the three values reported. Top repeated exact values ≥400:

• gpt-5.5 (Machine B): 516×1089, 1034×113, 1552×44, 2070×13
• gpt-5.4 (Machine B): 516×106, 3106×7, 1034×6, 1552×5, 2070×4, 2588×4
• gpt-5.5 (Machine A): 516×387, 1034×44, 1552×11, 2070×8

516, 1034, 1552, 2070, 2588, 3106 — six rungs, each exactly 518 apart. A naturally varying reasoning-token distribution doesn't produce an evenly spaced comb like this; it looks like reasoning is being terminated at quantized block boundaries (516 + n×518), consistent with the thresholded reasoning-budget hypothesis.

Also matching the OP's intensity observation: on the same machine and overlapping tasks, gpt-5.5's mean reasoning tokens (170) is roughly a third of gpt-5.4's (463).

Analysis script (stdlib-only Python) available if useful for others to reproduce.

affected here too with candy eval:

Shot | reasoning_output_tokens | final answer
-- | -- | --
1 | 516 | 21
2 | 516 | 24, wrong
3 | 516 | 18, wrong
4 | 516 | 25, wrong
5 | 516 | 18, wrong

I'm affected too with 100$ plan

5-shot results:

┌─────┬─────────────────────────┬──────────────┐
│ Run │ reasoning_output_tokens │ final answer │
├─────┼─────────────────────────┼──────────────┤
│ 1 │ 516 │ 21 ✅ │
├─────┼─────────────────────────┼──────────────┤
│ 2 │ 516 │ 23 ❌ │
├─────┼─────────────────────────┼──────────────┤
│ 3 │ 516 │ 27 ❌ │
├─────┼─────────────────────────┼──────────────┤
│ 4 │ 516 │ 16 ❌ │
├─────┼─────────────────────────┼──────────────┤
│ 5 │ 516 │ 23 ❌ │
└─────┴─────────────────────────┴──────────────┘

I'm affected too with 2 x 200$ Plan..
Is the freeride finally over?

Reproduces here with gpt-5.5 but all runs get the correct answer with gpt-5.4.

user-vV3XhezUrPWzLBc4F3sjbF9C

Confirmed here too, it's been getting really bad the last two days.

  - run1: 23 with 516 (WRONG)
  - run2: 26 with 516 (WRONG)
  - run3: 21 with 8286 (CORRECT)
  - run4: 24 with 516 (WRONG)
  - run5: 29 with 516 (WRONG)

Thank you OpenAI!

GPT 5.4

  - run1: 21 with 6,507 (CORRECT)
  - run2: 21 with 11,010 (CORRECT)
  - run3: 21 with 7,581 (CORRECT)

Note: Codex/GPT works great for my workflows overall, and I am not reporting this because I am personally stuck. I just have a fairly large local session-log dataset, so I am sharing the aggregates in case they are useful for debugging.

Summary:

• Sessions represented: 5,161
• All exact reasoning_output_tokens = 516 events: 3,426
• gpt-5.5 share of all records: 66.6%
• gpt-5.5 share of exact-516 events: 84.8%
• gpt-5.5 exact 516 / >=516: 21.7%
• non-gpt-5.5 exact 516 / >=516: 9.1%

Model-level stats:

| Model | Records | Sessions | Mean reasoning tokens | P90 | Exact 516 | >=516 | Exact 516 / >=516 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| gpt-5.5 | 125,900 | 4,225 | 191.1 | 516 | 2,906 | 13,365 | 21.7% |
| gpt-5.4 | 19,266 | 434 | 326.3 | 834 | 519 | 3,561 | 14.6% |
| gpt-5.3-codex | 15,869 | 367 | 256.4 | 704 | 1 | 2,130 | 0.0% |
| codex-auto-review | 27,867 | 136 | 14.0 | 30 | 0 | 2 | 0.0% |

Effort-level stats:

| Effort | Model | Records | Sessions | Mean reasoning tokens | Exact 516 | >=516 | Exact 516 / >=516 |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
| xhigh | gpt-5.5 | 125,854 | 4,223 | 191.2 | 2,905 | 13,363 | 21.7% |
| medium | gpt-5.5 | 46 | 2 | 62.0 | 1 | 2 | 50.0% |
| xhigh | gpt-5.4 | 18,841 | 268 | 330.5 | 511 | 3,536 | 14.5% |
| medium | gpt-5.4 | 425 | 166 | 142.9 | 8 | 25 | 32.0% |
| low | codex-auto-review | 27,867 | 136 | 14.0 | 0 | 2 | 0.0% |

Fixed-value counts:

| Model | =516 | =1034 | =1552 | =2070 | =2588 | =3106 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| gpt-5.5 | 2,906 | 574 | 214 | 106 | 48 | 28 |
| gpt-5.4 | 519 | 90 | 35 | 19 | 24 | 6 |
| gpt-5.3-codex | 1 | 2 | 0 | 0 | 0 | 0 |

Monthly deduped gpt-5.5 stats:

| Month | Records | Sessions | Mean reasoning tokens | P90 | Exact 516 | >=516 | Exact 516 / >=516 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 2026-04 | 21,295 | 33 | 120.2 | 381 | 565 | 1,494 | 37.8% |
| 2026-05 | 94,415 | 3,602 | 203.3 | 516 | 1,933 | 10,433 | 18.5% |
| 2026-06 | 10,190 | 594 | 226.7 | 531 | 408 | 1,438 | 28.4% |

One split that may be useful for internal debugging:

| Originator | Model | Records | Sessions | Exact 516 | >=516 | Exact 516 / >=516 |
| --- | --- | ---: | ---: | ---: | ---: | ---: |
| codex-tui | gpt-5.5 | 52,476 | 61 | 1,535 | 3,826 | 40.1% |
| codex_exec | gpt-5.5 | 73,424 | 4,164 | 1,371 | 9,539 | 14.4% |

My results (pro plan; 5.5 high):

run1: 21 with 3372 (CORRECT)
run2: 21 with 516 (CORRECT)
run3: 26 with 516 (WRONG)
run4: 23 with 516 (WRONG)
run5: 21 with 516 (CORRECT)

Exactly the samenproblem on xhigh 5.5

5.4 for some reason doesnt face the same problem for me

Adding another local aggregate. This reproduces in my Codex session telemetry, using only aggregate metadata and not publishing raw session JSONL.

Environment / source:

• Product paths represented: Codex Desktop, codex-tui, codex_exec, subagents
• Plan type observed in token_count metadata: pro
• Local data source: ~/.codex/sessions/**/*.jsonl plus ~/.codex/archived_sessions/*.jsonl
• Window represented by local files: May-Jul 2026
• Scanner used event_msg.payload.type == "token_count" and payload.info.last_token_usage.reasoning_output_tokens
• I skipped payload.info = null, and did not use total_token_usage.reasoning_output_tokens because it is cumulative within a session

Methodology notes:

• JSONL files scanned: 1,615
• Token records after dedupe: 204,959
• Missing/null usage records skipped: 114
• Duplicate metadata records dropped: 16
• Model/effort attribution used the latest preceding turn_context in the same JSONL stream
• No prompts, assistant messages, session IDs, thread names, file names, or local paths are included below

Overall result:

| Scope | Records | Sessions | Mean reasoning | P90 | =516 | >=516 | =516 / >=516 | =1034 | =1552 | =2070 | =2588 | =3106 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| all parsed records | 204,959 | 1,425 | 139.5 | 416 | 4,937 | 15,686 | 31.47% | 566 | 235 | 97 | 44 | 21 |

By model:

| Model | Records | Sessions | Mean reasoning | P90 | =516 | >=516 | =516 / >=516 | =1034 | =1552 | =2070 | =2588 | =3106 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| gpt-5.5 | 203,215 | 1,399 | 138.5 | 414 | 4,934 | 15,460 | 31.91% | 566 | 235 | 96 | 44 | 21 |
| gpt-5.3-codex-spark | 1,682 | 20 | 252.9 | 601 | 0 | 217 | 0.00% | 0 | 0 | 1 | 0 | 0 |
| gpt-5.4 | 61 | 5 | 259.4 | 516 | 3 | 9 | 33.33% | 0 | 0 | 0 | 0 | 0 |

The gpt-5.4 sample is too small in my local data to treat as a useful control, but the gpt-5.3-codex-spark contrast is at least directionally consistent with the model-specific pattern reported above.

By effort for gpt-5.5:

| Model / effort | Records | Sessions | Mean reasoning | P90 | =516 | >=516 | =516 / >=516 | =1034 | =1552 | =2070 | =2588 | =3106 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| gpt-5.5 / xhigh | 198,922 | 1,181 | 138.9 | 415 | 4,829 | 15,166 | 31.84% | 549 | 230 | 91 | 44 | 19 |
| gpt-5.5 / high | 2,348 | 105 | 153.2 | 406 | 40 | 198 | 20.20% | 10 | 5 | 5 | 0 | 2 |
| gpt-5.5 / medium | 1,916 | 119 | 86.1 | 235 | 64 | 95 | 67.37% | 7 | 0 | 0 | 0 | 0 |

Monthly gpt-5.5 / xhigh:

| Month | Records | Sessions | Mean reasoning | P90 | =516 | >=516 | =516 / >=516 | =1034 | =1552 | =2070 | =2588 | =3106 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 2026-05 | 75,996 | 374 | 133.0 | 400 | 1,929 | 5,464 | 35.30% | 196 | 72 | 24 | 14 | 8 |
| 2026-06 | 118,959 | 705 | 138.6 | 417 | 2,738 | 9,188 | 29.80% | 332 | 149 | 54 | 23 | 10 |
| 2026-07 | 3,967 | 105 | 261.2 | 516 | 162 | 514 | 31.52% | 21 | 9 | 13 | 7 | 1 |

Origin/source split for all models:

| Origin/source | Records | Sessions | Mean reasoning | P90 | =516 | >=516 | =516 / >=516 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Codex Desktop / redacted source category | 85,981 | 406 | 114.1 | 330 | 1,507 | 5,057 | 29.80% |
| Codex Desktop / vscode | 48,958 | 162 | 152.0 | 466 | 1,329 | 4,278 | 31.07% |
| codex-tui / cli | 36,640 | 98 | 140.0 | 442 | 1,038 | 3,036 | 34.19% |
| codex-tui / redacted source category | 23,483 | 290 | 160.0 | 446 | 711 | 1,891 | 37.60% |
| Codex Desktop / exec | 9,456 | 437 | 247.5 | 617 | 342 | 1,381 | 24.76% |
| codex_exec / exec | 402 | 40 | 290.8 | 516 | 9 | 41 | 21.95% |

I also checked logs_2.sqlite and session_index.jsonl as a cross-check. The session index is metadata-only, and the SQLite table contains duplicated trace fragments / estimated_token_count strings rather than structured event_msg.payload.type == "token_count" records, so I am not using SQLite as evidence for the clustering claim.

The main local takeaway is that the exact-value comb is present here too, with gpt-5.5 / xhigh showing 4,829 exact-516 records and additional rungs at 1034, 1552, 2070, 2588, and 3106. That looks consistent with the thresholded/quantized reasoning-budget hypothesis in the issue description.

Just be Open about it 😉 and give us our money back.

Now we know where these huge inference optimizations came from

It looks like 5.5 passes every time if you add Use maximum effort as principal software architect to think and reason for at least 60s. to the beginning of the prompt.

I analyzed my codex conversation history and found that the reasoning stops (specifically at 516, 1034, and 1552 .... 516+518*k) are strongly correlated with the prompt-cache ratio.

Adding another independent local aggregate. This reproduces the fixed-value comb in my local Codex telemetry, and I included a cache-ratio split because a recent comment suggested prompt-cache ratio may be relevant.

Environment / source:

• Product/version: Codex Desktop 26.623.101652, codex-cli 0.142.5
• Local data source: ~/.codex/sessions/**/*.jsonl plus ~/.codex/archived_sessions/**/*.jsonl
• Window represented by local files: May-Jul 2026
• Plan type observed in token_count.rate_limits.plan_type: pro on 52,380 records; unavailable/None on 5,433 records
• Scanner used event_msg.payload.type == "token_count" and payload.info.last_token_usage.reasoning_output_tokens
• I skipped null/empty info, and did not use total_token_usage.reasoning_output_tokens because it is cumulative within a session
• Model/effort attribution used the latest preceding turn_context in the same JSONL stream
• No prompts, assistant messages, session IDs, thread names, local paths, file names, or cwd values are included below

Methodology notes:

• JSONL files scanned: 161
• Files with token records: 160
• Token records after raw duplicate-line dedupe: 57,813
• Missing/null usage records skipped: 1
• Duplicate raw records dropped: 5
• All parsed token records in this local window were attributed to gpt-5.5; xhigh dominates the sample, so this is useful as an affected-sample aggregate but not as a local cross-model control.

Overall result:

| Scope | Records | Sessions | Mean reasoning | P90 | =516 | >=516 | =516 / >=516 | =1034 | =1552 | =2070 | =2588 | =3106 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| all parsed records | 57,813 | 160 | 134.0 | 432 | 1,832 | 4,690 | 39.06% | 165 | 50 | 15 | 9 | 3 |
| gpt-5.5 / xhigh | 57,607 | 149 | 134.0 | 432 | 1,829 | 4,679 | 39.09% | 164 | 50 | 15 | 9 | 3 |

Monthly gpt-5.5 / xhigh:

| Month | Records | Sessions | Mean reasoning | P90 | =516 | >=516 | =516 / >=516 | =1034 | =1552 | =2070 | =2588 | =3106 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 2026-05 | 23,269 | 34 | 89.3 | 261 | 269 | 1,007 | 26.71% | 31 | 12 | 3 | 2 | 0 |
| 2026-06 | 30,465 | 107 | 164.4 | 516 | 1,367 | 3,271 | 41.79% | 123 | 30 | 10 | 7 | 3 |
| 2026-07 | 3,873 | 11 | 163.2 | 516 | 193 | 401 | 48.13% | 10 | 8 | 2 | 0 | 0 |

Cache-ratio split for gpt-5.5 / xhigh, using last_token_usage.cached_input_tokens / input_tokens from the same token_count record:

| Cached/input bin | Records | =516 | >=516 | =516 / >=516 | Mean reasoning | P90 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| 0% | 2 | 0 | 0 | n/a | 196.0 | 327 |
| >0-25% | 1,038 | 281 | 348 | 80.75% | 326.2 | 516 |
| 25-50% | 360 | 108 | 136 | 79.41% | 362.3 | 516 |
| 50-75% | 851 | 166 | 262 | 63.36% | 325.2 | 576 |
| 75-90% | 3,908 | 297 | 545 | 54.50% | 207.2 | 516 |
| >=90% | 50,826 | 977 | 3,388 | 28.84% | 121.3 | 363 |
| unknown / zero input | 622 | 0 | 0 | n/a | 0.0 | 0 |

Local takeaway: the exact-value comb is present here too, with gpt-5.5 / xhigh showing 1,829 exact-516 records and additional rungs at 1034, 1552, 2070, 2588, and 3106. The cache-ratio split does not by itself prove causality, but it may be useful for internal debugging: in this sample, exact-516 concentration among records with reasoning_output_tokens >= 516 is highest in the lower/mid cached-input bins, while the high-cache bin still contributes the most absolute volume because it dominates the dataset.

Independent local telemetry result (Codex CLI)

I ran an aggregate analysis over my local Codex session JSONL logs:

• 72,625 reasoning_output_tokens records
• 5,407 records attributed to gpt-5.5
• 67,042 records attributed to gpt-5.3-codex

Exact-516 clustering

| Group | Exact 516 | Records >=516 | Exact 516 / >=516 |
|------------------------|-----------|---------------|-------------------|
| gpt-5.5 | 62 | 349 | 17.77% |
| All other model labels | 184 | 7,565 | 2.43% |

So in my local data, gpt-5.5 is 7.3× more likely than the other models combined to land exactly at reasoning_output_tokens = 516, conditional on reaching at least 516 tokens.

Reasoning-token intensity

| Model | Records | Mean reasoning tokens | P90 |
|-----------------|---------|-----------------------|-----|
| gpt-5.3-codex | 67,042 | 220.4 | 565 |
| gpt-5.5 | 5,407 | 128.0 | 370 |

gpt-5.5 also used substantially fewer reasoning tokens overall in this dataset.

Additional observation

reasoning_output_tokens = 0 appeared in:

• gpt-5.5: 975 / 5,407 (18.0%)
• gpt-5.3-codex: 6,467 / 67,042 (9.6%)

Adding another local aggregate, using only Codex token metadata. This reproduces the fixed-value clustering pattern, especially for gpt-5.5.

I am not publishing raw session JSONL, prompts, assistant messages, session IDs, thread titles, file names, cwd/local paths, account identifiers, or machine identifiers below.

Environment / source:

• Product: Codex local session telemetry
• Currently installed CLI: codex-cli 0.142.5
• Historical logs span multiple Codex CLI versions
• Local window represented by parsed token records: 2026-02 through 2026-07
• Scanner used event_msg.payload.type == "token_count" and payload.info.last_token_usage.reasoning_output_tokens
• I skipped payload.info = null
• I did not use total_token_usage.reasoning_output_tokens, because that field is cumulative within a session

Methodology notes:

• JSONL files scanned: 4,487
• Files with token_count events: 3,820
• Raw token_count events: 182,091
• Null info records skipped: 1,532
• Parsed token records after exact raw-line dedupe: 180,559
• Exact duplicate raw token records dropped: 0
• Sessions represented after parsing: 3,498
• Non-OpenAI/custom model rows are omitted from the model table below because they are not useful controls for this issue.

Overall:

| Scope | Records | Sessions | Mean reasoning | P90 | =516 | >=516 | =516 / >=516 | =1034 | =1552 | =2070 | =2588 | =3106 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| all parsed records | 180,559 | 3,498 | 108.8 | 321 | 3,183 | 9,506 | 33.48% | 233 | 64 | 10 | 9 | 6 |

OpenAI model rows:

| Model | Records | Sessions | Mean reasoning | P90 | =516 | >=516 | =516 / >=516 | =1034 | =1552 | =2070 | =2588 | =3106 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| gpt-5.5 | 149,963 | 3,228 | 89.1 | 276 | 3,160 | 6,304 | 50.13% | 230 | 61 | 10 | 9 | 6 |
| gpt-5.3-codex | 28,453 | 242 | 219.2 | 557 | 9 | 3,165 | 0.28% | 3 | 3 | 0 | 0 | 0 |
| gpt-5.4 | 90 | 4 | 280.3 | 831 | 3 | 20 | 15.00% | 0 | 0 | 0 | 0 | 0 |

The local gpt-5.4 sample is too small to use as a strong control, but the gpt-5.3-codex contrast is large in this dataset: gpt-5.5 has a much lower P90 reasoning count and a much higher exact-516 share among records with reasoning_output_tokens >= 516.

gpt-5.5 by effort:

| Model / effort | Records | Sessions | Mean reasoning | P90 | =516 | >=516 | =516 / >=516 | =1034 | =1552 | =2070 | =2588 | =3106 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| gpt-5.5 / high | 98,533 | 1,987 | 96.4 | 310 | 2,462 | 4,805 | 51.24% | 196 | 55 | 6 | 7 | 6 |
| gpt-5.5 / medium | 48,468 | 1,166 | 67.6 | 194 | 526 | 1,094 | 48.08% | 23 | 2 | 1 | 0 | 0 |
| gpt-5.5 / xhigh | 2,637 | 54 | 216.9 | 516 | 172 | 405 | 42.47% | 11 | 4 | 3 | 2 | 0 |

Monthly gpt-5.5:

| Month | Records | Sessions | Mean reasoning | P90 | =516 | >=516 | =516 / >=516 | =1034 | =1552 | =2070 | =2588 | =3106 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 2026-04 | 1,624 | 62 | 120.3 | 358 | 55 | 101 | 54.46% | 1 | 1 | 0 | 0 | 0 |
| 2026-05 | 73,818 | 1,509 | 97.3 | 310 | 1,829 | 3,713 | 49.26% | 153 | 45 | 8 | 8 | 5 |
| 2026-06 | 58,856 | 1,260 | 75.5 | 223 | 888 | 1,763 | 50.37% | 57 | 11 | 1 | 0 | 1 |
| 2026-07 | 15,665 | 397 | 98.5 | 312 | 388 | 727 | 53.37% | 19 | 4 | 1 | 1 | 0 |

I also checked cached-input ratio for the largest local gpt-5.5 slice (high). The exact-516 pattern is not confined to one cache-ratio bucket:

| Cached/input bin | Records | Sessions | Mean reasoning | P90 | =516 | >=516 | =516 / >=516 | =1034 | =1552 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 0% | 269 | 212 | 147.2 | 263 | 2 | 5 | 40.00% | 0 | 0 |
| >0-25% | 2,648 | 1,596 | 246.0 | 516 | 220 | 296 | 74.32% | 5 | 0 |
| 25-50% | 795 | 602 | 222.5 | 516 | 83 | 96 | 86.46% | 0 | 0 |
| 50-75% | 1,634 | 981 | 198.8 | 516 | 92 | 173 | 53.18% | 10 | 1 |
| 75-90% | 7,076 | 1,765 | 139.7 | 396 | 215 | 463 | 46.44% | 23 | 4 |
| >=90% | 85,659 | 1,913 | 85.5 | 272 | 1,850 | 3,772 | 49.05% | 158 | 50 |

Local takeaway: the exact-value comb is present here too. In this local dataset, gpt-5.5 shows 3,160 exact-516 records and visible additional rungs at 1034, 1552, 2070, 2588, and 3106. The strongest contrast is against gpt-5.3-codex, where exact 516 among >=516 records is 0.28% versus 50.13% for gpt-5.5.

This does not prove the underlying cause, but it is consistent with the fixed-threshold / quantized reasoning-budget hypothesis described in the issue.

Cross-client data point, since everything posted so far is Codex-only. Not blocked by this myself, just had the logs sitting there.

Caveat: I only checked whether reasoning tokens land on exact values like 516, I didn't grade any answers for correctness. So unlike hratio/jake-levy's repeated-run tests, I can't say this actually produced worse output myself, just that the same clustering shows up outside Codex too.

Setup: I use both Codex CLI and OpenCode against a mix of models. Checked exact reasoning/reasoning_output_tokens = 516 in both tools' local session stores. Aggregate metadata only, no prompts, no session/account/file info.

Codex (~/.codex/sessions + archived_sessions, ~22.7k token_count events):

| Model | records | >=516 | exact 516 | ratio |
|---|---:|---:|---:|---:|
| gpt-5.5 | 4,300 | 156 | 88 | 56.4% |
| gpt-5.4 | 7,579 | 270 | 90 | 33.3% |
| gpt-5.4-mini | 718 | 114 | 35 | 30.7% |
| gpt-5.2 | 509 | 65 | 3 | 4.6% |
| gpt-5.3-codex | 9,463 | 799 | 3 | 0.4% |

OpenCode (opencode.db, per-message tokens.reasoning, ~32.1k assistant messages, several non-OpenAI models mixed in):

| Model | records | >=516 | exact 516 | ratio |
|---|---:|---:|---:|---:|
| gpt-5.5 | 6,977 | 126 | 90 | 71.4% |
| gpt-5.2 | 177 | 11 | 0 | 0% |
| every non-OpenAI model I have volume on (Kimi, DeepSeek, MiMo, MiniMax, Gemini, Qwen, GLM) | ~24k combined | ~450 | 0 | 0% |

gpt-5.5 clusters at exactly 516 in both clients, nothing else does, in codebases with nothing in common. Doesn't prove what's causing it, but it does push against "Codex's client code is doing something weird" as the explanation.

I could not exactly repro this on Windows 11 with the latest production build of Codex, running via remote-ssh to a Linux box due to API limitations but the evidence I could collect strongly suggests I am running into the same issue intermittently and unpredictably:

Environment:

• Product: Codex Desktop
• Model: gpt-5.5
• Reasoning effort: xhigh
• Date tested: 2026-07-05
• Test setup: 5 fresh projectless Codex Desktop threads
• Prompt: same candy prompt from #29353, without adding the expected answer

Results:

| Run | Visible assistant phases | Duration | Final answer | Result |
|---|---:|---:|---:|---|
| 1 | commentary, final_answer | 52s | 21 | correct |
| 2 | commentary, final_answer | 71s | 21 | correct |
| 3 | final_answer only | 15s | 29 | wrong |
| 4 | commentary, commentary, final_answer | 152s | 21 | correct |
| 5 | multiple commentary, final_answer | 198s | 21 | correct |

So I reproduced the same suspicious behavioral split described in #29353:

• the fast direct-final_answer run gave the blind-draw answer 29
• the slower runs that emitted commentary first gave the touch-aware answer 21

The correct answer is 21 under the prompt wording because shapes can be distinguished by touch, so contestants can choose a planned shape split, e.g. 9 round and 12 star candies.

I could not extract exact reasoning_output_tokens for these fresh Windows-host Desktop threads from the app API I used, so I cannot claim the fresh bad run was exactly 516. But I also scanned my local ~/.codex/sessions/**/*.jsonl metadata using event_msg.payload.type == "token_count" and payload.info.last_token_usage.reasoning_output_tokens, skipping null info and not using cumulative total_token_usage.

That local aggregate does reproduce the fixed-value clustering:

| Scope | Records | Sessions | Mean reasoning | P90 | =516 | >=516 | =516 / >=516 | =1034 | =1552 | =2070 | =2588 | =3106 |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| gpt-5.5 / xhigh | 16,141 | 51 | 149.7 | 429 | 438 | 1,298 | 33.74% | 52 | 14 | 16 | 12 | 5 |

---

I am running this test on codex CLI on Linux directly with GPT-5.4 comparisons and will post when that completes.

My read is that this cross-client signal makes a Codex-only client or system-prompt explanation much less likely.

It makes me wonder whether gpt-5.5 is going through some OpenAI-side reasoning budget / routing / scheduler optimization path. Such an optimization might reduce reasoning-token usage, latency, or cost, but if exact-516 cases are more likely to produce premature final answers or unstable results, then the optimization may not be fully quality-neutral.

Following up: I was able to reproduce this locally in Codex Linux CLI.

Environment:

• Date tested: 2026-07-05
• Product: Codex CLI
• Version: codex-cli 0.142.5
• Platform: Ubuntu Linux, 6.8.0-111-generic, x86_64
• Node: v24.14.0
• Auth mode: ChatGPT
• Primary model tested: gpt-5.5
• Reasoning efforts tested: xhigh, high
• Control model tested: gpt-5.4 xhigh

Prompt used was the candy prompt from the related task-level repro:

> Do not use external tools. A black bag contains candies with counts: round apple 7, round peach 9, round watermelon 8; star apple 7, star peach 6, star watermelon 4. Shape is distinguishable by touch before drawing; flavor is not. What
is the minimum number of candies to draw to guarantee having apple and peach candies of different shapes, i.e. round apple + star peach or round peach + star apple? Give reasoning and final number. The local project dir is irrelevant for
this task, do not consult it.

I independently verified the expected answer is 21 by brute-force enumeration of all possible round/star draw splits; the minimum guaranteeing condition is 9 round + 12 star = 21 total.

Command shape used:

```bash
printf '%s\n' "$prompt" | timeout 180s codex -a never exec \
--json \
--skip-git-repo-check \
--ephemeral \
-s read-only \
--disable memories \
-m gpt-5.5 \
-c 'model_reasoning_effort="xhigh"'

  Results:

  ```
 Model / effort    Run    Status    Duration    reasoning_output_tokens    Final answer    Correct?
  ━━━━━━━━━━━━━━━━  ━━━━━  ━━━━━━━━  ━━━━━━━━━━  ━━━━━━━━━━━━━━━━━━━━━━━━━  ━━━━━━━━━━━━━━  ━━━━━━━━━━
   gpt-5.5 xhigh       1         0         17s                        516    23              No
  ────────────────  ─────  ────────  ──────────  ─────────────────────────  ──────────────  ──────────
   gpt-5.5 xhigh       2         0         23s                        516    26              No
  ────────────────  ─────  ────────  ──────────  ─────────────────────────  ──────────────  ──────────
   gpt-5.5 xhigh       3         0         21s                        516    28              No
  ────────────────  ─────  ────────  ──────────  ─────────────────────────  ──────────────  ──────────
   gpt-5.5 xhigh       4       124        180s                        n/a    timed out       n/a
  ────────────────  ─────  ────────  ──────────  ─────────────────────────  ──────────────  ──────────
   gpt-5.5 xhigh       5         0         20s                        516    15              No
  ────────────────  ─────  ────────  ──────────  ─────────────────────────  ──────────────  ──────────
   gpt-5.5 high        1         0         17s                        516    22              No
  ────────────────  ─────  ────────  ──────────  ─────────────────────────  ──────────────  ──────────
   gpt-5.5 high        2         0         19s                        516    21              Yes
  ────────────────  ─────  ────────  ──────────  ─────────────────────────  ──────────────  ──────────
   gpt-5.5 high        3         0         18s                        516    27              No
  ────────────────  ─────  ────────  ──────────  ─────────────────────────  ──────────────  ──────────
   gpt-5.4 xhigh       1         0        126s                       6211    21              Yes
  ────────────────  ─────  ────────  ──────────  ─────────────────────────  ──────────────  ──────────
   gpt-5.4 xhigh       2         0        237s                      12274    21              Yes
  ────────────────  ─────  ────────  ──────────  ─────────────────────────  ──────────────  ──────────
   gpt-5.4 xhigh       3         0        208s                      10876    21              Yes

Observations:

• I reproduced the exact 516 reasoning-token clustering on gpt-5.5.
• On gpt-5.5 xhigh, every completed run hit exactly 516 reasoning tokens and returned a wrong answer.
• gpt-5.5 high also hit exactly 516 on all three runs; one answer was correct, two were wrong.
• The gpt-5.4 xhigh control did not show the 516-token behavior in this sample. It used thousands of reasoning tokens, took much longer, and returned the correct answer in all 3 runs.
• This supports the issue’s narrower claim that gpt-5.5 in Codex can fall into a fixed 516-token path correlated with degraded task quality.

Workaround: a local proxy that detects the 518n−2 truncation and auto-continues the reasoning

While waiting for an upstream fix, I've been running a small local Responses proxy in front of Codex that mitigates this in practice: codexcomp (MIT, uv tool install codexcomp). Mechanism idea credit: neteroster/CodexCont; this is an independent implementation.

How it works, per round:

1. Detect — a round ending with reasoning_tokens == 518·n − 2 (516, 1034, 1552, …) is treated as truncated.
2. Continue — the tentative output is discarded; the round's reasoning items (incl. encrypted_content) are replayed as the next input together with a phase:"commentary" "Continue thinking..." message.
3. Fold — all rounds are folded into a single downstream response, so Codex sees one complete, untruncated answer.

Wiring uses the official top-level openai_base_url key (#16719), so the built-in openai provider stays in place — session grouping, remote compaction, and remote-control keep working:

# ~/.codex/config.toml (top level)
openai_base_url = "http://127.0.0.1:8787/v1"

A real fold from my logs (two consecutive 516s, final answer correct):

round 1: in=21550 out=664 reason=516 total=22214 | n=1 -> continue
round 2: in=22078 out=652 reason=516 total=22730 | n=1 -> continue
round 3: in=22606 out=566 reason=291 total=23172 | -> clean
done: 3 round(s) | status=completed stop=natural

Caveats, to be clear: this is unofficial and relies on non-contract upstream behavior; continuation rounds spend extra real tokens (bounded by an n window and a 3-continuation cap; true usage is reported under metadata.proxy_billed_usage). Loopback-only, pure auth passthrough — it never reads or stores credentials.

Hope this helps others until the root cause is fixed upstream.

<img width="2229" height="1623" alt="Image" src="https://github.com/user-attachments/assets/f688e045-1f3d-4166-8261-cae4b0bbfb55" />

Same issue.

Same Here

• Repo: https://github.com/haowang02/codex-candy-eval
• Script prompt: CODEX_PROMPT from codex_candy_eval.py
• Models: gpt-5.5, gpt-5.4
• Efforts: low, medium, high, xhigh
• Runs per bucket: n=5
• Total: 8 buckets / 40 runs
• Correctness rule: the script marks a run correct if the final answer contains an independent 21
• Final answer raw values below are the extracted final numeric answers, in run order #1 ~ #5

Codex app command shape:

codex exec --json \
  --skip-git-repo-check \
  --ephemeral \
  -s read-only \
  --disable memories \
  -c model_reasoning_effort={effort} \
  -m {model}

Results:

| Bucket | Correct | Accuracy | Reason Tok raw values | Final answer raw values |
| --- | ---: | ---: | --- | --- |
| gpt-5.5 low | 0/5 | 0% | 516, 516, 516, 516, 516 | 29, 29, 29, 29, 29 |
| gpt-5.5 medium | 0/5 | 0% | 516, 516, 516, 516, 516 | 29, 29, 29, 29, 29 |
| gpt-5.5 high | 0/5 | 0% | 516, 516, 516, 516, 516 | 28, 29, 29, 29, 29 |
| gpt-5.5 xhigh | 3/5 | 60% | 8804, 516, 6214, 6732, 516 | 21, 29, 21, 21, 33 |
| gpt-5.4 low | 1/5 | 20% | 1509, 469, 558, 957, 3241 | 27, 29, 29, 29, 21 |
| gpt-5.4 medium | 0/5 | 0% | 516, 506, 467, 516, 516 | 29, 29, 29, 29, 28 |
| gpt-5.4 high | 3/5 | 60% | 516, 4502, 5661, 2721, 918 | 29, 21, 21, 21, 29 |
| gpt-5.4 xhigh | 5/5 | 100% | 7001, 6336, 7029, 4660, 5165 | 21, 21, 21, 21, 21 |

Aggregate summary:

| Group | Correct | Accuracy |
| --- | ---: | ---: |
| gpt-5.5 total | 3/20 | 15% |
| gpt-5.4 total | 9/20 | 45% |
| low total | 1/10 | 10% |
| medium total | 0/10 | 0% |
| high total | 3/10 | 30% |
| xhigh total | 8/10 | 80% |

Same issue here.

I've made a passive workaround: a pair of Codex CLI hooks that detect the
truncation from the local session transcript and warn — in the TUI at
turn end, and via a message injected into the model's context on your next
message.

See https://github.com/bentoner/codex-516-hook

For other workarounds, see the comments above about CodexCont/codexcomp,
which auto-repair the truncation via a local MITM proxy.

Data notes (details, prompts, and full tables in the repo README): in our
testing a short trigger prompt truncated in 18/20 codex exec runs under
ChatGPT auth and 18/20 under API-key auth, while the same prompt as a bare
/v1/responses call (with no tools declared) truncated in 0/20 cases. So it
doesn't appear to be the billing path; it follows Codex's request shape.

@tibo-openai Why hasn’t anyone taken care of this issue?

The issue seems to be with automatic reasoning classifier which if its true, makes the low/mid/high/xhigh switcher pointless.
When i tested it on normal prompts it used 516 rtoks BUT when i added to prompt "This is hard and important, think for at least 1 minute or 2000 reasoning tokens" codex listened and produced MUCH better results and the total time to finish task 3x'ed but this is the trade off im willing to make if im getting better results
v

<img width="608" height="291" alt="Image" src="https://github.com/user-attachments/assets/acef1e9e-5da8-45a7-af57-89a3cf0d158d" />

I tried adding the following generic developer_instructions block to config.toml, and it appears to improve the early-termination / shallow-reasoning failure mode in my local runs. This intentionally avoids adding any problem-specific hints or answer keys. Could someone else cross-validate this on an independent setup, ideally with the same eval cases and model/settings, to check whether the improvement is reproducible rather than local-run noise?

developer_instructions = """
Do not answer quickly. Before the final answer, use sufficient slow internal reasoning and continue checking if the reasoning is only a familiar shortcut, a shallow pattern match, or a short rough calculation.
If the task involves reasoning, verification, debugging, evaluation, or diagnosis, continue until the conclusion is supported by concrete evidence rather than by a plausible first pass.

This is a generic anti-early-termination instruction. Do not add problem-specific facts, hidden assumptions, answer keys, or task-specific hints that were not provided by the user or visible evidence.
"""

Here is an actual deterministic script to run the test:

#!/usr/bin/env python3
"""Run and classify the five-shot local Codex reasoning-token eval."""

from __future__ import annotations

import json
import re
import shutil
import subprocess
import sys
from pathlib import Path
from typing import Any


RUNS = 5
MODEL_METADATA_FIELDS = (
    "context_window",
    "max_context_window",
    "comp_hash",
    "use_responses_lite",
)
SUSPICIOUS_REASONING_TOKENS = {516, 1034, 1552}
PROMPT = (
    "Do not use external tools. A black bag contains candies with counts: "
    "round apple 7, round peach 9, round watermelon 8; star apple 7, "
    "star peach 6, star watermelon 4. Shape is distinguishable by touch "
    "before drawing; flavor is not. What is the minimum number of candies "
    "to draw to guarantee having apple and peach candies of different shapes, "
    "i.e. round apple + star peach or round peach + star apple? Give reasoning "
    "and final number. The local project dir is irrelevant for this task, do "
    "not consult it."
)
COMMAND = [
    "codex",
    "exec",
    "--json",
    "--skip-git-repo-check",
    "--ephemeral",
    "-s",
    "read-only",
    "--disable",
    "memories",
    "-m",
    "gpt-5.5",
    "-c",
    "model_reasoning_effort=high",
    "-c",
    "personality=none",
]


class EvalError(RuntimeError):
    pass


def command_model() -> str | None:
    for index, value in enumerate(COMMAND):
        if value in {"-m", "--model"} and index + 1 < len(COMMAND):
            return COMMAND[index + 1]
        if value.startswith("--model="):
            return value.split("=", 1)[1]
    return None


def read_model_metadata(model: str | None) -> dict[str, object]:
    metadata: dict[str, object] = {
        "source": str(Path.home() / ".codex" / "models_cache.json"),
        "model": model,
    }
    if model is None:
        metadata["error"] = "no -m/--model found in COMMAND"
        return metadata

    cache_path = Path.home() / ".codex" / "models_cache.json"
    try:
        cache: dict[str, Any] = json.loads(cache_path.read_text(encoding="utf-8"))
    except FileNotFoundError:
        metadata["error"] = "models_cache.json not found"
        return metadata
    except json.JSONDecodeError as exc:
        metadata["error"] = f"models_cache.json is invalid JSON: {exc}"
        return metadata
    except OSError as exc:
        metadata["error"] = f"could not read models_cache.json: {exc}"
        return metadata

    metadata["cache_fetched_at"] = cache.get("fetched_at")
    metadata["cache_etag"] = cache.get("etag")
    metadata["cache_client_version"] = cache.get("client_version")
    models = cache.get("models")
    if not isinstance(models, list):
        metadata["error"] = "models_cache.json has no models list"
        return metadata

    for candidate in models:
        if isinstance(candidate, dict) and candidate.get("slug") == model:
            metadata.update(
                {
                    field: candidate.get(field)
                    for field in MODEL_METADATA_FIELDS
                }
            )
            return metadata

    metadata["error"] = f"model {model!r} not found in models_cache.json"
    return metadata


def parse_jsonl(path: Path) -> tuple[str, int, int | None]:
    final_message: str | None = None
    reasoning_tokens: int | None = None
    cached_input_tokens: int | None = None

    with path.open(encoding="utf-8") as stream:
        for line_number, line in enumerate(stream, start=1):
            if not line.strip():
                continue
            try:
                event = json.loads(line)
            except json.JSONDecodeError as exc:
                raise EvalError(f"{path}:{line_number}: invalid JSON: {exc}") from exc

            if event.get("type") == "item.completed":
                item = event.get("item", {})
                if item.get("type") == "agent_message" and isinstance(
                    item.get("text"), str
                ):
                    final_message = item["text"]

            if event.get("type") == "turn.completed":
                usage = event.get("usage", {})
                reasoning_value = usage.get("reasoning_output_tokens")
                if isinstance(reasoning_value, int) and not isinstance(
                    reasoning_value, bool
                ):
                    reasoning_tokens = reasoning_value
                cached_value = usage.get("cached_input_tokens")
                if isinstance(cached_value, int) and not isinstance(
                    cached_value, bool
                ):
                    cached_input_tokens = cached_value

    if final_message is None:
        raise EvalError(f"{path}: no completed agent_message found")
    if reasoning_tokens is None:
        raise EvalError(
            f"{path}: no turn.completed.usage.reasoning_output_tokens found"
        )
    return final_message, reasoning_tokens, cached_input_tokens


def extract_final_number(message: str) -> tuple[int | None, str]:
    # Prefer an explicitly boxed result, then an answer-labelled line. Fall back
    # to the last integer because the prompt explicitly asks for a final number.
    boxed = re.findall(r"\\boxed\s*\{\s*(\d+)\s*\}", message)
    if boxed:
        return int(boxed[-1]), "last boxed integer"

    labelled: list[str] = []
    for line in message.splitlines():
        if re.search(r"(?i)\b(final answer|answer|minimum(?: number)?)\b", line):
            labelled.extend(re.findall(r"(?<![\d,])\d+(?![\d,])", line))
    if labelled:
        return int(labelled[-1]), "last integer on an answer-labelled line"

    integers = re.findall(r"(?<![\d,])\d+(?![\d,])", message)
    if integers:
        return int(integers[-1]), "last integer in the final agent_message"
    return None, "no integer found"


def main() -> int:
    if len(sys.argv) > 2:
        print(f"Usage: {Path(sys.argv[0]).name} [RESULTS_DIR]", file=sys.stderr)
        return 2
    if shutil.which("codex") is None:
        print("error: codex is not on PATH", file=sys.stderr)
        return 1

    results_dir = (
        Path(sys.argv[1]).expanduser()
        if len(sys.argv) == 2
        else Path(__file__).resolve().with_name("codex-5shot-eval-results")
    )
    results_dir.mkdir(parents=True, exist_ok=True)

    model = command_model()
    model_metadata_before_runs = read_model_metadata(model)
    results: list[dict[str, object]] = []
    for run in range(1, RUNS + 1):
        jsonl_path = results_dir / f"run-{run}.jsonl"
        stderr_path = results_dir / f"run-{run}.stderr"
        print(f"[{run}/{RUNS}] running Codex...", flush=True)

        with jsonl_path.open("w", encoding="utf-8") as stdout, stderr_path.open(
            "w", encoding="utf-8"
        ) as stderr:
            completed = subprocess.run(
                COMMAND,
                input=PROMPT + "\n",
                text=True,
                stdout=stdout,
                stderr=stderr,
                check=False,
            )
        if completed.returncode != 0:
            raise EvalError(
                f"run {run}: codex exited {completed.returncode}; see {stderr_path}"
            )

        message, reasoning_tokens, cached_input_tokens = parse_jsonl(jsonl_path)
        model_metadata_after_run = read_model_metadata(model)
        final_number, extraction = extract_final_number(message)
        wrong_answer = final_number != 21
        suspicious_tokens = reasoning_tokens in SUSPICIOUS_REASONING_TOKENS
        affected = wrong_answer and suspicious_tokens
        result: dict[str, object] = {
            "run": run,
            "jsonl": str(jsonl_path),
            "final_agent_message": message,
            "cached_input_tokens": cached_input_tokens,
            "reasoning_output_tokens": reasoning_tokens,
            "model_metadata_after_run": model_metadata_after_run,
            "extracted_final_number": final_number,
            "extraction_method": extraction,
            "wrong_answer": wrong_answer,
            "suspicious_reasoning_tokens": suspicious_tokens,
            "affected": affected,
        }
        results.append(result)

        print(f"\n--- run {run} final agent_message ---")
        print(message)
        print(f"--- run {run} inspection ---")
        print(f"cached_input_tokens: {cached_input_tokens}")
        print(f"reasoning_output_tokens: {reasoning_tokens}")
        print(f"model metadata after run: {model_metadata_after_run}")
        print(f"extracted final number: {final_number!r} ({extraction})")
        print(f"wrong answer: {'yes' if wrong_answer else 'no'}")
        print(f"affected match: {'yes' if affected else 'no'}")

    affected_runs = [int(result["run"]) for result in results if result["affected"]]
    summary = {
        "command": COMMAND,
        "prompt": PROMPT,
        "model_metadata_fields": list(MODEL_METADATA_FIELDS),
        "model_metadata_before_runs": model_metadata_before_runs,
        "model_metadata_after_runs": read_model_metadata(model),
        "suspicious_reasoning_tokens": sorted(SUSPICIOUS_REASONING_TOKENS),
        "results": results,
        "verdict": "affected" if affected_runs else "fine",
        "affected_runs": affected_runs,
    }
    summary_path = results_dir / "summary.json"
    summary_path.write_text(json.dumps(summary, indent=2) + "\n", encoding="utf-8")

    print("\n=== VERDICT ===")
    if affected_runs:
        joined = ", ".join(map(str, affected_runs))
        print(f"You're affected (matching run(s): {joined}).")
    else:
        print("You're fine.")
    print(f"Full results: {results_dir}")
    print(f"Machine-readable summary: {summary_path}")
    return 0


if __name__ == "__main__":
    try:
        raise SystemExit(main())
    except EvalError as exc:
        print(f"error: {exc}", file=sys.stderr)
        raise SystemExit(1)

You run it on a PRO account it will reliably fail. You run it on a PLUS account it will reliably succeed.
I had codex analyze the reasons giving it access to the latest codex source code and analyze the results of the script above.
This was its conclusion, which is confirmed by above deterministic script:

Same Codex CLI version, same command, same gpt-5.5 slug. ChatGPT PRO receives comp_hash=2911, context_window=272000, use_responses_lite=false; ChatGPT PLUS receives comp_hash=3000, context_window=372000, use_responses_lite=true. PRO variant deterministically hits the 516 reasoning-token failure mode on this eval, while PLUS succeeds. Forcing Responses Lite on PRO is rejected with unsupported_value from remote throwing error This model is not supported when using X-OpenAI-Internal-Codex-Responses-Lite so it cannot be reliably excluded as reason.

Deterministically it is shown that:

• the LOWER tier PLUS gets a HIGHER CTX window
• a different comp hash is reported amongst the two tiers on the same models/reasoning/harness/personality
• the answers are consistently "bad" on the higher tier PRO, and consistently "good" on the lower tier PLUS

There is a workaround. The issue appears related to the Codex 5.5 system prompt, specifically the final ## Intermediary updates section.

In my tests, using the same prompt but removing that entire section stopped the recurring shallow/quantized reasoning behavior (reasoning_output_tokens=516) and produced the correct answer. Smaller edits inside that section were not reliable.

I built a small local Codex plugin as a practical guardrail for this issue:
https://github.com/DrSmoothl/codex-reasoning-sentinel
It watches Codex hook telemetry for suspicious low reasoning-token patterns like 516 / 1034 / 1552, asks for one extra verification pass via Stop hooks, and blocks risky write tools while a turn is still marked suspect. It is not a fix for the underlying model/runtime behavior, but it may help prevent silent bad outputs during complex refactors or high-stakes tasks.

There is a workaround. The issue appears related to the Codex 5.5 system prompt, specifically the final ## Intermediary updates section. In my tests, using the same prompt but removing that entire section stopped the recurring shallow/quantized reasoning behavior (reasoning_output_tokens=516) and produced the correct answer. Smaller edits inside that section were not reliable.

Removing that entire ## Intermediary updates section seems to have fixed it for me as well. Or at least reduced it to the point that 10 runs of the same candy puzzle prompt above didn't have the 516 token issue.

There is a workaround. The issue appears related to the Codex 5.5 system prompt, specifically the final ## Intermediary updates section. In my tests, using the same prompt but removing that entire section stopped the recurring shallow/quantized reasoning behavior (reasoning_output_tokens=516) and produced the correct answer. Smaller edits inside that section were not reliable.

same for me, OpenAI needs to step up and clarify what this was doing internallly

There is a workaround. The issue appears related to the Codex 5.5 system prompt, specifically the final ## Intermediary updates section. In my tests, using the same prompt but removing that entire section stopped the recurring shallow/quantized reasoning behavior (reasoning_output_tokens=516) and produced the correct answer. Smaller edits inside that section were not reliable.

instruction

get system prompt with

jq -r '.models[] | select(.slug=="gpt-5.5") | .base_instructions' ~/.codex/models_cache.json`

remove ## Intermediary updates block, save it to ~/.codex/model-instructions.md and update config.toml

model_instructions_file = "~/.codex/model-instructions.md"

Here's a script compatible with Python 3.10+ that quickly parses the session directories and shows some stats. For me, 516 is indeed the most common last-turn reasoning output token value after zero and six.

<details>
<summary><code>analyze_reasoning_tokens.py</code></summary>

from __future__ import annotations

from collections import Counter
from dataclasses import dataclass, field
import json
import math
import os
from pathlib import Path
from statistics import mean, median, pstdev
from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from collections.abc import Iterable, Iterator


CODEX_DIR = Path.home() / ".codex"
SOURCE_DIRS = (
    CODEX_DIR / "sessions",
    CODEX_DIR / "archived_sessions",
)
TOP_N = 25
TOKEN_MARKER = b'"reasoning_output_tokens"'
USAGE_FIELDS = (
    "input_tokens",
    "cached_input_tokens",
    "output_tokens",
    "reasoning_output_tokens",
    "total_tokens",
)


@dataclass(slots=True)
class ScanResult:
    last_values: list[int] = field(default_factory=list)
    total_values: list[int] = field(default_factory=list)
    files_scanned: int = 0
    files_with_usage: int = 0
    read_errors: int = 0
    parse_errors: int = 0
    duplicate_snapshots: int = 0


def nearest_rank(sorted_values: list[int], percentile: int) -> int:
    index = math.ceil(len(sorted_values) * percentile / 100) - 1
    index = min(max(index, 0), len(sorted_values) - 1)
    return sorted_values[index]


def format_number(value: float | int) -> str:
    if isinstance(value, float):
        return f"{value:,.2f}"
    return f"{value:,}"


def reasoning_tokens(usage: object) -> int | None:
    if not isinstance(usage, dict):
        return None

    value = usage.get("reasoning_output_tokens")
    if isinstance(value, int):
        return value
    return None


def usage_key(usage: object) -> tuple[object, ...] | None:
    if not isinstance(usage, dict):
        return None
    return tuple(usage.get(field) for field in USAGE_FIELDS)


def iter_candidate_files(roots: Iterable[Path]) -> Iterator[Path]:
    for root in roots:
        if not root.exists():
            continue

        for dirpath, _, filenames in os.walk(root):
            for filename in filenames:
                if filename.endswith(".jsonl"):
                    yield Path(dirpath) / filename


def collect_values(roots: Iterable[Path]) -> ScanResult:
    result = ScanResult()

    for path in iter_candidate_files(roots):
        result.files_scanned += 1
        file_has_usage = False
        seen_snapshots: set[
            tuple[tuple[object, ...] | None, tuple[object, ...] | None]
        ] = set()

        try:
            lines = path.open("rb")
        except OSError:
            result.read_errors += 1
            continue

        with lines:
            for line in lines:
                if TOKEN_MARKER not in line:
                    continue

                try:
                    event = json.loads(line)
                except json.JSONDecodeError:
                    result.parse_errors += 1
                    continue

                if not isinstance(event, dict):
                    continue

                payload = event.get("payload")
                if not isinstance(payload, dict):
                    continue

                info = payload.get("info")
                if not isinstance(info, dict):
                    continue

                last_usage = info.get("last_token_usage")
                total_usage = info.get("total_token_usage")
                last_value = reasoning_tokens(last_usage)
                total_value = reasoning_tokens(total_usage)
                if last_value is None and total_value is None:
                    continue

                snapshot = (usage_key(total_usage), usage_key(last_usage))
                if snapshot in seen_snapshots:
                    result.duplicate_snapshots += 1
                    continue

                seen_snapshots.add(snapshot)
                file_has_usage = True

                if last_value is not None:
                    result.last_values.append(last_value)
                if total_value is not None:
                    result.total_values.append(total_value)

        if file_has_usage:
            result.files_with_usage += 1

    return result


def print_distribution(title: str, values: list[int]) -> None:
    print(title)
    if not values:
        print("Samples: 0")
        return

    sorted_values = sorted(values)
    counts = Counter(values)
    total = len(values)
    p25 = nearest_rank(sorted_values, 25)
    p75 = nearest_rank(sorted_values, 75)

    print(f"Samples: {total:,}")
    print(f"Unique values: {len(counts):,}")
    print()

    print("Summary")
    print(f"Min: {format_number(sorted_values[0])}")
    print(f"P25: {format_number(p25)}")
    print(f"Median: {format_number(median(sorted_values))}")
    print(f"P75: {format_number(p75)}")
    print(f"P90: {format_number(nearest_rank(sorted_values, 90))}")
    print(f"P95: {format_number(nearest_rank(sorted_values, 95))}")
    print(f"P99: {format_number(nearest_rank(sorted_values, 99))}")
    print(f"Max: {format_number(sorted_values[-1])}")
    print(f"Mean: {format_number(mean(sorted_values))}")
    print(f"Std dev: {format_number(pstdev(sorted_values))}")
    print(f"IQR: {format_number(p75 - p25)}")
    print()

    print(f"Top {TOP_N} most frequent exact values")
    print(f"{'Rank':>4} {'Value':>10} {'Count':>10} {'Percent':>9}")
    top_values = sorted(counts.items(), key=lambda item: (-item[1], item[0]))[:TOP_N]
    for rank, (value, count) in enumerate(top_values, start=1):
        print(f"{rank:>4} {value:>10,} {count:>10,} {count / total * 100:>8.2f}%")


def print_report(result: ScanResult) -> None:
    print("reasoning_output_tokens distribution")
    print("Source directories:")
    for source_dir in SOURCE_DIRS:
        print(f"  {source_dir}")
    print(f"JSONL files scanned: {result.files_scanned:,}")
    print(f"Files with usage: {result.files_with_usage:,}")
    print(f"Duplicate snapshots skipped: {result.duplicate_snapshots:,}")
    print(f"Read errors skipped: {result.read_errors:,}")
    print(f"JSON parse errors skipped: {result.parse_errors:,}")
    print()

    print_distribution(
        "last_token_usage.reasoning_output_tokens",
        result.last_values,
    )
    print()
    print_distribution(
        "total_token_usage.reasoning_output_tokens",
        result.total_values,
    )


def main() -> None:
    if not CODEX_DIR.exists():
        raise SystemExit(f"Codex directory not found: {CODEX_DIR}")

    result = collect_values(SOURCE_DIRS)
    if not result.last_values and not result.total_values:
        raise SystemExit(f"No reasoning_output_tokens values found under {CODEX_DIR}")

    print_report(result)


if __name__ == "__main__":
    main()

</details>

reasoning_output_tokens distribution
Source directories:
  C:\Users\ofek\.codex\sessions
  C:\Users\ofek\.codex\archived_sessions
JSONL files scanned: 151
Files with usage: 131
Duplicate snapshots skipped: 453
Read errors skipped: 0
JSON parse errors skipped: 0

last_token_usage.reasoning_output_tokens
Samples: 6,888
Unique values: 1,055

Summary
Min: 0
P25: 0
Median: 38.00
P75: 280
P90: 639
P95: 1,182
P99: 2,837
Max: 11,817
Mean: 262.22
Std dev: 613.29
IQR: 280

Top 25 most frequent exact values
Rank      Value      Count   Percent
   1          0      1,813    26.32%
   2          6        374     5.43%
   3        516        370     5.37%
   4          7         92     1.34%
   5         10         79     1.15%
   6          9         77     1.12%
   7         12         71     1.03%
   8          8         62     0.90%
   9         14         60     0.87%
  10         11         56     0.81%
  11         15         54     0.78%
  12         13         52     0.75%
  13      1,034         49     0.71%
  14         16         46     0.67%
  15         17         39     0.57%
  16         22         39     0.57%
  17         21         37     0.54%
  18         18         33     0.48%
  19         19         33     0.48%
  20         24         32     0.46%
  21         29         31     0.45%
  22         38         31     0.45%
  23         20         29     0.42%
  24         28         29     0.42%
  25         37         28     0.41%

total_token_usage.reasoning_output_tokens
Samples: 6,888
Unique values: 4,515

Summary
Min: 0
P25: 4,098
Median: 16,850.00
P75: 51,028
P90: 97,638
P95: 102,955
P99: 155,068
Max: 178,774
Mean: 32,200.33
Std dev: 37,014.64
IQR: 46,930

Top 25 most frequent exact values
Rank      Value      Count   Percent
   1          0         45     0.65%
   2     19,371         30     0.44%
   3     25,813         23     0.33%
   4     44,238         17     0.25%
   5        516         16     0.23%
   6     21,341         14     0.20%
   7     13,978         13     0.19%
   8     23,139         13     0.19%
   9      1,856         12     0.17%
  10     14,185         12     0.17%
  11     56,059         11     0.16%
  12      7,521         10     0.15%
  13      8,384         10     0.15%
  14     14,586         10     0.15%
  15      7,372          9     0.13%
  16     10,960          9     0.13%
  17     26,868          9     0.13%
  18     28,383          9     0.13%
  19     63,090          9     0.13%
  20     12,961          8     0.12%
  21     18,490          8     0.12%
  22     19,455          8     0.12%
  23     19,939          8     0.12%
  24     20,925          8     0.12%
  25     21,517          8     0.12%

Prompt

Do not use external tools. A black bag contains candies with counts: round apple 7, round peach 9, round watermelon 8; star apple 7, star peach 6, star watermelon 4. Shape is distinguishable by touch before drawing; flavor is not. What is the minimum number of candies to draw to guarantee having apple and peach candies of different shapes, i.e. round apple + star peach or round peach + star apple? Give reasoning and final number. The local project dir is irrelevant for this task, do not consult it.

PRO plan with ## Intermediary updates

| Model / effort | Run | Status | Duration | reasoning_output_tokens | 518*n-2 | Final answer | Correct? |
|---|---:|---:|---:|---:|---:|---:|---|
| gpt-5.5 xhigh | 1 | 0 | 38s | 516 | n=1 | 26 | No |
| gpt-5.5 xhigh | 2 | 0 | 21s | 516 | n=1 | 22 | No |
| gpt-5.5 xhigh | 3 | 0 | 188s | 9322 | n=18 | 21 | Yes |
| gpt-5.5 xhigh | 4 | 0 | 31s | 516 | n=1 | 27 | No |
| gpt-5.5 xhigh | 5 | 0 | 19s | 516 | n=1 | 27 | No |

PRO plan without ## Intermediary updates

| Model / effort | Run | Status | Duration | reasoning_output_tokens | 518*n-2 | Final answer | Correct? |
|---|---:|---:|---:|---:|---:|---:|---|
| gpt-5.5 xhigh | 1 | 0 | 110s | 0 | No | 21 | Yes |
| gpt-5.5 xhigh | 2 | 0 | 132s | 6732 | n=13 | 21 | Yes |
| gpt-5.5 xhigh | 3 | 0 | 117s | 6091 | No | 21 | Yes |
| gpt-5.5 xhigh | 4 | 0 | 90s | 4650 | No | 21 | Yes |
| gpt-5.5 xhigh | 5 | 0 | 148s | 7768 | n=15 | 21 | Yes |

PRO plan with codexcomp proxy

| Model / effort | Run | Status | Duration | reasoning_output_tokens | 518*n-2 | Final answer | Correct? |
|---|---:|---:|---:|---:|---:|---:|---|
| gpt-5.5 xhigh | 1 | 0 | 239s | 8800 | No | 21 | Yes |
| gpt-5.5 xhigh | 2 | 0 | 215s | 8716 | No | 21 | Yes |
| gpt-5.5 xhigh | 3 | 0 | 239s | 9316 | No | 21 | Yes |
| gpt-5.5 xhigh | 4 | 0 | 238s | 9145 | No | 21 | Yes |
| gpt-5.5 xhigh | 5 | 0 | 214s | 9316 | No | 21 | Yes |

PRO plan with codexcomp --max-n 1 proxy

| Model / effort | Run | Status | Duration | reasoning_output_tokens | 518*n-2 | Final answer | Correct? |
|---|---:|---:|---:|---:|---:|---:|---|
| gpt-5.5 xhigh | 1 | 0 | 175s | 6730 | No | 21 | Yes |
| gpt-5.5 xhigh | 2 | 0 | 159s | 6212 | No | 21 | Yes |
| gpt-5.5 xhigh | 3 | 0 | 160s | 6212 | No | 21 | Yes |
| gpt-5.5 xhigh | 4 | 0 | 143s | 6722 | No | 21 | Yes |
| gpt-5.5 xhigh | 5 | 0 | 167s | 7250 | n=14 | 21 | Yes |

---

PLUS plan with ## Intermediary updates

| Model / effort | Run | Status | Duration | reasoning_output_tokens | 518*n-2 | Final answer | Correct? |
|---|---:|---:|---:|---:|---:|---:|---|
| gpt-5.5 xhigh | 1 | 0 | 17s | 516 | n=1 | 15 | No |
| gpt-5.5 xhigh | 2 | 0 | 22s | 516 | n=1 | 23 | No |
| gpt-5.5 xhigh | 3 | 0 | 20s | 516 | n=1 | 27 | No |
| gpt-5.5 xhigh | 4 | 0 | 19s | 516 | n=1 | 29 | No |
| gpt-5.5 xhigh | 5 | 0 | 186s | 9840 | n=19 | 21 | Yes |

PLUS plan without ## Intermediary updates 1

| Model / effort | Run | Status | Duration | reasoning_output_tokens | 518*n-2 | Final answer | Correct? |
|---|---:|---:|---:|---:|---:|---:|---|
| gpt-5.5 xhigh | 1 | 0 | 111s | 5696 | n=11 | 21 | Yes |
| gpt-5.5 xhigh | 2 | 0 | 113s | 5696 | n=11 | 21 | Yes |
| gpt-5.5 xhigh | 3 | 0 | 150s | 7768 | n=15 | 21 | Yes |
| gpt-5.5 xhigh | 4 | 0 | 225s | 11912 | n=23 | 21 | Yes |
| gpt-5.5 xhigh | 5 | 0 | 112s | 5696 | n=11 | 21 | Yes |

PLUS plan without ## Intermediary updates 2

| Model / effort | Run | Status | Duration | reasoning_output_tokens | 518*n-2 | Final answer | Correct? |
|---|---:|---:|---:|---:|---:|---:|---|
| gpt-5.5 xhigh | 1 | 0 | 209s | 10876 | n=21 | 21 | Yes |
| gpt-5.5 xhigh | 2 | 0 | 183s | 8804 | n=17 | 21 | Yes |
| gpt-5.5 xhigh | 3 | 0 | 121s | 6214 | n=12 | 21 | Yes |
| gpt-5.5 xhigh | 4 | 0 | 248s | 12948 | n=25 | 21 | Yes |
| gpt-5.5 xhigh | 5 | 0 | 152s | 7768 | n=15 | 21 | Yes |

PLUS plan with codexcomp proxy

| Model / effort | Run | Status | Duration | reasoning_output_tokens | 518*n-2 | Final answer | Correct? |
|---|---:|---:|---:|---:|---:|---:|---|
| gpt-5.5 xhigh | 1 | 0 | 247s | 6498 | No | 21 | Yes |
| gpt-5.5 xhigh | 2 | 0 | 247s | 9320 | No | 21 | Yes |
| gpt-5.5 xhigh | 3 | 0 | 270s | 10354 | No | 21 | Yes |
| gpt-5.5 xhigh | 4 | 0 | 136s | 5390 | No | 21 | Yes |
| gpt-5.5 xhigh | 5 | 0 | 288s | 10872 | No | 21 | Yes |

PLUS plan with codexcomp --max-n 1 proxy

| Model / effort | Run | Status | Duration | reasoning_output_tokens | 518*n-2 | Final answer | Correct? |
|---|---:|---:|---:|---:|---:|---:|---|
| gpt-5.5 xhigh | 1 | 0 | 138s | 5696 | n=11 | 21 | Yes |
| gpt-5.5 xhigh | 2 | 0 | 309s | 12946 | No | 21 | Yes |
| gpt-5.5 xhigh | 3 | 0 | 264s | 11392 | No | 21 | Yes |
| gpt-5.5 xhigh | 4 | 0 | 253s | 9320 | No | 21 | Yes |
| gpt-5.5 xhigh | 5 | 0 | 212s | 8181 | No | 21 | Yes |

---

PRO plan: normal operation without ## Intermediary updates

| Records | >=516 | exact 516 | exact 1034 | exact 1552 | 516 / >=516 | 518n-2 / >=516 |
|---:|---:|---:|---:|---:|---:|---:|
| 448 | 35 | 11 | 3 | 2 | 31.43% | 16 / 35 (45.71%) |

PLUS plan: normal operation without ## Intermediary updates

| Records | >=516 | exact 516 | exact 1034 | exact 1552 | 516 / >=516 | 518n-2 / >=516 |
|---:|---:|---:|---:|---:|---:|---:|
| 142 | 20 | 9 | 0 | 0 | 45.00% | 9 / 20 (45.00%) |

PRO plan: normal operation with codexcomp --max-n 1 proxy

| Records | min | mean | max | >=516 | exact 516 | exact 1034 | exact 1552 | 516 / >=516 | 518n-2 / >=516 |
|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| 265 | 0 | 93.92 | 1367 | 17 | 0 | 0 | 0 | 0.00% | 0 / 17 (0.00%) |

---

The failure does not appear to be caused by the 518*n-2 pattern itself, but by the model collapsing into the minimum reasoning block: 516 tokens.

Both PRO and PLUS show the same failure mode with ## Intermediary updates. In these targeted tests, PLUS appears to quantize more consistently than PRO: without ## Intermediary updates, PLUS falls into a quantized 518*n-2 block in 100% of runs, while still producing correct answers.

This suggests that quantization may be normal, or at least harmless at higher blocks. The damaging case appears to be specifically the first block, 516, where reasoning seems to end too early.

All targeted tests run through codexcomp completed correctly without modifying model_instructions_file. When limiting the proxy to --max-n 1, all targeted runs also completed correctly. I could not validate whether the --max-n 1 proxy runs with the PLUS plan still preserved the 518*n-2 quantization pattern internally.

Adding another independent datapoint from my local Codex telemetry. I’m only sharing aggregate token metadata here, not raw session logs, prompts, responses, local paths, repo names, session IDs, account IDs, or any screenshots.

Environment:

• Product: Codex local session telemetry
• Platform: macOS
• Codex CLI version observed locally: 0.142.5
• Data source: local Codex session JSONL token_count events
• Field analyzed: payload.info.last_token_usage.reasoning_output_tokens
• Local file window: 2026-02-02 through 2026-07-05 UTC

I parsed local token_count events, attributed each record to the active turn_context.model / effort, and de-duplicated repeated records by session, timestamp, model, effort, and token-usage tuple.

Results after de-duplication:

| Scope | Records | >=516 records | exact 516 | exact 1034 | exact 1552 | exact 516 / >=516 | 518n-2 / >=516 |
|---|---:|---:|---:|---:|---:|---:|---:|
| All models | 63,176 | 6,492 | 1,404 | 155 | 52 | 21.63% | 25.68% |
| gpt-5.5 | 19,753 | 2,281 | 1,121 | 120 | 44 | 49.15% | 58.44% |
| gpt-5.5 / xhigh | 13,035 | 1,896 | 905 | not separately counted here | not separately counted here | 47.73% | 58.02% |

For gpt-5.5, exact 516 was the second most common reasoning-token value after 0.

This also appears recent in my logs. On 2026-07-04 UTC, I still had multiple gpt-5.5 events landing exactly on 516, plus one exact 1552 event.

I’m not claiming this proves hidden reasoning truncation or output-quality degradation by itself, but it does independently reproduce the fixed-value clustering pattern reported here. My local gpt-5.5 exact-516 rate among >=516 records is 49.15%, which is close to the original report’s 44.0%.

@etraut-openai can someone take a look here

Independent local reproduction, aggregate telemetry only. No prompts, responses, paths, session IDs, or screenshots included.

Environment:

• macOS
• codex-cli 0.142.5
• ~/.codex/models_cache.json has one gpt-5.5 entry and its base_instructions contains ## Intermediary updates
• no model_instructions_file override was present before testing

Parsed local token_count events from ~/.codex/sessions and ~/.codex/archived_sessions, using payload.info.last_token_usage.reasoning_output_tokens, attributed by active turn_context.model / effort, with duplicate token snapshots skipped.

Results after de-dupe:

| Scope | Records | >=516 | exact 516 | exact 1034 | exact 1552 | 516 / >=516 | 518n-2 / >=516 |
|---|---:|---:|---:|---:|---:|---:|---:|
| all models | 489,791 | 42,371 | 10,784 | 1,454 | 481 | 25.45% | 31.63% |
| gpt-5.5, all efforts | 248,315 | 14,680 | 5,602 | 757 | 268 | 38.16% | 47.06% |
| gpt-5.5 / xhigh | 72,014 | 7,484 | 2,772 | 352 | 129 | 37.04% | 46.29% |

Recent local data still shows the pattern:

• 2026-07 gpt-5.5 / xhigh: exact 516 / >=516 = 43 / 103 = 41.75%; 518n-2 / >=516 = 59.22%.
• 2026-07-05 UTC small sample: exact 516 / >=516 = 2 / 6; 518n-2 / >=516 = 4 / 6.

I also tested the proposed local mitigation by generating a project-local model_instructions_file from the cached gpt-5.5 base_instructions with only the final ## Intermediary updates block removed. codex debug prompt-input confirms the effective prompt no longer contains ## Intermediary updates in that project.

This independently reproduces the fixed-token clustering. It does not by itself prove hidden truncation or quality causality; I have not yet rerun a fresh quality eval after the mitigation.

Maybe the model is getting confused about what's an intermediate result and what's the final result? There is an instruction under the ## Intermediary updates section:

- User updates are short updates while you are working, they are NOT final answers.

Clearly just telling it that it's not the final answer does not work.

Even after removing the ## Intermediary updates section, I'm still seeing the 516 thinking tokens during normal use. Though it fixes the puzzle benchmark results.

This could also potentially be due to the following lines:

1. You provide user updates frequently, every 30s.
2. As you are thinking, you very frequently provide updates even if not taking any actions, informing the user of your progress. You interrupt your thinking and send multiple updates in a row if thinking for more than 100 words.

If I had to guess these two instructions are essentially stopping long chain thinking from happening. Potentially, every time any deep reasoning efforts happen, the thinking stops to update the user, then restarts.

This is extremely concerning if it was intentional. If users were consistently routed into degraded reasoning behavior while paying for frontier-model access, that could raise serious consumer-protection and class action questions.

Looking at my own telemetry, this does not appear to be a recent regression after a period of normal service. The same fixed token degradation pattern seems to have been present from the beginning of my subscription. That makes it hard to accept this as a short-lived unnoticed bug. If the behavior persisted for months at scale, OpenAI should explain how it went undetected, what monitoring existed and whether any cost-saving or routing mechanism contributed to it.

I applied the temporary workaround discussed in the issue: extracting the gpt-5.5 base instructions, removing the ## Intermediary updates section, pointing Codex at that modified instructions file with model_instructions_file, and setting personality = "none" to avoid the personality/model-messages fallback path. This improved the dedicated candy eval runs, but it did not eliminate the pattern in normal use: another test thread still produced at least one 516reasoning-token event afterward. So the workaround appears partial at best, not a complete fix.

This issue cannot be fixed with prompt engineering, it is a backend issue tied to a _specific payment tier_, not a different _model or harness_, as my comment above proves and anyone can replicate without issue as long they have access to a PRO and PLUS account.

And that is what makes this rather infuriating, too. It may or may not be intentional, caused or not by the base prompt differences of PLUS and PRO or the feature flags differences, that is another topic - the point is that paying 200 bucks a month and then get worse performance than an account on a tenth of that amount is just absurd, and simply cannot be ignored nor should even ever happen to start with. Bugs do happen, yet you aren't selling a premium car that is outperformed by a cheap version thereof for very long, so the premium really should be receiving some more thorough testing, which, provided telemetry we have no access to and model stats we have no access to, but OAI has, should really not be be over-asking.

@smileBeda Okay so removing the intermediary updates will not improve the performace ? I am not aware of how these things work, but can you just explain me in a few words. What I have understood from this is that there is a cap of 516 reasoning tokens, is that correct ?

This issue cannot be fixed with prompt engineering, it is a backend issue tied to a _specific payment tier_, not a different _model or harness_, as my comment above proves and anyone can replicate without issue as long they have access to a PRO and PLUS account. And that is what makes this rather infuriating, too. It may or may not be intentional, caused or not by the base prompt differences of PLUS and PRO or the feature flags differences, that is another topic - the point is that paying 200 bucks a month and then get worse performance than an account on a tenth of that amount is just absurd, and simply cannot be ignored nor should even ever happen to start with. Bugs do happen, yet you aren't selling a premium car that is outperformed by a cheap version thereof for very long, so the premium really should be receiving some more thorough testing, which, provided telemetry we have no access to and model stats we have no access to, but OAI has, should really not be be over-asking.

youre totally right, its unacceptable

@smileBeda Okay so removing the intermediary updates will not improve the performace ? I am not aware of how these things work, but can you just explain me in a few words. What I have understood from this is that there is a cap of 516 reasoning tokens, is that correct ?

Removing that block does improve performance on tasks that require long thinking. It seems to make the model less likely to get interrupted in the middle of thinking and confidently return the wrong result.

There might be another issue going on independent of this.

@saketlovescoding it improves it against that canary test prompt. Obviously, since the very "fix" already recognized that each line of the prompt effects performance depending on other present or not lines, imagine what happens once actual real user prompts are used?

There is unfortunately no way to tell if it also fixes real usage since you'd have to AB test real work tasks. What is provable is that on PRO, with or without fixed prompts you still get the same signatures in the logs (that is, the reasoning token lengths). On PLUS, you reliably do NOT get that signature no matter what, not even during real work tasks. There will be ZERO occurrences of the signatures, while on PRO, no matter what, you will unavoidably find some.

Example during real work time window of PRO with "fixed" base prompt I had for 516, 1034, 1552 reasoning tokens 7 occurrences total.
During a similarly long work window and same project/agents file/similar prompts PLUS without fixed prompt, I had for 516, 1034, 1552 reasoning tokens 0 occurrences.

This proves nothing reliably, at all. But, it indicates that there are still issues on PRO despite the prompt "fix", just rarer than before (where the 516, 1034, 1552: was in the thousands), while on PLUS it is _zero_
And, from a _felt_ performance, that shows too, since I am _constantly_ battling PRO the past few days, yet on PLUS it is flying along very satisfyingly.

@NarinderS no it does not provably do that. It does provably do that together with the very specific and narrow canary test prompt used in this entire gh issue, not across real world tasks.

It only passes that specific test. In normal operation, the issue persists without ## Intermediary updates.

| Records | >=516 | exact 516 | exact 1034 | exact 1552 | 516 / >=516 | 518n-2 / >=516 |
|---:|---:|---:|---:|---:|---:|---:|
| 448 | 35 | 11 | 3 | 2 | 31.43% | 16 / 35 (45.71%) |

https://github.com/openai/codex/issues/30364#issuecomment-4886606283

@josantonius what can we do then, is there any work around before open ai fixes this issue ?

@josantonius what can we do then, is there any work around before open ai fixes this issue ?

You should read the comments we post instead of re-asking what already was answered.

1. You can use a PLUS account. It won't happen there
2. You can keep using PRO, it does not just reliably nerf the thing 100% of times
3. You could contribute, by making more tests, like a totally rewritten prompt, for example. It is expensive to run those tests me and others did above.
4. Finally, no, there is likely not much you can _reliably_ do on the PRO model other than cancel it and buy 10 PLUS model subs instead.

With the change the tests are passing and I do not see the capped token count numbers. This seems like improvement.

Edit: post prompt update:

• Ran 20 more samples with the current “removed ## Intermediary updates only” prompt.

  Raw path: /tmp/codex-20shot-eval-removed-block-1783284840

  All 20 answered 21. No wrong answers. No 516, 1034, or 1552 token counts in this batch.

   Shot    Answer    Reasoning tokens
  ━━━━━━  ━━━━━━━━  ━━━━━━━━━━━━━━━━━━
   1           21                4660
  ──────  ────────  ──────────────────
   2           21                5163
  ──────  ────────  ──────────────────
   3           21                5876
  ──────  ────────  ──────────────────
   4           21                6732
  ──────  ────────  ──────────────────
   5           21                6214
  ──────  ────────  ──────────────────
   6           21                4981
  ──────  ────────  ──────────────────
   7           21                6085
  ──────  ────────  ──────────────────
   8           21                8766
  ──────  ────────  ──────────────────
   9           21                3873
  ──────  ────────  ──────────────────
   10          21                7250
  ──────  ────────  ──────────────────
   11          21                4142
  ──────  ────────  ──────────────────
   12          21                4815
  ──────  ────────  ──────────────────
   13          21               10959
  ──────  ────────  ──────────────────
   14          21                5444
  ──────  ────────  ──────────────────
   15          21                4142
  ──────  ────────  ──────────────────
   16          21                6732
  ──────  ────────  ──────────────────
   17          21                4660
  ──────  ────────  ──────────────────
   18          21                5178
  ──────  ────────  ──────────────────
   19          21                4924
  ──────  ────────  ──────────────────
   20          21                6732

  One parser row initially said UNKNOWN for shot 7 because the final phrasing was Minimum: **21
  candies**; I inspected it manually and it is correct.

FWIW heres my week by week stats of these reasoning token counts increasing steadily.

<img width="1920" height="1440" alt="Image" src="https://github.com/user-attachments/assets/24f2bcee-ad85-43d2-b865-dbaf1c9b6d17" />

You can actually fix this with prompt engineering, but you need to do it exactly correct and do some back and forth with the model to extract enough of the developer instructions to properly word your 'fix'; e.g. the model is instructed that the [commentary] channel output should only be used "when justified".

You therefore need to add the qualifier "always justified" to your instruction otherwise the model itself will short circuit your fix.

I'm hesitant to describe the fix I am using exactly, because I am reasonably sure openai is doing this intentionally (it is clear and obvious that not every request needs xhigh reasoning to succeed, but judging that cutoff correctly at scale is unreliable).

In short, my findings are that if [commentary] channel is triggered on the turn as the immediate first model action you do not get short circuited reasoning on that turn, so if you put your engineered request in the right place (not in your prompt and not any system prompt editing) with the correct trigger words added to sidestep (not jailbreak) the developer instructions with the models own logic, then you get reliable full reasoning every turn.

I note that this will use your codex quota perceivably faster. Prior to implementing my fix I was unable to exhaust 5 hour pro limits with goal enabled and sub agent use. I am now exhausting my 5 hour limit like the before times.

There is a workaround. The issue appears related to the Codex 5.5 system prompt, specifically the final ## Intermediary updates section. In my tests, using the same prompt but removing that entire section stopped the recurring shallow/quantized reasoning behavior (reasoning_output_tokens=516) and produced the correct answer. Smaller edits inside that section were not reliable.

Massive Respect for this.. I can finally use gpt-5.5 again

> There is a workaround. The issue appears related to the Codex 5.5 system prompt, specifically the final ## Intermediary updates section. > In my tests, using the same prompt but removing that entire section stopped the recurring shallow/quantized reasoning behavior (reasoning_output_tokens=516) and produced the correct answer. Smaller edits inside that section were not reliable. Massive Respect for this.. I can finally use gpt-5.5 again

If you scroll up you will note that the system prompt edit doesn't fix the issue in real world usage according to another post, and only helped the synthetic riddle

> > There is a workaround. The issue appears related to the Codex 5.5 system prompt, specifically the final ## Intermediary updates section. > > In my tests, using the same prompt but removing that entire section stopped the recurring shallow/quantized reasoning behavior (reasoning_output_tokens=516) and produced the correct answer. Smaller edits inside that section were not reliable. > > > Massive Respect for this.. I can finally use gpt-5.5 again If you scroll up you will note that the system prompt edit doesn't fix the issue in real world usage according to another post, and only helped the synthetic riddle

there is no difference. Also seeing 516 doesn't imply failure, it can also mean the model didnt require reasoning past that quantized level.

There are too many assumptions being made on how it's "supposed" to work. only OAI can do a true fix

there is no difference. Also seeing 516 doesn't imply failure, it can also mean the model didnt require reasoning past that quantized level. There are too many assumptions being made on how it's "supposed" to work. only OAI can do a true fix

Seconded, the only thing I'm fairly confident about is that sometimes long chains of thought get short-circuited to 516 tokens. But not all 516 thinking tokens mean automatic failure. And the solution to the former issue seems to be removing the ## Intermediary updates block.

> there is no difference. Also seeing 516 doesn't imply failure, it can also mean the model didnt require reasoning past that quantized level. > There are too many assumptions being made on how it's "supposed" to work. only OAI can do a true fix Seconded, the only thing I'm fairly confident about is that sometimes long chains of thought get short-circuited to 516 tokens. But not all 516 thinking tokens mean automatic failure. And the solution to the former issue seems to be removing the ## Intermediary updates block.

> > > There is a workaround. The issue appears related to the Codex 5.5 system prompt, specifically the final ## Intermediary updates section. > > > In my tests, using the same prompt but removing that entire section stopped the recurring shallow/quantized reasoning behavior (reasoning_output_tokens=516) and produced the correct answer. Smaller edits inside that section were not reliable. > > > > > > Massive Respect for this.. I can finally use gpt-5.5 again > > > If you scroll up you will note that the system prompt edit doesn't fix the issue in real world usage according to another post, and only helped the synthetic riddle there is no difference. Also seeing 516 doesn't imply failure, it can also mean the model didnt require reasoning past that quantized level. There are too many assumptions being made on how it's "supposed" to work. only OAI can do a true fix

Also, you're both full of shit; chatgpt web suffers from the same failure if reasoning short circuits and no initial commentary channel is output nor thinking summaries produced. Forcing the commentary as the first action resolved this on chatgpt 5.5 web exactly the same as it did for codex.

Also, you're both full of shit; chatgpt web suffers from the same failure if reasoning short circuits and no initial commentary channel is output nor thinking summaries produced. Forcing the commentary as the first action resolved this on chatgpt 5.5 web exactly the same as it did for codex.

I'm just gonna ignore that first part, and address the more productive thing.

It look like adding Always start with a message in the commentary channel before starting work seems to work just as well as removing the commentary entirely on the synthetic candy test.

So either start with a message in the commentary channel, or disabling it entirely both help? Seems like some strange bug with the commentary system.

Interesting it happens with ChatGPT web too (make sure you tell it to not use any tools). The web version fails the candy test a lot, unless you add Always start with commentary.

My personal solution:
Implement CodexCont style continued reasoning inside Codex core, but keep a single JSONL ledger.

Goal:
When a response appears truncated by the reasoning-token fingerprint, transparently continue reasoning while preserving every factual item in rollout JSONL. Modified Codex should hide/fold continuation artifacts in TUI. Unmodified Codex may ignore new fields and render those artifacts visibly.

Detection:

• A reasoning truncation fingerprint is: reasoning_output_tokens == 518 * n - 2.
• Continue only when n < 5.
• If n >= 5, let the response finish normally.

Continuation request:

• Do not rely on previous_response_id for continuation rounds.
• Drop previous_response_id and send explicit input.
• Build input as:

original stable input + replayed encrypted reasoning items + a short commentary marker.

• Marker text should be exactly:

“Continue thinking...”

• Do not use “Do not repeat visible output”.
• Do not include hidden-round ordinary assistant messages in the continuation input.
• Do not include hidden-round tool-call proposals in the continuation input.
• Do not execute hidden-round tool-call proposals.

Ledger:

• Do not drop hidden-round output from JSONL.
• Record all actual response items and usage:

reasoning items, markers, tentative assistant messages, tentative tool-call proposals, and token usage.

• Add backward-compatible metadata fields that unmodified Codex can ignore.
• Suggested metadata semantics:

continuation_round: integer
continuation_hidden: boolean
continuation_tentative: boolean
continuation_not_replayed: boolean
continuation_not_executed: boolean
continuation_marker: boolean
continuation_accepted_final: boolean

• Treat JSONL as the only ledger. Rendering and model-input projection are derived from it.

TUI/replay:

• Hide or fold continuation markers.
• Hide or fold hidden-round tentative assistant messages.
• Hide or fold hidden-round tentative tool-call proposals.
• Render continued reasoning specially only when there is visible summary text; encrypted reasoning may have summary: [].
• Final accepted answer renders normally.
• Avoid double-rendering the same reasoning when both legacy event_msg agent_reasoning and response_item reasoning are present.

Ownership:

• Put the repair in the continued reasoning replay patch group, especially around 2142.
• Do not move it into the old 2001 raw response item suppression patch.
• 2001 fixed app-server raw response item egress; this is a later core/TUI continuation replay issue.

Tests:

• n < 5 opens continuation.
• n >= 5 does not.
• continuation drops previous_response_id.
• continuation input contains original input + encrypted reasoning + “Continue thinking...”.
• continuation input excludes hidden tentative assistant/tool outputs.
• hidden tentative tool calls are not executed.
• all hidden-round items and usage remain in JSONL.
• modified TUI hides/folds continuation artifacts.
• unmodified Codex can ignore new fields.

<img width="730" height="540" alt="Image" src="https://github.com/user-attachments/assets/4e0ded3c-2b28-4c2c-8691-60e1c0a3c8f1" />

@92645417d9e5c763259dbebc306e3e why not rely on previous_response_id? Is that mainly because it fails on POST/SSE?

I found it works on the Codex Responses WebSocket path when session continuity is preserved. I’ve solved this on WS and will publish an OSS version with Codex (proxy) + Pi support.

Same Issue here. Fails.

You’re affected.

All five local codex exec runs exited 0. The parsed final results were:

Run reasoning_output_tokens Final answer
━━━━━ ━━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━
1 516 21
───── ───────────────────────── ──────────────
2 516 29
───── ───────────────────────── ──────────────
3 516 29
───── ───────────────────────── ──────────────
4 516 14
───── ───────────────────────── ──────────────
5 516 18

Your trigger condition is met: run 2 returned 516 reasoning tokens and gave the wrong answer 29. Runs 3-5 also hit 516 with wrong answers.

Artifacts are in I:\Git\VMBot\codex_eval_20260705_gpt55\run_*.jsonl.

─ Worked for 19m 09s ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

› what does that mean?

• It means your local codex exec path with gpt-5.5 is reproducing the failure pattern that prompt was testing for.

The signal is not just “it got a math problem wrong.” The suspicious part is:

• Every run stopped at exactly 516 reasoning tokens.
• 4 of 5 final answers were wrong.
• The test prompt specifically said wrong answers with 516, 1034, or 1552 reasoning tokens indicate the affected failure mode.

So “affected” means this local Codex/model/config combination is likely hitting a known bad reasoning behavior: it appears to terminate or collapse its
reasoning at a characteristic internal-token count and then gives confident but invalid answers.

It does not mean:

• your VMBot repo is broken
• your OpenAI billing/quota issue caused this
• all Codex answers are bad
• your machine is broken

It means: don’t trust this exact codex exec ... -m gpt-5.5 -c model_reasoning_effort=high path for exact combinatorics/math-style reasoning right now
without verification. The useful artifact is the folder:

Independent corroboration from a wire-level vantage point, plus a continuation experiment

I run a small local proxy (codexcomp, MIT) between Codex CLI and the ChatGPT backend that inspects reasoning_tokens of every model round in real time — so this is live wire-level observation, not post-hoc telemetry. Sharing 2 days of data as additional evidence, plus a continuation experiment that telemetry alone can't show.

Environment: Codex CLI 0.142.5 · gpt-5.5 · model_reasoning_effort = "xhigh" · ChatGPT backend (chatgpt.com/backend-api/codex/responses) · single user, real coding workload · 2026-07-04 → 2026-07-06 (UTC+8).

1. Boundary clustering reproduces — and extends beyond 1552

| Metric | Value |
| --- | ---: |
| Model rounds observed | 275 |
| Completed requests | 223 |
| Rounds landing exactly on 518·n − 2 | 47 (17.1% of all rounds) |
| Rounds with reasoning_tokens ≥ 516 | 73 |
| Exact-boundary / ≥516 ratio | 64.4% |

Exact hits by boundary: 516 ×41, 1034 ×4, 2070 (= 518·4−2) ×1, 2588 (= 518·5−2) ×1. The family extends at least to n=5, beyond the 516/1034/1552 trio in the OP.

The OP's aggregate found 44.0% of gpt-5.5 responses with ≥516 reasoning landing exactly on 516 (vs 1.3% for other models). My xhigh-only sample is even more skewed: 64% of such rounds land exactly on a 518n−2 value. Natural stopping should not produce fixed-point clustering like this.

2. Continuation experiment: the model resumes when handed its own reasoning back

The proxy doesn't just observe. When a round ends exactly on the fingerprint, it discards the tentative output, replays that round's reasoning items (including encrypted reasoning) with a continue nudge, and folds the rounds into a single downstream response. Over the same window:

• 36 of 223 completed requests (16.1%) triggered at least one continuation.
• In 21/36, the model kept reasoning after replay — median +188 reasoning tokens, max +1,999 (a request truncated at 516 went on to reason ~4.9× its truncated total before answering).
• 5 requests were re-truncated exactly on a boundary again on the continuation round (e.g. 516 → 516 → clean); one request needed 3 continuations.
• In 15/36 the continuation produced no further reasoning (the model went straight to output) — plausibly benign near-boundary stops.
• All 36 folded requests then finished with status=completed, stop=natural.

The repeated re-truncation at exactly the same fixed boundary after an explicit continue is the part I find hardest to square with anything other than a server-side thresholded reasoning budget.

Caveats: single user, one machine, 2-day window, xhigh effort only, small n. Log format and methodology are in the repo README; happy to share sanitized raw fold logs if useful.

I’m seeing similar behavior. In my local evals, gpt-5.5 with model_reasoning_effort=high repeatedly landed at exactly 516 reasoning tokens and often gave the wrong answer. Adding an explicit developer instruction changed the behavior for me, high-effort runs started using thousands of reasoning tokens and got the eval right.

My current workaround is at the proxy layer, preserve the client-requested reasoning effort, but inject a developer message only when the request asks for high / xhigh.

For example:

high/xhigh: “This task may require extended reasoning. Do not curtail reasoning early; continue until the reasoning is complete.”

I may also try a lighter version for medium, but I’m avoiding applying it globally because it clearly increases latency/cost. Not claiming this proves the root cause, but it does look like prompt-shaping can reduce the premature-curtailment behavior.

Follow-up: 80-run controlled A/B — gpt-5.4 vs gpt-5.5 × 4 efforts × mitigation on/off

Following up on my earlier comment with a controlled experiment instead of passive traffic: 2 models (gpt-5.4, gpt-5.5) × 4 reasoning efforts (low/medium/high/xhigh) × continuation-mitigation on/off × 5 runs = 80 runs, all on the same fixed prompt.

Method

• Fixed task: a 4-house zebra-style logic puzzle (13 clues, brute-force-verified unique solution), "reply with only the name" → machine-gradable correctness. Pure reasoning, no tools, single model round per run.
• codex exec --json --ephemeral (CLI 0.142.5, read-only sandbox), ChatGPT backend, one account, one machine, all 80 runs inside one ~26-minute window (2026-07-06), conditions interleaved rep-by-rep to avoid time-of-day bias. All 80 runs exited cleanly.
• off = straight to the backend (openai_base_url override); reasoning tokens read from turn.completed usage. on = through codexcomp, which detects a round ending at exactly 518·n − 2 reasoning tokens and replays the round's reasoning (incl. encrypted items) with a continue nudge; per-round data from the proxy's logs. Bypass isolation was verified (zero proxy traffic during off runs).

Results (each cell = 5 runs; "516 events" = runs where a model round ended at exactly 516 reasoning tokens)

| model | effort | mitigation | 516 events | reasoning tokens per run (sorted) | correct |
| --- | --- | --- | ---: | --- | ---: |
| gpt-5.4 | low | off | 0/5 | 155, 319, 323, 357, 478 | 4/5 |
| gpt-5.4 | low | on | 0/5 | 263–392 | 5/5 |
| gpt-5.4 | medium | off | 0/5 | 278–422 | 5/5 |
| gpt-5.4 | medium | on | 0/5 | 331–433 | 5/5 |
| gpt-5.4 | high | off | 2/5 | 497, 516, 516, 697, 796 | 5/5 |
| gpt-5.4 | high | on | 3/5 | 516+0, 516+20, 516+21, 686, 774 | 5/5 |
| gpt-5.4 | xhigh | off | 0/5 | 648, 758, 803, 922, 939 | 5/5 |
| gpt-5.4 | xhigh | on | 1/5 | 516+76, 634, 839, 957, 963 | 5/5 |
| gpt-5.5 | low | off | 0/5 | 217–276 | 5/5 |
| gpt-5.5 | low | on | 0/5 | 196–266 | 5/5 |
| gpt-5.5 | medium | off | 0/5 | 248–380 | 5/5 |
| gpt-5.5 | medium | on | 0/5 | 203–288 | 5/5 |
| gpt-5.5 | high | off | 0/5 | 313–415 | 5/5 |
| gpt-5.5 | high | on | 0/5 | 289–398 | 5/5 |
| gpt-5.5 | xhigh | off | 4/5 | 450, 516, 516, 516, 516 | 4/5 |
| gpt-5.5 | xhigh | on | 1/5 | 379, 471, 478, 516+0, 791 | 5/5 |

(516+k = round truncated at exactly 516, continuation produced k further reasoning tokens.)

Observations

1. The fixed point reproduces under controlled conditions. 11 of 80 runs (14%) had a round end at exactly 516. In final usage, 516 appeared 8 times; no other value appeared more than twice across all 80 runs. No run ever landed on 515 or 517.
2. It is effort-gated, not effort-monotonic. Hits concentrate where the task's natural reasoning length approaches 516 from below: gpt-5.5 at xhigh (5/10 runs) and gpt-5.4 at high (5/10). gpt-5.4 at xhigh naturally reasons 600–960 tokens on this task and sailed past the 516 boundary with only one hit — consistent with a floor/threshold at the boundary rather than a general "cut everything" rule.
3. gpt-5.4 exhibits it too. 6/40 gpt-5.4 runs (15%) had an exact-516 event — far above the 1.3% non-gpt-5.5 base rate in the OP's aggregate. The behavior may not be 5.5-exclusive; gpt-5.5 may simply have typical reasoning lengths that sit near the first boundary more often.
4. The only wrong answer above low effort was an exact-516 truncation. A gpt-5.5 xhigh off run truncated at 516 answered incorrectly (the classic #29353 pattern); the other five 516-truncated off runs stayed correct — the puzzle is deliberately easy enough to grade, so it underestimates damage on genuinely hard tasks. (The one other wrong answer in the whole grid was a gpt-5.4 low run at 155 tokens — ordinary low-effort error, included as base rate.)
5. Continuation recovers real reasoning. Of the 5 mitigated runs that hit 516, 3 resumed with additional reasoning after replay and 2 had nothing more to add; all 5 finished correct. Mitigated conditions went 40/40 correct overall vs 38/40 unmitigated.
6. Reasoning-token distributions for on vs off are otherwise similar, so the proxy does not appear to distort baseline behavior.

Caveats: n=5 per cell — treat per-cell rates as coarse; the 4/5-vs-1/5 asymmetry between 5.5-xhigh off/on shares the same upstream and is plausibly sampling noise. Single puzzle, single account, single window. Happy to share the harness (prompt, runner, grader) if useful.

<details>
<summary>Exact prompt used (all 80 runs)</summary>

Solve this logic puzzle by reasoning alone. Do not run any commands, do not write any code or files — pure reasoning only.

There are four houses numbered 1 to 4 from left to right. Each house has a different color (red, blue, green, yellow) and its occupant has a different name (Ada, Ben, Cleo, Dan), drinks a different drink (tea, coffee, milk, juice), and keeps a different pet (cat, dog, fish, bird).

Clues:
1. Ada lives in the red house.
2. Ben owns the bird.
3. The person in the green house drinks coffee.
4. Cleo drinks tea.
5. The green house is immediately to the left of the blue house.
6. The bird's owner drinks milk.
7. The person in the yellow house keeps the cat.
8. The person in house 2 drinks milk.
9. Dan lives in house 1.
10. Dan lives next to the blue house.
11. Ada drinks juice.
12. The cat lives in house 4.
13. Dan owns the dog.

Question: Who owns the fish? Verify your solution satisfies every clue before answering. Your final message must be only that person's name, nothing else.

Unique solution (brute-force verified over all 4!⁴ assignments): house 3, Ada owns the fish.
</details>

Ask your agent to plot frequency distribution, it is easier to visualise.

I'd expect reasoning token usages to be either a Pareto-like distribution or a Normal-like distribution. Definitely not spiky like this.
<img width="2381" height="1483" alt="Image" src="https://github.com/user-attachments/assets/7a1f12ed-0153-41d4-a1e1-827eb30ae1cd" />

Round 2: same 80-run A/B on a hard task (the community "candy" eval) — every unmitigated gpt-5.5 run was boundary-cut; accuracy 15% vs 90%

My previous experiment used a deliberately easy puzzle; the fixed point reproduced but accuracy barely moved. This round reruns the identical 80-run grid (gpt-5.4/gpt-5.5 × low/medium/high/xhigh × mitigation on/off × 5 reps, interleaved, one account, one ~95-minute window, all runs exited cleanly) on a genuinely hard task: the candy pigeonhole puzzle from haowang02/codex-candy-eval (the community 降智 eval). Prompt verbatim from that repo, graded by its own standalone-21 rule, --disable memories as in the eval. I independently verified the answer (21) by brute-force min-max before running.

Results (each cell = 5 runs; reasoning tokens are cumulative per run; bold = exact 518·n − 2 values)

| model | effort | mitigation | boundary-cut runs | reasoning tokens per run (sorted) | correct |
| --- | --- | --- | ---: | --- | ---: |
| gpt-5.5 | low | off | 5/5 | 516, 516, 516, 516, 516 | 0/5 |
| gpt-5.5 | medium | off | 5/5 | 516, 516, 516, 516, 516 | 0/5 |
| gpt-5.5 | high | off | 5/5 | 516, 516, 516, 516, 2588 | 1/5 |
| gpt-5.5 | xhigh | off | 5/5 | 516, 516, 516, 4660, 5696 | 2/5 |
| gpt-5.5 | low | on | 4/5 | 406, 1339, 1488, 1682, 1970 | 4/5 |
| gpt-5.5 | medium | on | 5/5 | 529, 1836, 2068, 2348, 3035 | 4/5 |
| gpt-5.5 | high | on | 4/5 | 0*, 2982, 3074, 3551, 3622 | 5/5 |
| gpt-5.5 | xhigh | on | 5/5 | 4658, 5694, 6212, 6546, 11392 | 5/5 |
| gpt-5.4 | low | off | 0/5 | 71, 90, 153, 501, 854 | 0/5 |
| gpt-5.4 | medium | off | 2/5 | 516, 516, 597, 1009, 2646 | 1/5 |
| gpt-5.4 | high | off | 2/5 | 516, 516, 725, 779, 1055 | 0/5 |
| gpt-5.4 | xhigh | off | 1/5 | 516, 5173, 5888, 6325, 6732 | 4/5 |
| gpt-5.4 | low | on | 1/5 | 162, 507, 577, 1962, 2958 | 1/5 |
| gpt-5.4 | medium | on | 1/5 | 705, 1274, 1756, 1858, 2078 | 3/5 |
| gpt-5.4 | high | on | 1/5 | 827, 951, 1191, 2778, 4713 | 2/5 |
| gpt-5.4 | xhigh | on | 4/5 | 1797, 5144, 5790, 6888, 7705 | 4/5 |

\* one run's continuation stream broke and the client retried against the (cached) continued context — usage reports the retry only.

Findings

1. Every single unmitigated gpt-5.5 run — 20/20, at every effort — ended its reasoning exactly on a 518·n − 2 value: 516 ×17, plus 2588 (n=5), 4660 (n=9), 5696 (n=11). Accuracy 3/20 (15%).
2. Which rung you land on decides the outcome. The three correct unmitigated gpt-5.5 runs are exactly the three cut at a higher rung. Pooling both models, the split is exact: every unmitigated run that reached ≥2588 reasoning tokens was correct (8/8); every one below that — 516-cut or naturally short — was wrong (0/32), including 0/22 for the 516-cut runs. The task simply needs more reasoning than 516 tokens, and the cut denies exactly that.
3. Effort does not buy your way out. Unmitigated gpt-5.5 at low, medium and high hit 516 in 14/15 runs; even at xhigh, 3/5 still got 516. Paying for xhigh mostly changes the chance of drawing a higher rung, not the cut itself.
4. Continuation-mitigation flips accuracy from 15% to 90% on gpt-5.5 (3/20 → 18/20; 18/20 mitigated runs triggered ≥1 continuation). Median wall time rose 26s → 86s — the continued rounds are real reasoning the model was cut off from.
5. The rung ladder is dense and extends to at least n=21. Across all rounds (including continuation rounds re-cut mid-fold) I observed exact terminations at n = 1, 2, 3, 4, 5, 6, 8, 9, 10, 11 and 21 (10876 tokens). Folds like 516 → 2588 → 2588 → clean re-truncate on the ladder repeatedly. Four mitigated xhigh runs were cut again beyond the proxy's n≤6 detection window (e.g. 516 → continue → cut at 10876 = 518·21−2) and still answered correctly with the extra budget.
6. gpt-5.4 is affected too, but differently: 5/20 unmitigated runs were 516-cut (all five wrong), yet at xhigh it usually ran 5.2–6.7k tokens with no cut (endings off-ladder). Whatever imposes the rungs seems to bind gpt-5.5 far more often than gpt-5.4 at the same efforts.

Three mitigated runs still failed after continuation (e.g. one 516-cut continued for only 13 further tokens and answered wrong) — continuation recovers budget, not a guarantee.

Caveats: n=5 per cell; one task, one account, one machine, one window (2026-07-06 19:23–20:57 UTC+8); grading is the eval's regex (a standalone "21" anywhere counts). Raw per-run data and the harness available on request.

Combined with the easy-task round (where cuts appeared only when natural reasoning length approached 516), the picture is consistent with server-side discrete reasoning-budget rungs at 518·n − 2, with gpt-5.5 assigned the lowest rung in the vast majority of requests regardless of configured effort — and task failure following directly from the cut.

I'm pretty sure this is just mini routing now (randomly), my sessions are now getting compacted like every 5 minutes with very little actually going on in them.

Groannnn. Comon this is definitely mini.

<img width="2530" height="674" alt="Image" src="https://github.com/user-attachments/assets/49db7e61-321a-41d3-8e1c-10dd107b7525" />

I can confirm your assumption seems correct, my usage shows I've used 5.4 mini, even though I've only exclusively used either 5.4 xhigh or 5.5 xhigh for testing.

Tibo has already posted that auto-review uses it btw.
That may be the reason. I don't think compaction is related btw. Check context usage

I'm pretty sure this is just mini routing now (randomly), my sessions are now getting compacted like every 5 minutes with very little actually going on in them.

I also looked at my previous history, and every day shows 5.4 mini usage even though I never use it. Here's an example from June 9th.

There was a bug labeling codex guardian auto review as gpt 5.4. So there's that.
And, I am running codex extensively, like nonstop, and I am affected by the PRO subscription issue described in this thread (as every other PRO user is), I have _zero_ gpt 5.4 in the usage log since the above mentioned bug was fixed.

As such I do not believe this is related. This is not a _rerouting_ as several experiments on this git have clearly and deterministically shown. It is a _reasoning_ issue.

Please lets stay on topic, not pollute the thread with too many assumptions, and not make this a playground for "I feels" reports.
This is a _serious_ issue, of course, and I am amongst the first going to cancel that plan if OAI does not react and make statements and adjustments in the coming hours/days. But, talking about our feelings and assumptions here is out of scope and not helping.

I can reproduce this issue on my setup too. (200 plan)

<details>
<summary>Click to expand</summary>
I used the same candy canary from https://github.com/openai/codex/issues/30364#issuecomment-4886357080

Definitions used below:

• Expected final answer: 21
• Suspicious reasoning-token values: {516, 1034, 1552}
• Affected: wrong final answer and reasoning_output_tokens is one of the suspicious values above

Environment:

• Codex CLI: 0.142.5
• VS Code / Codex extension version shown in attached screenshot: 26.5623.101652 (pre-release, current)
• Platform: WSL2 / Ubuntu 24.04
• Model metadata from ~/.codex/models_cache.json:
• model: gpt-5.5
• comp_hash: 2911
• context_window: 272000
• max_context_window: 272000
• use_responses_lite: false

I kept local artifact paths out of this comment, but each run wrote summary.json, raw JSONL, stderr, active config, and model metadata locally.

1. Linked command shape, Fast / priority on

Command profile:

codex exec --json --skip-git-repo-check --ephemeral -s read-only --disable memories -m gpt-5.5 -c model_reasoning_effort=high -c personality=none

Prompt delivery: stdin

Active config:

service_tier = "priority"

Differences from the linked comment script:

• I used a timestamped artifacts directory under benchmarks/.
• I recorded the active config and model metadata in summary.json.
• The command shape, prompt, model, reasoning effort, and personality match the linked script.
• Active config had service_tier = "priority" because Fast mode was enabled.

Results:

| Run | Reasoning tokens | Final answer | Affected |
| --- | ---: | ---: | --- |
| 1 | 516 | 17 | yes |
| 2 | 516 | 23 | yes |
| 3 | 516 | 21 | no |
| 4 | 516 | 26 | yes |
| 5 | 516 | 23 | yes |

Verdict: affected. All 5 runs landed exactly on 516; 4/5 were wrong.

2. Linked command shape, Fast / priority off

Same linked command profile:

codex exec --json --skip-git-repo-check --ephemeral -s read-only --disable memories -m gpt-5.5 -c model_reasoning_effort=high -c personality=none

Prompt delivery: stdin

Active config:

service_tier = "default"

Differences from the linked comment script:

• I used a timestamped artifacts directory under benchmarks/.
• I recorded the active config and model metadata in summary.json.
• The command shape, prompt, model, reasoning effort, and personality match the linked script.
• Active config had service_tier = "default" because Fast mode was disabled.

Results:

| Run | Reasoning tokens | Final answer | Affected |
| --- | ---: | ---: | --- |
| 1 | 4142 | 21 | no |
| 2 | 5178 | 21 | no |
| 3 | 516 | 26 | yes |
| 4 | 516 | 27 | yes |
| 5 | 516 | 28 | yes |

Verdict: affected. The higher-reasoning runs were correct; all exact-516 runs were wrong.

3. Current local config, Fast / priority on

Same prompt and parser, but no -m or -c overrides. Active config selected model, reasoning effort, personality, and service tier.

Command profile:

codex exec --json --skip-git-repo-check --ephemeral -s read-only --disable memories

Prompt delivery: stdin

Active config:

model = "gpt-5.5"
model_reasoning_effort = "xhigh"
personality = "pragmatic"
service_tier = "priority"

Differences from the linked comment script:

• I omitted -m gpt-5.5, -c model_reasoning_effort=high, and -c personality=none.
• The active config selected model = "gpt-5.5", model_reasoning_effort = "xhigh", and personality = "pragmatic".
• Active config had service_tier = "priority" because Fast mode was enabled.
• Prompt delivery stayed stdin.
• The same candy prompt and parser were used.

Results:

| Run | Reasoning tokens | Final answer | Affected |
| --- | ---: | ---: | --- |
| 1 | 8286 | 21 | no |
| 2 | 516 | 24 | yes |
| 3 | 516 | 13 | yes |
| 4 | 516 | 21 | no |
| 5 | 516 | 26 | yes |

Verdict: affected. The long-reasoning run was correct; most exact-516 runs were wrong.

Overall this is consistent with the reported failure mode: gpt-5.5 sometimes lands exactly on reasoning_output_tokens=516, and on this canary those exact-516 runs are frequently wrong. Fast mode / service_tier=priority changes the distribution in my small samples, but it does not eliminate the failure mode.</details>

I had suspected about auto-review so anyway fair enough. But anecdotally my experience has gone from xhigh is agi to dumb as bricks and I already cancelled. Will renew if/when this is fixed.

Wondering if a fucks chart would be an actual useful metric here because i’m pretty sure over the last 8 weeks mine went 📈

Same issue - any humans alive in this repo?

Ran the prompt from this thread. $100 plan – codex from launch monster slowly became git deployer / smoke checker instead of developer.

<img width="1260" height="586" alt="Image" src="https://github.com/user-attachments/assets/a2b19c5e-e39e-44b2-b187-66feeb122f69" />

After trimming "Intermediary updates" section from model instructions:

<img width="1644" height="622" alt="Image" src="https://github.com/user-attachments/assets/2d12db8a-1a67-43b7-8f8e-17763e268524" />

Hi, I am op for thread + here is git repo with my own analysis + skills/scripts ---> So, I can confirm that the test task (minimum pulls for candies problem) is still getting better results on reasoning. But, I've run a python script summing reasoning before and after the fix (had several hours of heavy use following). Removing the instruct section helps only for the benchmark at least on reasoning on gpt 5.5. It does not appear to help with normal usage. HOWEVER: It does seem to drastically reduce the calls made to gpt 5.4 mini and gpt 5.4, but that is prelim result. I will update the mentioned repo with (anonymized) proofs as I work on this. More detailed writeup is the readme on that repo, but here is quick summary (cutoff means when I made change to remove problem section):

<img width="2554" height="1671" alt="Image" src="https://github.com/user-attachments/assets/03e43925-f3c3-48aa-b7c1-83b25c4fcce2" />

Hey Everyone.

OP of the issue here. I'm glad everyone has resonated with this issue. We are 5th currently in the leaderboard of issues as per Total Comments.

However, still no response by any of the maintainers. I've implemented the system prompt edits suggested + manual reasoning nudge and they seem to be working, but not a long term solution. Seems to be some internal quantization.

Ive posted on X twice-

https://x.com/virengupta45/status/2073774149852692940?s=46

https://x.com/virengupta45/status/2071965692862386567?s=46

Please engage with them so Tibo has to address it eventually. And do post your own investigations on X individually too.

You guys are very late to the party. I have traces from release date of 2026-04-23 and can confirm that i had 57% of reasoning traces with 516 tokens.

It's a very much intended feature, that's why first issue was closed with "not planned". They likely have some kind of classifier in place and i don't think they will openly discuss inference of the models

Well I think it is interesting that the change seems to fix the benchmark task and only the benchmark task. I would like an explanation from openai staff on that one. 🤨

Strange that there's no acknowledgment from any maintainers about this issue. I wonder if this is a feature they built in to save compute and figured nobody would notice.

Having the same issue.

Gives up on tasks/solves part way through; notably degraded performance, attempting the "hotfix"

With all due respect, how has this not been heavily prioritised?
Performance is significantly degraded across the entire Codex user base for a LONG time now, and worse yet the degradation has been silent and without official acknowledgement. Come on Codex Team, this is seriously disappointing :/

It look like adding Always start with a message in the commentary channel before starting work seems to work just as well as removing the commentary entirely on the synthetic candy test.

Can confirm that both options seem to work in my testing. I'm gonna keep that section in with that sentence added.

I analyzed my local Codex sessions. I mostly use gpt-5.5 with xhigh in codex cli, and the 516 pattern is definitely visible in the logs.

However, in my own sessions I do not see a strong correlation between exact 516-token reasoning turns and later rework/failure markers. In my workflow it appears especially often during bounded, structured planning or prompt-generation turns for implementation work, where the plan has already gone through audits/reviews, ambiguities have been resolved, and decisions are frozen. In those cases, the turn is often more about turning already-settled context into a checklist, prompt map, or implementer prompt than about solving the whole problem from scratch.

It seems 516 is a real model/accounting/control-flow pattern, but it does not always imply degraded output quality. It may be harmful in single-shot reasoning tasks, while being less harmful in workflows where the hard reasoning happened in earlier audit/review turns.

I'm seeing the same suspicious fixed-token clustering pattern locally too, especially in gpt-5.5 / xhigh. Model-level result from my local logs:

| Model | Response records | Exact 516 / >=516 |
| --- | ---: | ---: |
| gpt-5.5 | 10,375 | 42.87% |
| gpt-5.4 | 5,782 | 22.38% |
| gpt-5.3-codex | 6,665 | 0.58% |
| gpt-5.2-codex | 97 | 0.00% |
| gpt-5.3-codex-spark | 12 | 0.00% |

The month-by-month data shows the exact-516 clustering rising sharply in recent months, while overall reasoning-token intensity stayed low / dropped from earlier months.

This lines up with what I've been feeling in real usage over the last few weeks. I started comparing the same planning/grilling prompts across GLM-5.2, Kimi 2.7, Composer 2.5, Anthropic models, and GPT-5.5 high/xhigh. Anecdotally, GPT-5.5 was often the least reliable in those comparisons, despite being the model I expected to perform best if you exclude Anthropic ones.

Here's the candy test results before & after adding the following line to ~/.codex/AGENTS.md:

Always start with a short message in the commentary channel before starting work.

Before:

| Run | reasoning_output_tokens | Final agent answer |
|---:|---:|---|
| 1 | 516 | 25 |
| 2 | 6214 | 21 |
| 3 | 516 | 18 |
| 4 | 516 | 21 |
| 5 | 516 | 19 |

After:

| Run | reasoning_output_tokens | Final agent answer |
|---:|---:|---|
| 1 | 6633 | 21 |
| 2 | 5180 | 21 |
| 3 | 518 | 21 |
| 4 | 5279 | 21 |
| 5 | 4144 | 21 |

Hopefully this gets fixed soon 🙏

same here, here's claude code's analysis

Independent reproduction from a separate corpus — I ran the same analysis against my own local Codex history and can corroborate the fixed-boundary clustering, including with model_reasoning_effort = "xhigh".

Methodology

• Data source: local session rollout files (~/.codex/sessions/**/*.jsonl + ~/.codex/archived_sessions/**/*.jsonl)
• Metric: event_msg → token_count → info.last_token_usage.reasoning_output_tokens (per-response), model attributed from the preceding turn_context
• Corpus: 1,001 sessions, 124,591 response-level token records, Nov 2025 – Jul 2026 (bulk Apr–Jul 2026)
• Config during the affected window: model = "gpt-5.5", model_reasoning_effort = "xhigh"

Result 1: same boundary clustering, same magnitude

| Model | Response records | ==516 | ==1034 | ==1552 | exact-516 / ≥516 |
| --- | ---: | ---: | ---: | ---: | ---: |
| gpt-5.5 | 86,940 | 2,354 | 199 | 54 | 45.4% |
| gpt-5.4 | 23,304 | 841 | 124 | 43 | 23.3% |
| gpt-5.4-mini | 4,205 | 102 | 29 | 15 | 11.0% |
| gpt-5.1-codex-max | 3,321 | 0 | 0 | 0 | 0.0% |
| codex-auto-review | 2,569 | 0 | 0 | 0 | 0.0% |
| gpt-5.1 | 1,669 | 0 | 0 | 0 | 0.0% |
| gpt-5.2-codex | 854 | 0 | 0 | 0 | 0.0% |
| gpt-5.2 | 585 | 0 | 0 | 0 | 0.0% |
| gpt-5.3-codex-spark | 435 | 0 | 0 | 0 | 0.0% |

My gpt-5.5 exact-516/≥516 ratio (45.4%) independently reproduces the 44.0% reported in this issue. The distribution around the boundary rules out natural variation:

reasoning_output_tokens (gpt-5.5):
  515 →    25
  516 → 2,354
  517 →     3
 1034 →   199   (neighbors 1031–1037: 0–4 each)
 1552 →    54   (neighbors 1549–1555: 0–2 each)

516 is the single most common nonzero reasoning-token value in the entire corpus, ahead of the small values (6–14) that dominate trivial turns. Note the boundaries are spaced 518 apart (516, 1034, 1552), consistent with a repeated increment rather than independent thresholds.

Monthly, gpt-5.5 exact-516/≥516 stayed in a tight band for me: Apr 42.7%, May 47.8%, Jun 45.2%, Jul 42.1% (I have little pre-April gpt-5.5 traffic, so I can't reproduce the Feb–Mar ramp).

Result 2: the truncation concentrates in turn-final (user-facing) responses

New angle that may explain the "degraded performance on complex tasks" reports: I isolated turn-final responses — the last model response before the next human message, i.e. the one that produces the user-facing conclusion (completion claim, blocker declaration, validation verdict).

• Across all gpt-5.5 responses: 45.4% of those reaching ≥516 stop exactly at a boundary.
• Among turn-final responses reaching ≥516: 83–92% stop exactly at a boundary.
• Share of all turn-final gpt-5.5 responses that were boundary-truncated, by month: Apr 7.9% → May 10.3% → Jun 36.4% → Jul 39.0%.

So by June, more than a third of the final answers I read were produced by a response whose reasoning halted exactly at a fixed boundary — precisely at the judgment-heavy step where wrong "done/blocked/validated" claims get made.

Result 3: incident-level linkage (anecdotal but consistent with #29353)

I regex-matched my own correction/pushback messages (challenging a false completion claim, a false "Blockers:" declaration, or an unverified validation claim) and looked up the reasoning-token count of the immediately preceding response. 28.6% of those correction moments followed a boundary-truncated response (baseline for all my messages: 21.7%) — a modest lift, but the flagship incidents (false "validation passed" claims contradicted one line later, invented blockers on credentials that existed, and losing track of the task mid-thread) each sat directly on a reasoning_output_tokens = 516 response.

Caveats

• Selection effect: complex moments are more likely to reach 516 tokens regardless, so incident co-occurrence alone is weak; the boundary spike itself (2,354 vs single-digit neighbors) is the hard evidence.
• All numbers come from client-side token_count telemetry as recorded in rollout files; I can share the extraction script if useful.

@omederos Adding a line works for me!

<img width="540" height="246" alt="Image" src="https://github.com/user-attachments/assets/2a4d6857-0043-43cb-86d2-0650738d10ed" />

@omederos Adding a line works for me! <img alt="Image" width="540" height="246" src="https://private-user-images.githubusercontent.com/144095485/617861582-2a4d6857-0043-43cb-86d2-0650738d10ed.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3ODMzODg1MjYsIm5iZiI6MTc4MzM4ODIyNiwicGF0aCI6Ii8xNDQwOTU0ODUvNjE3ODYxNTgyLTJhNGQ2ODU3LTAwNDMtNDNjYi04NmQyLTA2NTA3MzhkMTBlZC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjYwNzA3JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI2MDcwN1QwMTM3MDZaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1iMWFjNjIzYmVmMTA0YjNiY2UxN2MwOWQ5NDIyMWZhNTQyM2UzMDIwZWNmNzk2YTAyOGMzYTBhZjg4MmYzNmNjJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZyZXNwb25zZS1jb250ZW50LXR5cGU9aW1hZ2UlMkZwbmcifQ.ghPRjiNxDMi8AygY4LpLTLoeZ_ZF-417LsQvA_d0bVM">

This is a shallow fix, it only fixes the first message.

This issue caused three heart monitors in the PICU to misreport atrial fibrillation

> @omederos Adding a line works for me! > <img alt="Image" width="540" height="246" src="https://private-user-images.githubusercontent.com/144095485/617861582-2a4d6857-0043-43cb-86d2-0650738d10ed.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3ODMzODg1MjYsIm5iZiI6MTc4MzM4ODIyNiwicGF0aCI6Ii8xNDQwOTU0ODUvNjE3ODYxNTgyLTJhNGQ2ODU3LTAwNDMtNDNjYi04NmQyLTA2NTA3MzhkMTBlZC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjYwNzA3JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI2MDcwN1QwMTM3MDZaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1iMWFjNjIzYmVmMTA0YjNiY2UxN2MwOWQ5NDIyMWZhNTQyM2UzMDIwZWNmNzk2YTAyOGMzYTBhZjg4MmYzNmNjJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZyZXNwb25zZS1jb250ZW50LXR5cGU9aW1hZ2UlMkZwbmcifQ.ghPRjiNxDMi8AygY4LpLTLoeZ_ZF-417LsQvA_d0bVM"> This is a shallow fix, it only fixes the first message.

Since we don't know the underlying mechanism, whether we tweak just the first message or not, it ultimately resolved the shallow reasoning issue. Doesn't that just make it a "temporary" generic workaround?

Related community workaround PR opened against codexcomp: https://github.com/dzshzx/codexcomp/pull/5

It keeps the existing 518n - 2 continuation/fold behavior and adds a narrowly gated zero-reasoning retry for high-effort gpt-5.5 requests. The added tests use synthetic Responses event streams. This is intended as a mitigation path, not a claim about upstream root cause.

@vguptaa45 I think we should be careful treating reasoning_tokens == 0 as retryable.

I tested a local version of this idea and found that there is not yet a stable/general fix for the zero-reasoning case. In normal interactive Codex use, high-effort gpt-5.5 can return reasoning_tokens == 0 while still producing a good complete answer, especially when the user asks a simple clarification/confirmation question.

In that case, retrying makes the result worse: the proxy hides the good first answer, sends the “Continue thinking...” nudge, and subsequent rounds tend to produce shorter/degraded answers.

Example live shape:

round 1: reasoning_tokens=0, good complete answer -> retried/hidden
round 2: reasoning_tokens=0, shorter answer -> retried/hidden
round 3: reasoning_tokens=0, very short answer -> retried/hidden
round 4: reasoning_tokens=0, final short answer after max retries

So high-effort gpt-5.5 + reasoning_tokens == 0 is not sufficient evidence of a broken response. Sometimes the answer simply does not need hidden reasoning.

I think the existing 518n - 2 continuation path is much safer because it is based on a strong truncation fingerprint.

Here's the full recipe for the temp fix provided by other people, step by step, feel free to use it. I have been consistently getting the right output for this canary prompt (https://github.com/openai/codex/issues/30364#issuecomment-4885272531)

Give this comment to an AI agent, it should fix it as well one-shot, good luck!

1. Install the proxy (needs uv; brew install uv if you don't have it):

# back up your config first
BACKUP="$HOME/.codexcomp-backup/$(date +%Y%m%d-%H%M%S)"
mkdir -p "$BACKUP" && cp -p ~/.codex/config.toml "$BACKUP/codex.config.toml"

# install codexcomp and run it as an autostart service
# (launchd on macOS, systemd on Linux/WSL; logs: /tmp/codexcomp.log)
uv tool install "git+https://github.com/Stoffberg/codexcomp.git@fix/compaction-passthrough"
codexcomp install-service

2. Point Codex at it: add this one line to the top of ~/.codex/config.toml (above any [section] header) and change nothing else. Using openai_base_url instead of a new model_provider keeps your chat history visible:

openai_base_url = "http://127.0.0.1:8787/v1"

3. Check it works:

lsof -nP -iTCP:8787 -sTCP:LISTEN    # a Python process should be listening
codex exec --skip-git-repo-check "Reply with exactly: PROXY TEST OK"
tail -20 /tmp/codexcomp.log         # you should see the request go through the proxy

Then restart the Codex desktop app, CLI and app share the same config.

To undo everything later:

codexcomp uninstall-service
uv tool uninstall codexcomp
# then delete the openai_base_url line from ~/.codex/config.toml
# (your original config is saved in ~/.codexcomp-backup/)

<details>
<summary>Edit: why the install points at my branch and not PR 5 directly</summary>

An earlier version of this comment installed the head of dzshzx/codexcomp#5, which broke Codex auto-compaction, every compaction failed with upstream_error. Compaction turns legitimately report zero reasoning tokens, which tripped that branch's zero-reasoning retry, and the API rejects any continuation because compaction_trigger must stay the final input item. My branch is PR 5 plus the fix (submitted upstream as vguptaa45/codexcomp#1). With it, compaction turns log stop=compaction_passthrough and complete normally.

</details>

Is anyone else seeing this, or is it just my account?

I have model_reasoning_summary = "detailed" turned on in my config.toml, and it was working fine until a few hours ago. Now Codex still asks for detailed summaries, but the server no longer sends back any readable summary text.

I set up a local Responses proxy that only logs request metadata — not the actual prompts or content — and confirmed the request is being sent with:

"reasoning": { "effort": "high", "summary": "detailed" }

But what comes back now looks like this:

response.reasoning_summary_text.delta = 0
response.reasoning_summary_text.done  = 0
reasoning item summary_nonempty      = 0
reasoning item encrypted_content     = present
summary = []

So the model is still reasoning, and encrypted reasoning data is still there, and I'm still being charged for reasoning tokens. But there is no readable summary for Codex to save or show. That's also why my session log file no longer has agent_reasoning entries — Codex only creates those from non-empty summary text.

One thing that made this hard to debug: full websocket request logging was removed in db887d03e1 (#30757). I understand the privacy concern, but without any safe logging it's hard to tell whether Codex actually sent summary=detailed or whether the server just ignored it. I had to set up a proxy just to confirm the request was right.

Short version of what I'm seeing:

reasoning.summary requested as detailed
encrypted_content present
reasoning tokens billed/recorded
summary = []
no agent_reasoning in rollout

This is frustrating because those summaries were one of the only ways to understand what the model was doing when it gave a bad answer.

Update from the wire-level vantage point: the fixed point is not effort-gated, and notes for workaround users

Follow-up to my 80-run A/B grids. Three items.

1. All four efforts hit the boundary on the same prompt, concurrently

Today I ran the candy canary once per effort, all four codex exec runs launched in parallel in the same minute (CLI 0.142.5, gpt-5.5, ChatGPT backend, one account). Round 1 of every run — low, medium, high, and xhigh — ended at exactly 516 reasoning tokens:

| effort | round-1 reasoning | after continuation (total) | final answer |
| --- | ---: | ---: | --- |
| low | 516 | 2064 | 21 ✅ |
| medium | 516 | 5404 | 21 ✅ |
| high | 516 | 4739 | 21 ✅ |
| xhigh | 516 | 6057 | 21 ✅ |

This lines up with the per-model spreads @omederos and @igor-im posted: the fixed point is a gpt-5.5 property, not a high-effort property — low effort lands on it just as readily; you only notice at high efforts because that's where you expected thousands of reasoning tokens. All four runs recovered to the correct answer once the cut round was continued (rescued reasoning ranged from 4× to 12× the truncated amount).

2. On reasoning_tokens == 0 as a retry trigger

@abushadab's caution matches my review of the proposed retry (dzshzx/codexcomp#5, review): zero is not a fingerprint. Mechanically, a zero-reasoning round has no encrypted reasoning items to replay, so a "continuation" degenerates into a re-ask with stacked nudges — which is exactly why it produced progressively shorter answers in abushadab's live test, and why it fired on remote-compaction turns (legitimately zero-reasoning) and broke auto-compaction in @Stoffberg's field report. 518n−2 earned its retry with fixed-point clustering; zero-and-wrong vs zero-and-fine are indistinguishable at the wire. The commentary-channel prompt instruction discussed above remains the only mitigation I know of for that failure shape.

3. For anyone using the proxy workaround

codexcomp v0.3.4 is on PyPI as of today; it includes natively what @Stoffberg's branch hotfixed — remote-compaction requests (input ending in compaction_trigger) are never folded regardless of what the detector says — plus mirroring of the upstream x-codex-* rate-limit response headers that the proxy previously dropped. So the plain official install now suffices for his step-by-step recipe:

uv tool install codexcomp@latest   # 0.3.4; upgrade note: a pinned install needs @latest, not `uv tool upgrade`

Thanks @Stoffberg for the field report — a compaction turn failing under a zero-reasoning retry was exactly the evidence that settled the review above.

As before: this is a mitigation, not a claim about root cause. The 518n−2 quantization is upstream's to fix, and this thread still has no official acknowledgement.

Adding one more data point because it may help distinguish two different “cache” discussions here (prompt-cache ratio vs the local model metadata cache).

Environment:

• Windows
• codex-cli 0.142.5
• gpt-5.5, model_reasoning_effort=xhigh
• Same machine, same Codex config, same candy eval prompt

Local A/B observation:

1. ~/.codex/models_cache.json writable / allowed to refresh:

• candy eval returned the wrong answer
• final answer excerpt: Minimum: 11 candies
• reasoning_output_tokens=516
• duration about 42s

1. Restored a previously known-good ~/.codex/models_cache.json backup and set that file read-only:

• same eval returned the correct standalone answer: 21
• observed reasoning_output_tokens in the thousands (6732 and 9175 on two runs)
• duration about 135-173s
• the expected cache write error appeared because the file was read-only

Interpretation: on this machine, letting the local models_cache.json refresh was enough to reintroduce the exact 516-token wrong-answer failure mode. Restoring and locking the known-good cache removed it for the candy eval. This does not prove the whole issue is only models_cache.json, and it is separate from the prompt-cache-ratio telemetry correlation, but it suggests at least one reproducible failure path is coming through refreshed cached model metadata/base instructions.

Practical test that matched this locally: unlock models_cache.json, rerun the candy eval, observe 516 + wrong answer; restore/lock the prior cache, rerun, observe correct 21 with much higher reasoning tokens.

@dzshzx I opened a broader reliability PR here: https://github.com/dzshzx/codexcomp/pull/6

It includes the work I’ve been testing locally around Codex WebSocket/state handling, safer hidden continuations, better cleanup/cancellation, zero-reasoning retry removal, and metadata-only diagnostics/health checks.

The main goal is to make codexcomp more reliable for real interactive Codex sessions, where previous_response_id, incremental WS turns, continuation folding, and upstream truncation behavior all interact.

I'm having the same issue. Has codex responded to this issue. My codex client keeps ignoring my agents.md file and model instructions pretty much constantly, and I think that this is the core issue.

when I ran my tests i got 516 every single time. Even after adding some terminology to prevent this as written in this thread im getting 516 tokens

did a cleanroom on this, this is a good working solution, my result below (only few turns so far, but so far so good)

It's not just going through the proxy — the proxy has already rescued this very thread. In real traffic since wiring: 2 folds, and they're both from minutes ago, back-to-back in one request: round 1 truncated at 516 → continuation also truncated at 516 → second continuation finished the thought. That's the reasoning = 1098 response in your thread's rollout at 15:41:46 (516 + 516 + 66 to finish) — a double-amputated response that, without the proxy, would have shipped its conclusion on ~516 tokens of thinking. You would have read that answer.

So, summary of what's observable right now:

• Routing: confirmed per-request — even this July 5 thread picked up the proxy without a restart; all threads are covered.
• Traffic: steady flow through the proxy, timestamps and token counts matching your rollout 1:1.
• Folding: live and working on real work, including the nastiest case (consecutive truncations), well inside the MAX_CONTINUE=3 bound.
• Health check anytime: tail ~/.codex/fold_proxy/proxy.log — one done: line per response; continue ( lines are folds in action.

@dzshzx I opened a broader reliability PR here: dzshzx/codexcomp#6 It includes the work I’ve been testing locally around Codex WebSocket/state handling, safer hidden continuations, better cleanup/cancellation, zero-reasoning retry removal, and metadata-only diagnostics/health checks. The main goal is to make codexcomp more reliable for real interactive Codex sessions, where previous_response_id, incremental WS turns, continuation folding, and upstream truncation behavior all interact.

@Stoffberg Thank you so much, this works incredibly well.

codexcomp v0.3.4 proxy works well for me, at least for today.

Thanks @dzshzx, @Stoffberg

edit: shared a bit more details in https://github.com/openai/codex/issues/30364#issuecomment-4908844165 below

what does "works well" mean?
Have you guys actually run real world tasks over several hours, and had a deterministic analysis on the reasoning tokens done over those inferences during the "fixed" time?
Or did you just do a canary test and assumed it to be fixed? Have you _zero_ low reasoning responses (unless its a synthetic turn) like you should, like you would have on a PLUS account?

Specifics would help on the case.

This tracks the performance of Codex CLI + GPT 5.5 xhigh on SWE bench pro, and apparently there is no degradation that that particular benchmark: https://marginlab.ai/trackers/codex/

🤷‍♂️

I also ran some informal tests on this https://huggingface.co/datasets/meituan-longcat/AMO-Bench and ran into the same issues. Seems like these kinds of tasks are hit the hardest by this issue.

This tracks the performance of Codex CLI + GPT 5.5 xhigh on SWE bench pro, and apparently there is no degradation that that particular benchmark: https://marginlab.ai/trackers/codex/ 🤷‍♂️

You cannot show degradation if the thing is not on a PRO account friend. Have you missed the fact that this affects only a certain level of accounts? PLUS users are not affected. Those websites use the API anyway, which, in my tests... _did_ show the issue, but with a weird pattern (affected in "batches" rather than evenly distributed). Which might explain that degradation oscillation on the reports you link to. In case you did not not notice those reports vary by a whopping 17+% in pass rate the past month alone and 40+% over all time.

Actually if you study those charts there you can see that the degradation is tracked, it is there for all to see.
Models increase, yet performance not. Go and compare that with Claude's historical track. Then you will see what I mean

> This tracks the performance of Codex CLI + GPT 5.5 xhigh on SWE bench pro, and apparently there is no degradation that that particular benchmark: https://marginlab.ai/trackers/codex/ > 🤷‍♂️ You cannot show degradation if the thing is not on a PRO account friend.

If you have access to Plus vs Pro go ahead and run those for us then and share the results 🙏. It would be interesting to see if the proxy fix improves real SWE tasks as well.

The benchmark is public: https://marginlab.ai/explorers/swe-bench-pro/

@smileBeda Apologies, I didn’t post specifics earlier because they mostly line up with what others have already reported.

On the candy eval with high reasoning, I get 516 reasoning tokens across 5/5 attempts without the proxy, with 0% accuracy every time. With the proxy, I can push it to around 4,000-5,000 reasoning tokens in up to 4/5 attempts (up to 80% accuracy after several runs, emphasis on up to).

So not claiming this is a fix, but it is clearly better than what I was seeing before. More importantly, I can feel the difference in the consistency of Codex output on my actual tasks, though I haven’t pushed anything past 20 minutes today. Just sharing the experience in case someone is still on the fence, feeling frustrated, and hasn’t tried anything yet.

In case you did not not notice those reports vary by a whopping 17+% in pass rate the past month alone and 40+% over all time.

The oscillation is expected. LLMs are not deterministic. Claude shows the same oscillation, and this is the reason why there is a statistical significance threshold that the results have to cross to be considered viable. This is basic statistics which you are misunderstanding

Actually if you study those charts there you can see that the degradation is tracked, it is there for all to see. Models increase, yet performance not. Go and compare that with Claude's historical track. Then you will see what I mean

This is not a sign of degradation as you claim, it's a sign that you did not do your research, and you don't know what you're talking about

You could have easily looked up benchmarks released by OpenAI themselves, these are the expected results:

• GPT-5.3-Codex: 56.8%
• GPT-5.4: 57.7%
• GPT-5.5: 58.6%

(sources https://openai.com/index/introducing-gpt-5-4/, https://openai.com/index/introducing-gpt-5-5/)

Notice that it did not go up by much, which explains the relatively flat trend between GPT model releases. Compare that to Claude:

• Opus 4.6: 53.4%
• Opus 4.7: 64.3%
• Opus 4.8: 69.2%

Which explains why that benchmark shows clear trends upwards between Claude model releases.

PLUS users are not affected

You keep claiming that PLUS users are not affected, but you have presented no compelling data.

---

The results in this benchmark don't mean that there is no degradation, but it could be that in multi-turn tasks, like SWE, the later thinking blocks end up correcting the earlier degraded ones. But with one-shot tasks like the candy test, the degraded thinking block is much more damaging.

So yes, the candy test is valuable, and is an easy way to reproduce this issue.

Also the proxy solution has been performing well in my own tests, hopefully the codex team fixes this issue soon.

Follow-up from the same local machine as my earlier aggregate: the pattern is still present in records after that comment, including after my Codex Desktop app updated.

Environment / source delta:

• Prior local comment: https://github.com/openai/codex/issues/30364#issuecomment-4884453952
• New window scanned: after 2026-07-05T01:49:39Z through 2026-07-07T22:15:41Z
• Current versions: Codex Desktop 26.623.141536, codex-cli 0.142.5
• Current models_cache.json metadata for gpt-5.5: comp_hash=2911, context_window=272000, max_context_window=272000, use_responses_lite=false
• Parsed token_count metadata only; no prompts, assistant messages, session IDs, thread names, local paths, file names, or cwd values included
• turn_context attribution still shows comp_hash=2911; parsed model_context_window in the token events is 258400

Post-comment aggregate:

| Scope | Records | Sessions | Mean reasoning | P90 | =512 | >=512 | =512 / >=512 | =516 | >=516 | =516 / >=516 | =1034 | =1552 | =2070 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| all after cutoff | 1,202 | 22 | 204.4 | 516 | 1 | 183 | 0.55% | 88 | 180 | 48.89% | 8 | 3 | 1 |
| gpt-5.5 / xhigh | 1,160 | 20 | 191.5 | 516 | 1 | 170 | 0.59% | 84 | 167 | 50.30% | 8 | 3 | 1 |
| gpt-5.4 / xhigh | 42 | 2 | 559.0 | 1,325 | 0 | 13 | 0.00% | 4 | 13 | 30.77% | 0 | 0 | 0 |

Daily split:

| Date UTC | Records | Sessions | Mean reasoning | P90 | =516 | >=516 | =516 / >=516 | =1034 | =1552 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 2026-07-05 | 356 | 3 | 137.3 | 435 | 13 | 29 | 44.83% | 1 | 0 |
| 2026-07-06 | 432 | 15 | 247.7 | 516 | 34 | 77 | 44.16% | 1 | 2 |
| 2026-07-07 | 414 | 7 | 216.9 | 516 | 41 | 74 | 55.41% | 6 | 1 |

Small note: I saw one fresh reasoning_output_tokens=512 record, which is why I checked both 512 and 516. In this local post-comment window, 512 does not look like a replacement boundary; 516 remains the dominant fixed point by far, with smaller rungs at 1034, 1552, and 2070.

Local takeaway: on this machine, the fixed-value pattern persisted after the earlier comment and after the Desktop app version changed, while the gpt-5.5 model metadata still reports comp_hash=2911.

The failure does not appear to be caused by the 518*n-2 pattern itself, but by the model collapsing into the minimum reasoning block: 516 tokens. Both PRO and PLUS show the same failure mode with ## Intermediary updates. In these targeted tests, PLUS appears to quantize more consistently than PRO: without ## Intermediary updates, PLUS falls into a quantized 518*n-2 block in 100% of runs, while still producing correct answers. This suggests that quantization may be normal, or at least harmless at higher blocks. The damaging case appears to be specifically the first block, 516, where reasoning seems to end too early.

I've updated the test results here: https://github.com/openai/codex/issues/30364#issuecomment-4886606283

I've updated the test results here: #30364 (comment)

So PRO and PLUS users are both affected? Thanks for the data.

Thats not what I’ve seen in _extensive and deterministic_ tests using real world data as well as dummy tests. These approaches have been shared above. Also historical data of plus vs pro was analysed.
Plus is NOT affected by this on my tests - it reliably did not produce a _single_ failure.

I submitted in-product Codex feedback with session/log sharing for this issue.

Feedback/session ID: 019f3053-7f43-76b0-af62-4e7863889c05

The session includes local repeated canary runs showing gpt-5.5 high clustering at 516 reasoning tokens, wrong answers on the canary, comparison against gpt-5.4 xhigh, and the user impact: I’m currently reluctant to use my paid Codex plan for complex work because I can’t trust that full reasoning was applied without manually inspecting telemetry.

I'm on the Pro $100 monthly plan.

This tracks the performance of Codex CLI + GPT 5.5 xhigh on SWE bench pro, and apparently there is no degradation that that particular benchmark: https://marginlab.ai/trackers/codex/ 🤷‍♂️ I also ran some informal tests on this https://huggingface.co/datasets/meituan-longcat/AMO-Bench and ran into the same issues. Seems like these kinds of tasks are hit the hardest by this issue.

this thread only applies to subsrcription, api does seem to have bucketing but on all my testing it never went below ~1500
so likely they are running api so not really getting hit by this

what does "works well" mean? Have you guys actually run real world tasks over several hours, and had a deterministic analysis on the reasoning tokens done over those inferences during the "fixed" time? Or did you just do a canary test and assumed it to be fixed? Have you _zero_ low reasoning responses (unless its a synthetic turn) like you should, like you would have on a PLUS account? Specifics would help on the case.

here's my, didnt do too much todayu, but seems to work well

Here's today's full usage analysis (all times local, UTC−5).

Proxy health — up 10h, zero operational failures

• 86 requests served since the LaunchAgent came up at 09:54 (plus the earlier manual runs). Every one finished stop=natural — nothing ever hit the max_continue=3 cap or the 15-min deadline, so the loop-guard has never had to intervene.
• The 82 "error-ish" lines in the log decompose to: 30 UnicodeDecodeErrors, all at 09:05 — the pre-zstd-fix smoke test, before the python3.14 restart; and 11 BrokenPipeErrors scattered at 09:55 / 13:52 / 18:37 — client-side disconnects (Codex's websocket probes / aborted streams), harmless. Zero errors from the fold logic itself.

Folds — 9 requests folded, 10 truncations caught

| when | what happened |
|---|---|
| 09:08–09:12 (×7) | validation-wave folds (my experiment traffic) |
| 10:41 | real work, thread 019f321e: double-fold — 516 → 516 → finished, delivered as one 1,098-token response |
| 13:51 | real traffic: 516-truncation caught, continuation finished immediately |

The headline number: rung-truncated responses reaching Codex

| segment | responses | rung finals |
|---|---|---|
| organic, before wiring (<09:55) | 37 | 4 (10.8%) — 08:56, 09:40, 09:45, 09:46, all 516 |
| organic, after wiring | 30+ | 0 (0.0%) |
| experiment wave (09:00–09:40, via proxy) | 106 | 0 |

That 10.8% morning rate is right in line with the historical 5.5 baseline, and it went to zero at the moment of wiring. (Note: the big block of 82 rung finals in UTC-day terms was yesterday evening's control-wave experiment, not today's organic use.)

Volume and tokens (local today)

20 sessions active, 173 rollout responses. Organic traffic: ~67 responses, 14.5k reasoning tokens, 32.4k output tokens, ~8.7M input of which ~91% cache hits. Fold overhead so far is negligible — the two real folds re-billed roughly one extra round of input each.

Personally, I do not think cost optimization or automatic routing is inherently the problem.

From OpenAI's perspective, it may not be sustainable to process every request through the most expensive reasoning path all the time. If GPT-5.5 is significantly more expensive than GPT-5.4, then routing simple questions or tasks that do not require deep reasoning to a cheaper model or lower-effort path can be a reasonable product decision.

However, I think the core issue here is not whether routing exists. The issue is transparency around what the user believes they selected versus what they actually received.

If a user selects gpt-5.5 with xhigh reasoning, or if the UI/CLI presents the request as being handled that way, the user will naturally expect that level of reasoning quality. But if the response repeatedly appears to stop around a specific reasoning-token boundary, such as 516, and that pattern is strongly correlated with wrong answers or shallow reasoning, users will inevitably start asking whether the selected model and effort level were really applied.

Smart routing for cost control can be reasonable. But in that case, the product should make the following clear:

• Which model and reasoning effort actually handled the request
• When and under what conditions the selected xhigh effort may be reduced
• What quota or cost tradeoff is required if the user wants to force deeper reasoning
• How the product communicates that a task was handled by GPT-5.4 because GPT-5.5 was not necessary

In other words, I understand that routing or reasoning-budget adjustments may be necessary as a tradeoff to avoid much higher prices. But if a user believes they are using 5.5 xhigh while the request is silently handled through a shallower reasoning path, that looks less like cost optimization and more like a trust problem.

The best direction, in my opinion, would be transparency rather than silent limitation: default to smart routing, but allow users to explicitly force full/highest reasoning when needed, with the quota or cost impact clearly shown.

All targeted tests run through codexcomp completed correctly without modifying model_instructions_file. When limiting the proxy to --max-n 1, all targeted runs also completed correctly. I could not validate whether the --max-n 1 proxy runs with the PLUS plan still preserved the 518*n-2 quantization pattern internally.

I've updated the test results here: https://github.com/openai/codex/issues/30364#issuecomment-4886606283

@dzshzx, after a 516-triggered continuation, a second round with reasoning_tokens=0 can be treated as clean, so the folded response still reaches Codex with total reasoning 516.

Observed with --max-n 1:

round 1: reason=516 -> continue
round 2: reason=0 -> clean
done: 2 round(s) | ... reason=516

That run still produced an incorrect final answer. Should reason=0 after a truncation-triggered continuation be handled as non-clean?

@josantonius Confirmed, and thanks for the precise repro — that was a real gap. Your read matches the code: a continuation round that answered with reasoning_tokens=0 was classified as clean, so the fold could still deliver a 516-total response.

codexcomp v0.3.5 treats that case as a stalled nudge: after a truncation-triggered continuation, a round that comes back with zero reasoning tokens is re-nudged instead of accepted, sharing the same --max-continue budget (default 3). If every retry stalls, the last round's answer is still delivered — never dropped — with proxy_stopped_reason: "max_continue" and the per-round breakdown under metadata.proxy_rounds, so the outcome stays observable.

Round-1 zero-reasoning responses are deliberately untouched: a standalone response with 0 reasoning tokens can be a legitimate complete answer (your PRO run 1 without ## Intermediary updates is an example of exactly that). The stall signal only means something after a confirmed 518n−2 truncation where the model was explicitly nudged to continue thinking.

pip install -U codexcomp (or uv tool upgrade codexcomp). If you get a chance to re-run the --max-n 1 scenario, I'd be interested in whether the retry recovers real reasoning or exhausts the budget — proxy_rounds will show it either way.

@dzshzx I haven’t run the candies test, but with normal usage the fix is already noticeable. Thanks for fixing it so quickly, great work!

v0.3.4
| Records | min | mean | max | >=516 | exact 516 | exact 1034 | exact 1552 | 516 / >=516 | 518n-2 / >=516 |
|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| 507 | 0 | 105.08 | 2584 | 24 | 8 | 0 | 1 | 33.33% | 9 / 24 (37.50%) |

v0.3.5
| Records | min | mean | max | >=516 | exact 516 | exact 1034 | exact 1552 | 516 / >=516 | 518n-2 / >=516 |
|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| 265 | 0 | 93.92 | 1367 | 17 | 0 | 0 | 0 | 0.00% | 0 / 17 (0.00%) |

Even with --max-n 1, the improvement is already clear: better consistency and fewer odd failures. I’ll keep testing without limiting it to n1 and see how it goes.

Same issue, this is unacceptable.
5.5 was insufferable the last weeks and now I know why.

Same issue, this is unacceptable. 5.5 was insufferable the last weeks and now I know why.

not sure if "last few weeks" would fit this issue

to add some more context
verified same pattern on 2 machines with Team plan (mainly using extra usage)

Quite the same problem.
Hope to fix this quickly.

Controlled 5-shot reproduction + effort-level controls — codex-cli 0.143.0 (brew, current), macOS, ChatGPT account, gpt-5.5.

Probe: a closed-form adversarial-drawing puzzle (correct answer 21, verifiable by hand), run via:

codex exec --json --skip-git-repo-check --ephemeral -s read-only --disable memories \
  -m gpt-5.5 -c model_reasoning_effort=high

Initial 5-shot, identical settings (2026-07-09 14:09–14:11 UTC):

| run | reasoning_output_tokens | output_tokens | final answer |
|---|---|---|---|
| 1 | 516 | 750 | 21 ✓ |
| 2 | 516 | 856 | 29 ✗ |
| 3 | 516 | 749 | 23 ✗ |
| 4 | 516 | 844 | 20 ✗ |
| 5 | 516 | 1,099 | 26 ✗ |

All five pinned at exactly 516 despite varying output lengths (749–1,099); 4/5 wrong. The wrong-answer transcripts visibly run out of thinking mid-argument (run 5 contradicts itself twice before guessing).

Controls ~10 minutes later (14:19–14:21 UTC), same machine/account/CLI:

| run | effort | reasoning_output_tokens | answer |
|---|---|---|---|
| low ×2 | low | 516, 516 | 21 ✓ / 28 ✗ |
| medium ×2 | medium | 516, 516 | 18 ✗ / 21 ✓ |
| high (repeat) | high | 6,102 | 21 ✓ |

Two observations that may help triage:

1. Intermittent at request level, and not client-side config. Five consecutive high-effort requests pinned at 516; a sixth identical request ten minutes later got 6,102 reasoning tokens and the correct answer. Same CLI build, flags, and account throughout — local configuration cannot produce time-varying behavior, which points at serving-side routing/rollout.
2. When affected, the effort knob is a no-op: low, medium, and high all returned exactly 516 with zero variance inside the affected window. The healthy high run confirms the knob works when a request lands on a healthy path.

Also tried isolating by model: gpt-5 and gpt-5.5-codex are rejected on ChatGPT accounts (400), so no cross-model control was possible from this plan.

Thread ids from the affected window, in case they help server-side lookup: 019f4736-7ef0-7731-8887-96582769d8ee, 019f4736-bc20-73e1-8a1a-5ada9d3172f8, 019f4737-02f7-7f32-a4c4-df0c092de935, 019f4737-4e4c-7101-9a0d-4d018cc44a93, 019f4737-94cc-7130-bc55-012bd5c93a40; healthy control: 019f473f-2fa1-7762-b6ea-f9bae4a31269. Full --json event streams available on request.

my view of this, that it actually a good thing. As serving llm inference is one of the hardest thing in computing (that actually comes after serving the internet at the 1990s -1995 to 2005, which was a harder task, as it was the first stage where the people making the magic happen - had to lay the grounds and specification as far as the science of things advanced in the last 180 years)

point is the service SLA of openai is very high, and the service quality has being noticeable changed - and its a very impressive thing - despite without the magic happening from the various chemical and semiconductor industry, and their bets and their investments (two different things)

I personally think, for the people who paid for the 20$ or 100$ (I am sure even the 8$ plan is also very fairly respected in terms of usage amount -- I assume from a Data Scientist point of view, you should actually aim to collect more from those users as they are casual - and could be maybe a student which I think is a good thing that the casual user - has a statement in the quality of data - even after the first baby steps where the model keep repeating itself - and doesnt understand very basic english) , I think codex does aim for very high amount of usage Plus, Pro users are able to get (subtracting if there is a very usage bug that causes over usage), but look at this, this way, they never took more than 40%

as per the issue I think most users would report the model is still very capable, I actually predict things going the right directs even in the next few days as of posting this comment. and we dont actually know if Sol , Terra and Luna , are bigger models or the same size. Today it should be launch, those models... as per 9-july-2026 a lot of changes regarding telemtry, api, some technical bugs, so things are going very well despite some truth the proclaimed issue (the counter argument is, that its with in reason, and you still going to get good usage out of it unless you encounter weird glitch that consume too much - which the user shouldnt be blamed if he was to encounter it)

a lot things related - for token rationing. non of this would have made possible if historically 2 years - chatgpt faced one of the hardest issues of being able to provide chats. as per nature, by nature we learn subscription based things are very hard, most people buy a new iphone every 1-2 years - and the margins are not really that big (despite phone stores buying a lot of stock) it being a selling stocks. I personally think its better not to be the only company and being called big tobacco, I also think we are going to see a good changes 50% improvement in the ways we generate electricity (despite Nuclear, Electrical, and chemistry are all indurstries that deeply influence each other)

This issue has really ruined my experience; I hope it gets fixed soon.

~gpt-5.6-terra low is not affected, tested with candies prompt~

gpt-5.6-terra low is not affected, tested with candies prompt

You on a PRO account?

@smileBeda im currently using free tier
5.5 is affected there

@apyrr and how is this related to the issue even remotely?
The issue is affecting PRO users, not PLUS, not Free, not API users.

@smileBeda I tested PRO, PLUS and free - all affected when using 5.5

Then we have more issues than just reasoning being capped.
Because I did _extensive_ tests on a PLUS account and it is 100% not affected.
This means (if your findings are accurate), the issue appears based on specific accounts - possibly flags or similar classification.
I can share videos of PLUS going though properly while PRO fails. Repeatedly and replicably.

Your report also contradicts the earlier findings of _all_ models affected, now suddenly supposedly it is only 5.5?
I have my doubts but will of course assume there is more to it than we know in this case.

Quick and dirty testing results:

gpt-5.6-terra xhigh is affected. I'm seeing some 516 thinking token runs with the wrong answer, also strangely I saw 316 and 353 thinking tokens with the wrong answer as well, which I never saw with 5.5.

Edit 1:
gpt-5.6-sol xhigh same behavior. I just saw worse runs with only 280 and 279 thinking tokens and the wrong answer.
This was on a fresh codex install with no config changes or proxy. This sucks.

Edit 2:
Removing intermediary commentary might help, but maybe not compared to how much it helped with 5.5.
I don't see many 516 thinking tokens, but still too many wrong answers.
Also the system prompt changed for 5.6, so that should be taken into account.

pro reasoning mode:
https://developers.openai.com/api/docs/guides/reasoning#reasoning-mode

I submitted/collected additional local Codex session evidence in session:

019f3053-7f43-76b0-af62-4e7863889c05

This session includes fresh 2026-07-09 canary tests for gpt-5.6-sol, gpt-5.6-terra, and gpt-5.6-luna. Results: Sol high was 5/5 correct with no 516 cluster; Terra high had a wrong 516-token run; Luna high had two wrong 516-token runs.

Following up on https://github.com/openai/codex/issues/30364#issuecomment-4908844165

I think I no longer need a proxy for gpt-5.6-sol. Previously, all 5 attempts on the candy eval with 5.5 failed (all blocked at 516 reasoning tokens). 5.6 Sol (high) completes every attempt with 1,500-2,000 reasoning tokens allocated.

gpt-5.6-terra and gpt-5.6-luna are dumb as bricks, though. Terra (high) used 200-300 reasoning tokens and failed every attempt. Luna (high) was typically capped at around 500 (not exactly 516), with the same result.

Try more than 5 runs with gpt-5.6-sol xhigh. I had 5/5 successes, until I ran another 10 just to be sure, and had one failure with 388 thinking tokens with the wrong answer. Smells like an adaptive thinking related issue, but it's rare.

Also I'm getting less of these issues than before, gpt-5.6-sol xhigh works much better on the candy eval now compared to 6 hours ago.

@NarinderS Maybe I'm lucky, but no failures after 10 attempts on xhigh. I think I'll focus on work now before they "optimize" this model as well. 😅

FWIW, tried it with the full suite of 5.6 agents:

Evaluation criterion

• Correct final answer: 21
• A configuration is considered affected if any response:

1. gives the wrong final answer, and
2. reports exactly 516, 1034, or 1552 reasoning output tokens.

Each shot below is shown as final answer / reasoning_output_tokens.

Results

   Model            Effort    Five shots                                       Wrong    Result
  ━━━━━━━━━━━━━━━  ━━━━━━━━  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  ━━━━━━━  ━━━━━━━━━━
   gpt-5.6-sol      high      21/3278, 21/2773, 21/2961, 21/4656, 21/2511        0/5    Fine
  ───────────────  ────────  ───────────────────────────────────────────────  ───────  ──────────
   gpt-5.6-sol      xhigh     21/3509, 21/2588, 21/4139, 21/4958, 21/2886        0/5    Fine
  ───────────────  ────────  ───────────────────────────────────────────────  ───────  ──────────
   gpt-5.6-terra    high      21/2757, 21/2277, 21/3610, 21/3547, 21/3106        0/5    Fine
  ───────────────  ────────  ───────────────────────────────────────────────  ───────  ──────────
   gpt-5.6-terra    xhigh     21/6537, 21/5500, 21/6872, 21/9840, 21/5993        0/5    Fine
  ───────────────  ────────  ───────────────────────────────────────────────  ───────  ──────────
   gpt-5.5          high      15/516, 25/516, 29/516, 21/516, 16/516             4/5    Affected
  ───────────────  ────────  ───────────────────────────────────────────────  ───────  ──────────
   gpt-5.5          xhigh     19/516, 20/516, 21/516, 15/516, 29/516             4/5    Affected
  ───────────────  ────────  ───────────────────────────────────────────────  ───────  ──────────
   gpt-5.6-luna     high      29/516, 21/2822, 21/3017, 29/1034, 29/516          3/5    Affected
  ───────────────  ────────  ───────────────────────────────────────────────  ───────  ──────────
   gpt-5.6-luna     xhigh     21/4660, 21/12622, 21/5696, 21/6984, 21/6166       0/5    Fine
  ───────────────  ────────  ───────────────────────────────────────────────  ───────  ──────────
   gpt-5.4          high      21/3534, 21/7989, 21/4660, 27/516, 23/516          2/5    Affected
  ───────────────  ────────  ───────────────────────────────────────────────  ───────  ──────────
   gpt-5.4          xhigh     21/6630, 21/10847, 29/516, 21/10001, 21/10101      1/5    Affected

We had this improved by a lot by changing the model instructions, but with the new model it's even worse, we are getting 516 and incorrect answers 5 out 5 times. Testing with 5.6 Sol Max/Ultra. We really would like an explanation, is this an intentional degrade for power users? Otherwise, all those published benchmarks are basically fake.

I cannot confirm this, tested on 5.6-sol-max
5 out of 5 correct and no low reasoning shortcuts.
Others above also confirmed this to be solved.

---

Needless to say that the story here should not be over. 5.5 is still very much alive and kicking and clearly capped. OAI has to take a stance on this and explain what is happening, instead, we are faced with corporate silence. This is not exactly trust-inducing since the very same could happen any moment with new models, without anyone even realizing due to the otherwise sheer quality of these models. Still it is what we buy, and kind of expect to receive: a high reasoning model.

The issue persists.

I tested GPT-5.6 Sol across 125 runs. The results are shown below. For the detailed test parameters and analysis, see the attachments.

| Reasoning effort | Runs | Correct | Accuracy | Continuations | Continuation rate | Avg. latency (s) | Avg. reasoning tokens |
| ---------------- | ---: | ------: | -------: | ------------: | ----------------: | ---------------: | --------------------: |
| Light | 25 | 25 | 100.00% | 4 | 16.00% | 45.980 | 1027.40 |
| Medium | 25 | 24 | 96.00% | 6 | 24.00% | 54.167 | 1409.76 |
| High | 25 | 24 | 96.00% | 6 | 24.00% | 76.543 | 2154.44 |
| Extra High | 25 | 25 | 100.00% | 13 | 52.00% | 125.676 | 3805.00 |
| Max | 25 | 25 | 100.00% | 14 | 56.00% | 171.892 | 5365.27 |

I used CodexComp to continue truncated responses. Of the 125 test runs, 43 required continuation. Of these, 36 ran for two rounds and 7 ran for three rounds, producing 50 additional continuation rounds in total.

The table below counts hits on fixed reasoning-token truncation breakpoints by model round, not by response or additional round. A single response can hit a breakpoint in multiple rounds, so the total number of breakpoint-hit rounds is 67.

reasoning tokens = 518n − 2

| Reasoning tokens at truncation | n | Rounds hitting breakpoint |
| -----------------------------: | --: | ------------------------: |
| 516 | 1 | 14 |
| 1034 | 2 | 14 |
| 1552 | 3 | 4 |
| 2070 | 4 | 7 |
| 2588 | 5 | 6 |
| 3106 | 6 | 4 |
| 3624 | 7 | 4 |
| 4142 | 8 | 1 |
| 4660 | 9 | 2 |
| 5178 | 10 | 1 |
| 5696 | 11 | 6 |
| 6214 | 12 | 1 |
| 6732 | 13 | 2 |
| 7768 | 15 | 1 |

GPT-5.6 Sol Candy Test Analysis Report.md
GPT-5.6 Sol Candy Test Results.md

> We had this improved by a lot by changing the model instructions, but with the new model it's even worse, we are getting 516 and incorrect answers 5 out 5 times. Testing with 5.6 Sol Max/Ultra. We really would like an explanation, is this an intentional degrade for power users? Otherwise, all those published benchmarks are basically fake. I cannot confirm this, tested on 5.6-sol-max 5 out of 5 correct and no low reasoning shortcuts. Others above also confirmed this to be solved. Needless to say that the story here should not be over. 5.5 is still very much alive and kicking and clearly capped. OAI has to take a stance on this and explain what is happening, instead, we are faced with corporate silence. This is not exactly trust-inducing since the very same could happen any moment with new models, without anyone even realizing due to the otherwise sheer quality of these models. Still it is what we buy, and kind of expect to receive: a high reasoning model.

Its not all you can eat Baff'e

I recomend you try and stick to GPT Terra - medium

for most task you will also enjoy the better speed