GPT-5.6 often serializes independent Code Mode calls; explicit batching reduced weighted usage by 27–45%

Open 💬 25 comments Opened Jul 24, 2026 by MakerOfToys

What version of the Codex App are you using (From “About Codex” dialog)?

26.721.30844

What subscription do you have?

ChatGPT Pro for the personal-project tests; ChatGPT Business for the work-project tests.

What platform is your computer?

Microsoft Windows NT 10.0.26200.0 x64

What issue are you seeing?

Related to #32503, but this report adds controlled same-model benchmarks, credit-weighted usage measurements, quality comparisons, and a tested workaround across two unrelated codebases.

I began investigating this independently after noticing an unusually high number of recent complaints about Codex usage depletion. I had not personally noticed an obvious long-term reduction in my limits, so I inspected my own .codex session logs rather than assume that allowances had been silently changed.

Only after reaching the batching hypothesis and completing my initial tests did I search for similar reports and find #32503. Its author had already identified essentially the same Code Mode parallelism problem from a long observational trace, including only 5 of 739 GPT-5.6 exec cells using Promise.all. The tests below appear to provide the controlled validation that report said was still missing.

I tested two unrelated repositories:

one large proprietary work codebase;
one smaller personal finance application.

Each repository had one fixed read-only investigation task. Within each comparison, the subscription plan, repository state, task prompt, model, reasoning effort and permissions were unchanged. Only the presence of the batching instruction differed. No code was modified during the investigations.

Main results

| Dataset | Sample | Model cycles | Raw tokens | Weighted usage | Same-credit capacity |
|---|---:|---:|---:|---:|---:|
| Large work codebase | 4 batched runs vs 3 controls, High | 52.1% fewer | 63.1% lower | 45.0% lower | 81.8% more |
| Personal finance app | 4 matched pairs, High/XHigh | 54.7% fewer | 52.6% lower | 27.0% lower | 37.0% more |
| Personal finance app, Max | 1 matched pair, adjusted | 80.4% fewer | 79.5% lower | 47.4% lower | 90.2% more |

The Max result is a single pair and needs replication. It still covered all required areas and aligned on the core conclusions, but the batched report referenced fewer unique files than its control, so I treat both its efficiency and quality result cautiously. The repeated High/XHigh results are the stronger basis for the current conclusion.

The exact improvement varied substantially by workload. I am not claiming that every Codex task will gain 37–82% more capacity.

“Weighted usage” is the Codex-credit equivalent calculated from the returned uncached-input, cached-input and output-token categories. Reasoning tokens are included in output tokens.

Most of the raw-token reduction came from repeated context processing:

cached input fell by 56–65% in the repeated datasets;
uncached input fell by 5–26%;
output changed by only about 0–12%.

This is consistent with batching reducing outer model round trips rather than simply reducing the amount of investigation performed.

Normal-usage observation

At work I deleted my .codex directory and allowed it to rebuild from scratch.

By July 23, the fresh logs had accumulated roughly 1,600 calls that a post-hoc Codex analysis classified as independent and batchable. Only a little over 100 had actually been grouped by the agent.

This was not part of the controlled benchmark, but it suggests that GPT-5.6 sometimes discovers batching on its own while doing so very inconsistently without concrete instructions.

Quality

I did not find an apparent systematic quality reduction.

Across the Finance comparisons:

both conditions covered every required investigation area;
the main conclusions aligned;
the aggregate performed 106 batched inspections versus 105 control inspections.

On the large codebase, the batched runs performed substantially more shell operations than the controls while still using much less.

Batched runs sometimes found additional useful details. Subtle quality differences may still exist, and implementation-heavy or mutation-heavy tasks may behave differently from these read-only investigations.

---

What steps can reproduce the bug?

Choose a read-heavy repository task involving several independent searches, file reads or metadata inspections.

Freeze the repository state.

Start fresh GPT-5.6 Sol sessions using the same:

task prompt;
reasoning effort;
permissions;
repository state.

Run one condition without custom batching guidance.

Run the other with this exact instruction in either repository AGENTS.md or the Codex App custom instructions:

In Code Mode, within each bounded stage, run independent, functions.exec-available tool calls concurrently in one functions.exec call. Use await Promise.allSettled([...]) when partial results are useful, and inspect every result; use await Promise.all([...]) only when any failure should abort the batch. Keep dependencies, waits/resumes, approvals, conflicting or interdependent mutations, and adaptive investigations where each result may change the next step sequential. Do not split otherwise batchable inspections across outer tool calls.

Compare the session logs:

outer model/tool cycles;
direct tool calls and exec cells;
nested calls per exec cell;
use of Promise.all or Promise.allSettled;
cached, uncached and output tokens;
task coverage and final conclusions.

The controlled repository comparisons used AGENTS.md. I also used the same instruction through app-level custom instructions and observed the same multi-call Code Mode batching pattern.

Despite using older functions.exec terminology, it reliably caused GPT-5.6 Sol to group multiple nested operations into concurrent Code Mode stages.

---

What is the expected behavior?

GPT-5.6 Code Mode should receive concrete guidance to:

group independent, non-conflicting calls into one bounded exec stage;
use Promise.all or Promise.allSettled for nested concurrency;
keep dependent, adaptive, approval-sensitive and conflicting operations sequential;
avoid repeated one-call exec cells when multiple independent calls are already known;
avoid expanding investigation scope only because more calls can be launched concurrently.

The model-facing exec description should include an example containing multiple nested tools.* calls inside Promise.all(...) or Promise.allSettled(...).

It would also help if telemetry distinguished outer model cycles from nested tool calls and showed quota-weighted usage more directly.

---

Additional information

GPT-5.2 through GPT-5.5 were explicitly instructed to parallelize common independent reads using multi_tool_use.parallel.

GPT-5.6 moved to the Responses Lite and Code Mode path. Native top-level parallel calls are disabled there, while JavaScript inside exec is the practical replacement. The runtime supports nested concurrency, but the shipped GPT-5.6 guidance is generic and the exec example demonstrates only a single nested call.

I may be missing an internal constraint or implementation detail, but the public source, local traces, controlled behavior and the independent findings in #32503 are consistent with an instruction/tool-interface regression during that transition.

I do not see evidence that this was an intentional allowance reduction. OpenAI added a replacement batching mechanism; the model appears not to use it reliably without concrete guidance.

I also tested several rewritten instructions using the current exec and tools.* terminology. One essentially matched the original instruction on the large codebase, but none consistently matched its Finance results. Some expanded investigation scope, while others produced one-call Promise.allSettled cells that provided little real batching. One overly aggressive variant consumed 26.8% more weighted credits than its control.

Limitations

These tests covered read-heavy repository investigations.

The large-codebase result used grouped averages rather than matched simultaneous pairs. The Max result and each revised instruction variant currently have only one pair. Model sampling, service load and differences in peripheral investigation scope remain sources of variance.

“Same-credit capacity” means more comparable completed work from the same allowance. It does not mean the allowance necessarily lasts the same additional percentage in literal wall-clock time, because the batched tasks also tend to finish faster.

Additional sanitized per-run measurements are available on request.

View original on GitHub ↗

25 Comments

MakerOfToys · 1 month ago

Update: output-truncation caveat

A raw-log audit found that some batched exec responses exceeded Code Mode’s shared output limit and were truncated. This may have overstated the measured usage savings and introduces a quality risk, although the reduction in model/tool cycles remains clear and the repeated comparisons still showed aligned required coverage and core conclusions.

I am retesting with a minimally revised instruction that bounds combined output and narrowly retrieves any missing evidence. Until then, please treat the 27–45% savings range as provisional.

ForrestGrump · 1 month ago

Independent replication on a different setup (Codex CLI 0.145.0, WSL2,
ChatGPT Pro auth, gpt-5.6-sol), plus two findings I haven't seen in this
thread or #32503: the effect is effort-gated, and ultra effort is a
separate order of magnitude.

Setup

Controlled A/B, only variable = a batching instruction (the
Promise.allSettled paragraph from this issue) vs a length-matched placebo
instruction, injected via AGENTS.md (synthetic repos) or
-c developer_instructions=... (real repos, read-only sandbox). Fresh
session per trial, alternating arm order, MCP/plugins/memories disabled,
fixed model + effort per run. Metrics parsed from the
~/.codex/sessions rollout JSONL (token_count events = model cycles;
custom_tool_call name="exec" cells inspected for tools.*( counts and
Promise.all/allSettled). Harness: https://github.com/ForrestGrump/codex-batch-bench.

Results — synthetic 48-module repo, effort sweep (no-instruction ÷ instructed)

| effort | model cycles | raw tokens | weighted credits* |
|---|---|---|---|
| low | 1.00× | 1.11× | 0.98× |
| medium | 1.14× | 1.01× | 0.90× |
| high | 1.06× | 0.96× | 0.89× |
| xhigh | 1.50× | 1.56× | 1.16× |
| max | 1.33× | 1.36× | 0.99× |
| ultra | 4.83× | 4.44× | 3.65× |

\* at the published 125/12.5/750 per-1M rate card.

At low–high, the uninstructed model batches on its own (often via shell
loops in plain exec_command calls — zero Code Mode cells). At xhigh/max it
degenerates to one tool call per model cycle. A fix that only tests at
high effort will not see this bug.

Results — three real repos, xhigh, 2 trials/arm each

Raw-token multipliers (uninstructed ÷ instructed): 1.41× / 3.15× / 2.66×
(mean ≈ 2.4×, consistent with the 2.1–2.7× in the OP). Worst trial: 25
cycles / 1.71M raw tokens uninstructed vs 7 cycles / 400k instructed for the
identical task. Deliverable quality was equivalent in all trials, both arms
(graded on the synthetic repos: 48/48 modules + 40/40 config keys at every
effort level including ultra).

Ultra is its own problem

At ultra, the uninstructed runs spawned 23–25 sub-sessions despite
features.multi_agent=false, used Promise.all* zero times across ~50
exec cells, and consumed 7.5–17.6M raw tokens (267–605 credits) per
read-only investigation task — the worst single trial moved a Pro weekly
meter a full 1%. Filed separately: #35177.

Also verified: sub-agents inherit the parent's resolved config (including
-c developer_instructions overrides); they do not re-read
~/.codex/config.toml.

Repro

git clone https://github.com/ForrestGrump/codex-batch-bench && cd codex-batch-bench
python3 bench.py setup
python3 bench.py run --preset large --effort xhigh   # ~15 min, ~1.1M raw tokens

Happy to provide raw rollout logs / metrics CSVs for any trial.

MakerOfToys · 1 month ago

Further truncation-audit context

A more detailed comparison shows that roughly 80k of the batched-side omitted tokens came from two ALL_TOOLS catalogue dumps rather than repository evidence. Non-batched controls also truncated broad compound shell outputs.

Excluding those catalogue dumps and the single Max pair, the four repeated High/XHigh comparisons omitted 63.6k repository-output tokens with batching versus 20.9k in controls. Restoring all explicitly omitted repository output once changes the repeated weighted-saving estimate from 27.0% to approximately 23.9%; the large reduction in model cycles and cached-input usage remains.

Truncation is therefore still a real output-reliability issue, but the detailed logs do not suggest that it explains most of the measured efficiency gain or caused an apparent systematic collapse in answer quality. A cleaner output-bounded replication is still needed.

ForrestGrump · 1 month ago

@MakerOfToys I ran your truncation audit against my 52-trial replication (methodology upthread; harness: https://github.com/ForrestGrump/codex-batch-bench, audit script now in the repo as audit_ingestion.py). Metric: characters of all tool output entering each trial's transcript (function_call_output + custom_tool_call_output payloads in the rollout JSONL), split unique vs exact-duplicate.

| run | no-instruction | batching | ingestion ratio | raw-token ratio |
|---|---|---|---|---|
| synthetic 48-module @ xhigh | 55.8k | 55.9k | 1.00× | 1.56× |
| real repo A @ xhigh | 92.2k | 70.4k | 1.31× | 1.41× |
| real repo B @ xhigh | 245.9k | 169.7k | 1.45× | 3.15× |
| real repo C @ xhigh | 109.8k | 70.7k | 1.55× | 2.66× |
| synthetic @ ultra | 849k | 420k | 2.02× | 4.44× |

Three observations relevant to "a cleaner output-bounded replication is still needed":

The synthetic-xhigh pair is that replication. Fixed 48-file corpus, generated ground truth (answers graded 48/48 modules + 40/40 config keys in both arms), and ingestion parity to within 0.2% — yet a 1.56× raw-token and 1.50× cycle gap. On an evidence-bounded task the entire gap is round-trip overhead; zero truncation component.

On real repos the batched arm did ingest 24–36% less tool output, consistent with your concern — but the consumption gaps (1.41–3.15×) and cycle gaps (1.50–2.79×) substantially exceed the ingestion gaps, and the dominant cost term (cached-input re-reads scaling with cycle count) is independent of output volume. Deliverables were equivalent in every pair.

At ultra, the uninstructed arm was the over-ingester: the swarm read the ~250KB corpus ≈3.4× over (vs ≈1.7× batched) across 23–25 sub-sessions (#35177) — redundant re-reading, not deeper evidence; both arms graded perfect.

Caveat matching yours: exact-duplicate share was only 0–2% by hash, but overlapping re-reads via different commands/line ranges hash as unique, so byte-level dedup understates redundancy. Net: truncation is a real output-reliability concern on open-ended tasks, but on this dataset it cannot account for the majority of the measured gap, and on the bounded task it accounts for none of it.

MakerOfToys · 1 month ago

@ForrestGrump Thank you for doing all this work and for continuing to test the different explanations. The ingestion audit is especially useful - it’s reassuring to see a clean case where both arms received essentially the same amount of tool output while the batching run still used substantially fewer tokens. Really appreciate you publishing the harness and results too.

avoethan · 1 month ago

Edited after a source and benchmark audit (July 26, 2026).

I still think a description-only PR is the right first experiment, but I would narrow the causal claim and make the output-limit guidance more precise.

Why this scope still fits the current source

gpt-5.6-sol uses Responses Lite and code_mode_only.
Responses Lite disables top-level parallel_tool_calls, so JavaScript concurrency inside Code Mode exec is the practical batching path.
The current model-facing exec description shows a single awaited nested call and does not explain Promise.all(...) or Promise.allSettled(...).
The runtime test suite demonstrates that two nested tool calls can overlap. That proves the mechanism exists, but not that every tool, mutation, approval path, or failure mode is universally safe to parallelize.

Proposed PR scope

Change only the model-facing Code Mode exec description and focused tests. Do not change request serialization, parallel_tool_calls, model metadata, runtime scheduling, telemetry, or approval behavior in this first patch.

Suggested guidance:

When multiple nested tool calls for the current stage are already known, mutually independent, and safe to run without ordering, approval, or shared-state conflicts, run a small bounded group concurrently in one exec call. Prefer Promise.allSettled(...) when partial results are useful, inspect every settled result, and explicitly handle rejections or truncated results. Use Promise.all(...) only when any failure should abort the whole batch. Keep adaptive, approval-sensitive, waiting, potentially conflicting, or state-changing work sequential unless the tool contract explicitly guarantees parallel safety. Do not broaden the task merely because concurrency is available. The outer // @exec max_output_tokens value is a shared budget for the combined exec result. Keep batches bounded, request narrow fields or ranges, use nested tool-specific output limits where supported, and set the outer budget deliberately. If evidence is truncated, recover only the missing evidence with a narrow follow-up read.

A compact example could be:

// @exec: {"max_output_tokens": 3000}
const tasks = [
  ["status", tools.exec_command({ cmd: "git status --short", max_output_tokens: 800 })],
  ["diff", tools.exec_command({ cmd: "git diff --stat", max_output_tokens: 1200 })],
];

const settled = await Promise.allSettled(tasks.map(([, promise]) => promise));
const checked = settled.map(([name], index) => {
  const result = settled[index];
  return result.status === "fulfilled"
    ? { name, ok: true, value: result.value }
    : { name, ok: false, error: String(result.reason) };
});

text(JSON.stringify(checked));

The exact example can be adjusted to match the repository's preferred tool and platform-neutral style.

Tests I would add

Direct assertions on the rendered exec description for:

Promise.allSettled guidance;
inspection of every result;
sequencing exclusions;
shared outer-output-budget wording.

A runtime test with one successful and one failed nested call under Promise.allSettled, confirming that the successful result remains available and the failure is explicit.
A combined-output-budget test confirming that concurrent nested results still obey the single outer exec budget.
Existing Responses Lite request snapshots should remain unchanged.

Validation before calling it fixed

Compare stock and patched binaries with the public harness using:

stock placebo vs patched placebo as the primary comparison;
explicit batching instructions as a positive-control ceiling;
XHigh as the primary effort tier, Max as secondary;
Low–High as non-regression tiers;
Ultra reported separately because its automatic sub-session behavior is tracked in #35177.

I would look for a material reduction in XHigh outer model cycles and singleton exec stages, no more than a 5% regression at Low–High, no increase in runtime failures or unrecovered truncation, and exact task grading rather than only checking module-name and unused-key presence.

This remains a bounded, testable first fix rather than a claim that description text is the sole root cause. I am happy to prepare the focused patch and validation if a Codex-team member agrees with the approach and explicitly invites a PR under the repository's contribution policy.

ForrestGrump · 1 month ago

@avoethan Strong +1 on this scope from the replication side — the data says a description-only fix is aimed at the right layer:

The runtime path already works when the model knows about it. Across 30 instructed trials (six effort tiers + three real repos), the model used Promise.allSettled inside exec in every xhigh/max/ultra trial with zero runtime failures — the capability is fine; only the contract is silent about it. A prompt-contract fix matches the failure mode.
"Teach the capability and decision boundary rather than force maximum fan-out" is exactly what the effort curve demands. At high and below the model already groups reads on its own (often via shell compounds), and my always-batch instruction was mildly negative there (0.89× weighted credits vs placebo). Unconditional "batch everything" wording risks regressing the healthy tiers; conditional wording ("already-known, independent, non-mutating, one bounded stage") produced the wins without measurable quality cost (graded 48/48 + 40/40 in both arms at every tier).
Your item 3 (output-safety guidance) is empirically motivated too: in my real-repo runs the batched arm ingested 24–36% less tool output — "inspect every settled result, request narrow ranges, set appropriate max_output_tokens" is the right mitigation for exactly the truncation concern discussed above.

To help review such a PR: the harness now supports --codex-bin, so a patched build can be benchmarked against stock with identical prompts — compare the placebo arms across the two runs (no explicit instruction anywhere; any delta isolates the built-in description change), with the batching arm as a positive-control ceiling. Recipe in the README: https://github.com/ForrestGrump/codex-batch-bench#validating-a-tool-description-pr-binary-vs-binary

One review-criterion suggestion: validate at xhigh/max (and ideally ultra, where the orchestrator + sub-agents inherit the same description — see #35177), and check high-and-below for non-regression. A description change validated only at default efforts will not exercise the failure mode this issue is about.

Happy to run the pre/post sweep on a candidate branch and post numbers.

tylorsaling-source · 1 month ago

Relevant to the truncation caveat this thread has been weighing against the batching results: I traced the large truncations to source and filed #35421.

They are not a batching artifact. In my sample, across 4,841 legacy shell_command / function_call_output records, 9 hit the 1 MiB capture cap — and exec truncations were almost entirely trivial (96–100 characters). Roughly 96% of all dropped content came from the non-batched shell path.

The mechanism is that the shell capture retains the first 1 MiB and discards the remainder without counting it, then a separate formatter elides the survivor to the model budget and inserts …252144 tokens truncated… mid-payload. That figure is a constant — (1,048,576 − 40,000) / 4 — so it is the same regardless of how much was actually lost.

Practical implication for the measurements here: truncation of this kind affects both arms of the A/B and is not introduced by batching, so it should not be netted against the batched arm'''''s savings. @ForrestGrump this may also bear on the ingestion-ratio comparison — some of the difference could be capped output rather than reduced duplication.

ForrestGrump · 1 month ago

@tylorsaling-source Good catch — I re-audited all 52 trials with a full-text marker scan (my earlier scan only sampled output heads), and your hypothesis is partially confirmed on my dataset, in an instructive way:

No 1 MiB cap events fired here. Zero markers with the ≈252k constant figure; largest single output was 156 KB. All outputs were clamped by model-chosen max_output_tokens (2k–20k) well before your capture cap — so #35421's silent-discard path doesn't contaminate this dataset, though it clearly could at larger scales.

**But budget-level elision was asymmetric — and concentrated in the batched arm. 33 exec-path markers (median 1,235, max 48,634 claimed tokens): multi-call Promise.allSettled results overflowing the per-call budget. Placebo's shell-path markers were trivial (n=26, median 5** tokens claimed). Crediting the claimed-elided volume back as "attempted ingestion":

| real repo @ xhigh | seen ratio (no-instr ÷ batched) | attempted ratio |
|---|---|---|
| A | 1.31× | ≈1.09× |
| B | 1.45× | ≈0.90× |
| C | 1.55× | ≈1.36× |

So yes — much of the seen-ingestion difference I reported was capped output, not reduced duplication. Thanks for pushing on this.

Net effect on the thread's question, though, is to strengthen the no-confound conclusion: the batched arm fetched roughly as much repository data as the non-batched arm; the formatter trimmed the presentation; deliverables stayed equivalent (and on the graded synthetic task, perfect in both arms at every effort tier); and the 1.41–3.15× consumption gaps remain — attributable to round trips, not evidence volume. The synthetic-xhigh pair keeps its status as the clean bounded replication: 1.00× seen ingestion, equal grades, 1.56× consumption.

(Caveat inherited from #35421: I treat the claimed figures as approximate. Also practical support for the proposed exec-description scope above — batched multi-file results routinely exceeded model-chosen budgets, which is exactly why the "narrow fields/ranges, deliberate max_output_tokens" guidance belongs in the description.)

Updated audit in the repo: https://github.com/ForrestGrump/codex-batch-bench

haseeb-heaven · 1 month ago

can you please fix this issue? I only have like 32% left usage on my weekly. I just gave it a task and it completed the task and just one over and a half. It made the plan and it executed that so I was using GPT 5.6 Terra Medium for the implementation. And it did not complete the whole task for that plan as well. Still, my usage was really exhausted.

coryparrry · 1 month ago

Independent Codex Desktop replication at medium effort

I ran a small controlled A/B and saw the same direction of effect at medium effort.

Setup

Codex Desktop on macOS/Apple Silicon
Model/effort: gpt-5.6-sol, medium
Same frozen repository commit in two fresh Codex worktrees
Identical read-only prompt requesting 12 independent repository-health checks
No builds, tests, edits, web access, or repository mutations
Intended variable: explicit bounded functions.exec / Promise.allSettled batching guidance present vs absent

Results

| Metric | Batched | No batching guidance | Difference |
|---|---:|---:|---:|
| Outer tool calls | 1 | 2 | 50% fewer |
| token_count events | 2 | 3 | 33.3% fewer |
| Cached input | 32,835 | 78,726 | 58.3% lower |
| Uncached input | 45,142 | 48,102 | 6.2% lower |
| Output | 3,419 | 5,779 | 40.8% lower |
| Raw total tokens | 81,396 | 132,607 | 38.6% lower |
| Uncached input + output | 48,561 | 53,881 | 9.9% lower |
| End-to-end duration | 41.848 s | 71.805 s | 41.7% faster |

The batched arm used one functions.exec cell containing 13 concurrent exec_command calls through Promise.allSettled. The uninstructed arm used one large sequential compound exec_command, then a second adaptive exec_command to narrow and correct the result.

Raw shell time was not the source of the wall-clock win: the batched outer execution reported about 380 ms, versus about 290 ms across the control's shell batches. The overall improvement came from avoiding the additional model/tool round trip and its repeated context processing, consistent with the large cached-input reduction.

Quality/output caveat

Both arms covered all 12 requested areas and reported no command failures. However, both encountered an overly broad documented-command search. The batched arm acknowledged truncation but did not recover the missing evidence; the control used its second cycle to narrow that search and also refined the interpretation of “source files” to exclude test trees.

I therefore treat the 38.6% raw-token and 41.7% duration reductions as a useful single-pair replication, not a general savings estimate or proof of equal output quality. It also supports the output-safety wording proposed above: bound combined output and narrowly recover truncated evidence rather than stopping after the initial concurrent stage.

tylorsaling-source · 1 month ago

For anyone arriving here because your usage is vanishing: batching is one drain, and this thread has it well characterized — but it isn't the only one, and for a lot of "a single task ate my week" reports the larger, separate cause is the context auto-compaction loop, not tool-call serialization.

Quick way to tell which you're hitting:

Batching (this thread): you see many sequential independent tool/exec calls that could have run together. The instruction in this issue helps.
Compaction loop: you see repeated Context automatically compacted messages, the model rereading the same files, restating it "found the defect," and never finishing — while usage drains. The batching instruction won't help that.

If it's the second, these track it directly: #35226, #32809 ("eats up my limit … compaction loops"), #31351 (~30% of usage consumed), #34095, #32888. It's a long-standing cluster.

(And for the truncation caveat this thread weighed against the batching results: I traced those large truncations to source and filed #35421 — they come almost entirely from the non-batched legacy shell path, not batching.)

PRMS-magnet · 1 month ago

🤖

I made a local proof-of-concept patch against Codex main commit 61a44880a85d.

The part relevant to this issue adds concrete model-facing guidance and an example for using Promise.all/Promise.allSettled when nested tool calls are already known, independent, and read-only. It explicitly keeps dependent, mutating, approval-sensitive, and adaptive work sequential.

The patch also enables buffered execution by default and lengthens several interruptible wait defaults. Those changes are broader and may deserve separate consideration; maintainers may only find the description change useful.

A few important caveats:

I used Codex to inspect the source, implement the patch, and run the tests. I do not know Rust and have not personally audited the implementation.
Testing was narrow: targeted changed-behavior tests and the affected component suites passed, and a Windows release build completed.
I performed one small smoke test after installing it, which confirmed that one Promise.allSettled exec cell could contain two nested shell calls.
I did not run a proper stock-versus-patched benchmark, cross-platform testing, or broad quality/non-regression testing.
The full codex-core suite was not green in my Windows environment because of missing test helpers, symlink permissions, and related environmental failures, so I am not claiming full-suite validation.
I am not claiming a measured savings number for this particular patch.

Please treat this as an exploratory patch rather than something ready to merge as-is. Feel free to take, rewrite, split, or discard any portion that is useful. I do not need attribution and will not open a PR unless invited.

Patch: https://gist.github.com/PRMS-magnet/5e856fb33f1d293a7d03d37fa3cc2816

tho-stack · 1 month ago

Follow-up: stronger `AGENTS.md` wording substantially changed normal-session batching

I ran a longitudinal follow-up in a separate, read-heavy scientific repository. The only model-facing intervention was the repository’s AGENTS.md wording. The optional hook was audit-only: it did not block, rewrite, warn, or feed results back to the model.

This complements the controlled benchmarks in this issue, but it is not another controlled A/B. Some sessions were resumed or forked while the wording was refined, so I treat the results as normal-usage evidence rather than a causal effect estimate.

Historical behavior

Before adding the stronger batching guidance, historical rollout logs were dominated by one-tool exec cells:

| Surface | Root sessions | exec cells | Single-tool cells | Promise cells | Median single-tool rate |
| --- | ---: | ---: | ---: | ---: | ---: |
| CLI | 6 | 299 | 210 (70.2%) | 83 (27.8%) | 86.1% |
| Desktop | 16 | 33,218 | 31,718 (95.5%) | 1,296 (3.9%) | 96.1% |

The Desktop history is the clearest signal: 95.5% of exec cells contained one tool, while only 3.9% contained an explicit Promise.

First instruction

The first version established the mechanism and safety exceptions:

In Code Mode, within each bounded stage, run independent,
functions.exec-available tool calls concurrently in one functions.exec call.
Use await Promise.allSettled([...]) when partial results are useful, and inspect
every result; use await Promise.all([...]) only when any failure should abort
the batch. Keep dependencies, waits/resumes, approvals, conflicting or
interdependent mutations, and adaptive investigations where each result may
change the next step sequential. Do not split otherwise batchable inspections
across outer tool calls.

This improved batching, especially during initial discovery, but singleton follow-up calls still reappeared during later reads, diffs, hashes, status checks, and verification.

Hardened instruction

The next revision made batching an explicit decision gate that must be re-applied throughout the turn:

<details>
<summary>Current hardened wording</summary>

In Code Mode, batching is a hard pre-call gate. Before issuing any tool call,
inventory the operations already known for the current bounded stage. If two
or more independent functions.exec-available operations exist, issue exactly
one functions.exec call and launch them together with
await Promise.allSettled([...]); inspect every settled result. Use
await Promise.all([...]) only when any failure must abort the whole batch.

Do not issue a direct exec_command or a one-child functions.exec cell while
another independent operation is known. Re-apply the gate after every result:
newly revealed sibling reads, searches, status checks, diffs, hashes, and
verification steps must be batched during follow-up and closeout too, not only
during initial discovery.

Sequential execution is allowed only when the next call's arguments or safety
genuinely depend on the current result, for waits/resumes or approvals, or for
conflicting mutations and shared resources. Habit, convenience, and an
already-sequential investigation are not exceptions.

Keep batches evidence-safe: give each child narrow output bounds, inspect every
settled result, and reduce or persist oversized output instead of splitting
otherwise independent work across outer calls.

</details>

The important additions were:

| Addition | Intended effect |
| --- | --- |
| “Hard pre-call gate” | Force an explicit inventory before each call rather than relying on a general preference. |
| Re-apply after every result | Prevent regression to singleton calls during follow-up and closeout. |
| No direct or one-child call when a sibling is known | Make the failure mode concrete and locally testable. |
| Explicit sequential exceptions | Preserve dependencies, waits, approvals, adaptive investigation, and conflicting mutations. |
| Per-child output bounds | Avoid serializing independent reads merely because their combined output might be large. |

Observed behavior with the hardened wording

| Rollout observation | Shell calls | Calls grouped into multi-call stages | Pure / mixed / non-Promise cells | Direct shell calls |
| --- | ---: | ---: | ---: | ---: |
| First fresh observed stream | 205 | 95.1% | 30 / 12 / 6 | 0 |
| Longer monitored stream | 351 | 92.9% | 60 / 7 / 17 | 6 |
| Companion resumed/fork stream | 74 | 95.9% | 11 / 0 / 1 | 2 |
| Two newest fresh streams combined | 72 | 100.0% | 18 / 0 / 0 | 0 |

The longer streams were not perfect, but the newest fresh sessions issued all 72 shell calls through 18 multi-call Promise stages, with no direct shell calls.

The exact wording and optional audit tooling are available here:

https://github.com/tho-stack/codex-pretool-batching

My main takeaway is that “parallelize when possible” is too abstract. The model responded much more consistently when the instruction defined:

when the decision must happen: before every call and again after every result;
what failure looks like: a direct or one-child call while an independent sibling is already known;
how to execute it: one bounded functions.exec stage using Promise.allSettled;
when not to batch: genuine dependencies, adaptive steps, waits, approvals, and conflicting mutations.

This supports the issue’s core conclusion: the runtime can batch effectively, but the default model/tool guidance would benefit from a concrete multi-call example and an explicit pre-call decision rule.

ForrestGrump · 1 month ago

@tho-stack Your framework is a clear improvement in specification quality, so I gave it the controlled test you noted was missing: 3 fresh trials/arm, arm A = the short mechanism-plus-exceptions wording from my earlier comments, arm B = your hardened "hard pre-call gate" wording verbatim, same frozen 48-module repo, gpt-5.6-sol @ xhigh, fresh sessions, alternating order (harness: setup --agents-a/--agents-b, texts in instructions/ — https://github.com/ForrestGrump/codex-batch-bench).

| metric (mean, 3 trials) | short wording | hardened gate | Δ |
|---|---|---|---|
| model cycles | 5.3 | 5.7 | +8% |
| raw tokens | 167k | 194k | +16% |
| output tokens | 6.3k | 8.7k | +37% |
| weighted credits | 11.7 | 14.0 | +20% |
| exec cells with ≥2 calls | 1.3 | 2.7 | better grouping |
| Promise cells / exec cells | 11/13 | 14/14 | slightly better |
| answer grade | 48/48 + 40/40 | 48/48 + 40/40 | equal |

On a short bounded task, the hardened wording does group more consistently — but the gains never convert to savings: the gate discipline shows up as more output tokens (60× the weight of cached input on the rate card) plus an extra cycle, and the 3× longer instruction rides every request. Best hardened trial ≈ worst short-wording trial on credits.

Two caveats that keep your result and mine compatible: (1) my benchmark tasks are short and single-stage — your failure mode (singleton regression during follow-up and closeout of long sessions) is real-usage territory my harness barely probes, and your longitudinal data speaks to exactly that; (2) n=3/arm, ranges overlap.

Synthesis for the description-PR discussion: your items 1–4 (when / what failure looks like / how / when not) look like the right contract structure, but maximum-strength enforcement wording ("hard gate", "habit is not an exception") measurably costs tokens on tasks that were already batching fine. That's the same shape as the effort-curve result upthread — the wording should teach the decision boundary concretely, and stop there. A middle variant (short wording + one sentence extending it to follow-up/closeout, without the gate framing) is the obvious next A/B; happy to run it if useful.

MakerOfToys · 29 days ago

Today an OpenAI employee posted (see below) on X that the 5h limit is returning and that Sol should be 18% more efficient for typical use.

What exactly counts as "typical use"? Is it asking a simple question or renaming a file?

I went back to my original tests I described above and reran them without the batching instructions, in their original state, just like I did last week.

I ran every task without batching instructions twice. So that's 4 tasks × 2 = 8 runs. The average results were:

23.32% more API-equivalent usage
38.62% more total tokens
41.00% more cached input
32.31% more response steps

It's honestly confusing and so far from what OpenAI is claiming.

On X it's "Yay, more efficient!", the Codex pricing page shows more messages for 5.6, and the release post highlights improved token efficiency. Yet my results show the exact opposite.

It seems that four small, scoped analysis tasks in small repositories are not considered "typical use."

ID-Yo · 29 days ago

Today an OpenAI employee posted (see below) on X that the 5h limit is returning and that Sol should be 18% more efficient for typical use. What exactly counts as "typical use"? Is it asking a simple question or renaming a file? I went back to my original tests I described above and reran them without the batching instructions, in their original state, just like I did last week. I ran every task without batching instructions twice. So that's 4 tasks × 2 = 8 runs. The average results were: 23.32% more API-equivalent usage 38.62% more total tokens 41.00% more cached input 32.31% more response steps It's honestly confusing and so far from what OpenAI is claiming. On X it's "Yay, more efficient!", the Codex pricing page shows more messages for 5.6, and the release post highlights improved token efficiency. Yet my results show the exact opposite. It seems that four small, scoped analysis tasks in small repositories are not considered "typical use." <img alt="Image" width="889" height="1323" src="https://private-user-images.githubusercontent.com/214913486/628792371-f10c4d30-d5fd-4949-9f5d-1457c3ca080b.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3ODUzOTU0NzksIm5iZiI6MTc4NTM5NTE3OSwicGF0aCI6Ii8yMTQ5MTM0ODYvNjI4NzkyMzcxLWYxMGM0ZDMwLWQ1ZmQtNDk0OS05ZjVkLTE0NTdjM2NhMDgwYi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjYwNzMwJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI2MDczMFQwNzA2MTlaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT01ZmM3Y2ViYzA1MTJmOTc0ZjYyZGYzYThiOGExYWMzMzgzZTUwMTlhZjU3OTAwY2Y2OTUyZTkxNzMyODZjY2Y3JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZyZXNwb25zZS1jb250ZW50LXR5cGU9aW1hZ2UlMkZwbmcifQ.UamEaqfkf2N981PO2jJWLofLTB190DQ3EEJUd6FckOM">

I'm monitoring your work and today got curious. Are you using CLI or Codex desktop? And, have you applied the latest update (from yesterday 29th) before making the updated tests (those with the claim for 18% improvement). Thanks for your work!

MakerOfToys · 29 days ago

@ID-Yo To be honest, I didn’t run the same tests in the CLI because I use the Codex app about 99% of the time. I obviously can’t claim anything without testing it, but I suspect the CLI may have the same issue, since the root of the problem seems to be the model’s instructions and how it is guided to use batching.

As for the July 29 changes, I saw the model-instruction updates yesterday, so I automatically assumed they were related to the claimed efficiency improvement. I’ll probably run some more tests and make sure to include a CLI run this time.

Thanks for the idea!

ID-Yo · 29 days ago

@MakerOfToys That would be interesting to see. I myself rarely use the CLI and use mainly the Codex app, although sometimes see some benefits in CLI. I asked you about the because yesterday there was an update of Codex app, but I saw it just today, just before your update. Yesterday, after the reset, I saw no immediate positive effect on the usage, but I haven't made any tests, it is all based on how I feel it. Again, based on how it feels, I see that some accounts seem to perform better that the others, which corresponds with possible A/B testing that is in progress. This gave me an idea for you. It seems like you taking the scientific approach of rigorous testing, comparing apples with apples. If you are interested in this, I would suggest to make some form of software/proxy, something like OpenCodex (fork maybe) or similar, that makes neccessary changes for test A, executes the prompt, makes the measurement and then does test B.... Now as I think about it, part of this (settings, prompts) can be done in the easiest form of automation that is currently present: skills and/or plugins within Codex. The easier it is for others to replicate, the bigger group will be those that apply and test it (I myself included). The bigger group comes to the same conclusion, the bigger chances for update by OpenAI.

MakerOfToys · 29 days ago

@ID-Yo yeah, really great ideas, it's definitely something that should be considered. Actually, I just noticed one thing. Right now it appears that there is a big gap between latest Codex CLI and Codex (now ChatGPT) app.

Version comparison:
CLI (standalone): v0.146.0
CLI (integrated in app): 0.146.0-alpha.3.1

Looking and diff between the two reveals quite some gap of new changes that seem to affect the performance/usage.
Still needs reviewing but this seems promising. So at least in my case newest desktop (Windows) build does not have the newest CLI ver.

ID-Yo · 29 days ago

@MakerOfToys Here is one more thing to read on the topic, a great article from yesterday. Not sure where they had applied the things mentioned (hey guys from @codex / OpenAI, great article but next time just say where it was applied and when will be available), but they all look promising: https://openai.com/index/gpt-5-6-frontier-intelligence-efficiency/

naipi11 · 10 days ago

If GPT-5.6 burns a lot of quota on Code Mode, check whether independent reads are going out one turn at a time. Explicit batching is the user-side control that has been measured.

Method:

Put this in the project AGENTS.md (or the first prompt): when several tool calls are independent, read-only, and known up front, issue them in one turn.
Keep dependent, mutating, or approval-sensitive steps sequential.
Compare two fresh sessions on the same frozen repo: one with the batching instruction, one without.
Leave the instruction in AGENTS.md so later sessions inherit it.

Evidence: the reporter measured 27-45% lower weighted usage with explicit batching, and a later comment reproduced the same direction. This is a prompting/method change, not a config flag.

Independent community workaround; not an official OpenAI fix.

lyc280705 · 10 days ago

@rka-oai Sorry for the direct ping. This seems closely related to #26487, where Responses Lite intentionally disables top-level parallel tool calls.

The discussion here has since narrowed the remaining gap fairly well: Code Mode already supports concurrent nested calls, but the current model-facing exec description still only demonstrates a single await tools.exec_command(...) call and does not describe the Promise.all / Promise.allSettled replacement path or its decision boundary. Multiple controlled reproductions in this thread found that explicit, bounded batching materially reduces outer model cycles and repeated cached-context processing at xhigh/max, while high and below need a non-regression-sensitive/conditional rule rather than unconditional fan-out.

Could you clarify whether this serialization behavior is an intentional Responses Lite tradeoff, already tracked internally, or an unaddressed regression from the transition? A maintainer steer on the intended contract would be very helpful before the community keeps iterating on increasingly strong AGENTS.md workarounds.

phaynes · 8 days ago

<head></head><h2 style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">Independent production trace: 121 Sol calls and ~333.6 credits for one four-turn planning/review cycle</h2><p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">I have independently investigated this issue from the economics of a real, large-context Codex workflow.<p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">I think the problem is broader than individual tool-call batching. Codex currently appears to use the selected premium reasoning model for too many phases of the request pipeline, including repeated evidence acquisition and context processing where deterministic processing, batching or a cheaper model could do the same work.<h3 style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">Environment</h3><ul style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;"><li>ChatGPT Pro</li><li>Codex Desktop 26.814.41407</li><li>macOS / Apple silicon</li><li>GPT-5.6 Sol</li><li>reasoning effort: High</li><li>service tier: Standard/default</li><li>analysed session: <code>01a018f5-5d19-7bf2-a553-034dca485ed5</code></li><li>date: 19 August 2026, Australia/Sydney</li></ul><p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">Importantly, this particular trace was not Fast mode, so no Fast-mode multiplier is included below.<h3 style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">Observed session</h3><p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">The session contained:<ul style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;"><li>121 Sol model invocations</li><li>116 tool calls</li><li>15.64M input tokens<ul><li>approximately 15.00M cached</li><li>approximately 643k uncached</li></ul></li><li>87.6k output tokens</li><li>18.3k reasoning tokens</li><li>1 context compaction</li><li>18 tool outputs larger than 30 KB</li><li>approximately 1.23 MB of tool output</li><li>final model request: approximately 200,328 input tokens, of which 199,424 were cached</li></ul><p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">Using the published Sol rate of 125 credits/M uncached input, 12.5/M cached input and 750/M output gives:<pre style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;"><code>uncached input: 0.643M × 125 = 80.38 credits
cached input: 15.000M × 12.5 = 187.50 credits
output: 0.0876M × 750 = 65.70 credits
------
estimated total 333.58 credits</code></pre><p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">This is a rate-card reconstruction from local telemetry, not a claim about OpenAI's private billing ledger.<h3 style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">Where it occurred</h3><p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">The four user turns divide unusually clearly:
Work | Sol calls | Input tokens | Estimated credits
-- | -- | -- | --
Prepare implementation plan | 54 | 7.08M | ~137.8
Review bootstrap | 13 | 1.14M | ~27.5
Review Claude feedback | 2 | 96k | ~5.0
Update specification from feedback | 52 | 7.32M | ~163.3
Total | 121 | 15.636M | ~333.6

<p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">I am not proposing that the complete task should simply be moved to Luna.<p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">The point is that the current architecture appears to make model selection at too coarse a granularity.<p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">A more economical pipeline is:<pre style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;"><code>deterministic/batched retrieval
↓
bounded evidence packet
↓
Luna: mechanical synthesis / preparation where needed
↓
Terra: routine implementation/reasoning
↓
Sol: bounded load-bearing decision/review
↓
deterministic validation</code></pre><p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">Retaining Sol for the decisions for which Sol was selected, while moving only mechanically reducible work away from it, makes a 60–80% reduction in total weighted usage a conservative target. The 60% lower bound is available from Terra pricing alone before counting any reduction in repeated context processing.<h3 style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">Experimental follow-up</h3><p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">I subsequently tested the pipeline architecture in an overnight Varro governed trace-validation experiment.<p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">The experiment deliberately used no OpenAI credits. Deterministic Rust/local processing handled repository facts, graph checks, validation, tests and benchmarking; routine construction used Sonnet 5; expensive Opus 5 review received only bounded evidence packets and was prohibited from browsing the repository.<p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">The experiment completed within the overnight window.<p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">Its significance here is architectural rather than a Claude-vs-OpenAI price comparison: frontier reasoning did not need to own repository retrieval, repeated tool loops, mechanical validation or every intermediate transformation in order to remain the final high-quality review authority.<h3 style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">Compaction exposes the same architectural problem</h3><p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">There is also a closely related example in context compaction.<p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">Current Codex compaction is model-mediated and inherits the active turn/session model configuration. A user selecting Sol because a task requires frontier reasoning can therefore also spend Sol-class credits summarising the conversation history.<p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">That is tracked separately in #22486.<p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">Compaction may require semantic summarisation, but it does not follow that it should automatically inherit the most expensive active reasoning route. Deterministic extraction + cheaper synthesis + validation + fallback would provide a safer economic architecture.<h3 style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">What I think should change</h3><p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">I suggest treating model invocation itself as a scarce resource and planning the request pipeline before execution.<p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">In particular:<ol start="1" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;"><li>Batch independent repository and tool reads before returning to the model.</li><li>Do not invoke a frontier model merely to decide to perform the next mechanical operation.</li><li>Produce bounded evidence packets at model boundaries rather than repeatedly feeding accumulated repository history back into Sol.</li><li>Support phase-specific model selection for retrieval/synthesis, implementation, review and compaction.</li><li>Make compaction model and reasoning effort independently configurable.</li><li>Prefer deterministic code for mechanically derivable validation and state extraction.</li><li>Expose per-phase credit telemetry:<ul><li>model/reasoning;</li><li>evidence acquisition;</li><li>cached-context processing;</li><li>tool coordination;</li><li>waiting/polling;</li><li>compaction;</li><li>retries;</li><li>subagents.</li></ul></li><li>Add weighted-credit regression tests for representative long-context workflows, rather than optimising only model-output quality.</li></ol><h3 style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">Why this matters now</h3><p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">Under per-token charging this is no longer merely an implementation efficiency issue.<p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">The orchestration strategy is directly visible to the customer as cost.<p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">A model/tool pipeline which is 2–5× less efficient than necessary can turn a technically successful Codex workflow into an economically unusable one without improving the final answer.<p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-line: none; text-decoration-thickness: auto; text-decoration-style: solid;">I am happy to provide the sanitised rollout telemetry and analysis methodology for session <code>01a018f5-5d19-7bf2-a553-034dca485ed5</code>.

brandon-julio-t · 8 days ago

Cross-client validation: client-level batching (OpenCode) ≈ 3.3× lower credits per tool call, same account, no prompt changes

---

The controlled results in this thread show explicit batching instructions reduce weighted usage 27–45%. I can add a complementary data point from the opposite direction: instead of instructing the model to batch, I compared Codex against OpenCode, a third-party agent client that enforces batching at the client level, on the same ChatGPT Pro account and the same GPT-5.6 Sol model. The gap is consistent with this thread's findings and isolates the mechanism further.

All numbers are aggregate metadata from local logs (Codex session rollouts vs OpenCode's local DB): event types, timestamps, token counts. No prompt content, no account identifiers. Credit weighting uses the published rate card (125 / 12.5 / 750 credits per M uncached-input / cached-input / output; reasoning counted inside output).

---

Setup

| | Codex App/CLI | OpenCode |
|---|---|---|
| Auth / plan | ChatGPT Pro (same account) | ChatGPT Pro (same account) |
| Model | gpt-5.6-sol | gpt-5.6-sol (its "-fast" catalog alias; server bills standard) |
| Reasoning effort | mostly low/high | max (biases against OpenCode) |
| Window | 7 days of real mixed work (7,792 own API requests) | 3 days of read-heavy agent work (5,732 requests) |
| Data hygiene | multi-agent mirror token_count events excluded (~68% were sibling duplicates) | subagent sessions included |

Workloads are real usage, not controlled pairs — treat ratios as indicative, not exact. The reasoning-effort asymmetry (low vs max) makes the comparison conservative: OpenCode should be more expensive per request, and isn't.

---

Headline

| Metric | Codex | OpenCode | Ratio |
|---|--:|--:|--:|
| Tool calls per request (mean) | 0.81 | 2.72 | 3.4× |
| Max tool calls observed in one request | 1 (n=7,792) | 13 | — |
| Requests per tool call | 1.23 | 0.37 | 3.3× |
| Credits per tool call | ~2.4 | ~0.7 | ~3.3× |
| Credits per round trip | 1.96 | ~1.9–2.0 | ≈1× |
| Requests per user message (median) | 8 | 8–9 | ≈1× |

The last two rows are the point: a round trip costs ~2 credits in both clients, and both make a similar number of round trips per user interaction. The only difference is how much work each round trip carries. Codex never carried more than one tool call per request in 7,792 requests; 85.9% of its requests were auto-continuations fired immediately after a single tool output. OpenCode reasons once, emits up to 13 independent calls, executes them concurrently (97% measured wall-clock overlap), waits for all results, then makes one round trip.

---

Where the credits actually go (decomposition)

Codex, 201 Sol threads (~12.3k credits):

| Bucket | Share |
|---|--:|
| Cached-prefix re-reads on later requests | 65.4% |
| Later uncached input (appends + cache breaks + compaction) | 20.0% |
| Output (incl. reasoning) | 12.1% |
| First-request overhead (system prompt + tools + AGENTS.md) | 0.5% (fresh) + 1.9% (subagent spawns) |

OpenCode, 427 sessions (~11k credits): cache re-reads 44.1%, uncached appends 35.9%, output 15.9%, first-step overhead 4.1%.

This rules out the popular alternative explanations:

System prompt size is not the problem. Fresh Codex first requests are ~22k tokens total and account for 0.5% of spend.
Cache efficiency is not the problem. Codex's prefix cache hit rate is better than OpenCode's (96.8% vs 91.8%).
Reasoning effort is not the main lever. Running OpenCode at max added only ~5% of its total spend in reasoning tokens. I can afford Sol-max there while being pushed to Sol-low in Codex — the difference is round-trip count, not thinking.

What remains is arithmetic: with a ~90k-token median context (p90 178k), every extra round trip costs ~1.2 credits in cache re-reads plus fresh output before any new work happens. One-call-per-request multiplies exactly that. Serialization also compounds: it inflates transcript length, which inflates context for all subsequent trips (heaviest observed thread: 1,010 requests for 22 user messages, 140.6M cached tokens re-read, ~2.2k credits).

---

Two secondary findings from the same data

Remote compaction is cache-hostile. Across 18 compaction episodes, the first post-compaction request was only ~29% cached (median): the remotely compacted history comes back as uncached input, and 15 of 53 observed mid-thread cache breaks occurred right after compaction. Median post-compaction request: 29.2k uncached (p90 57.8k). Serialization → longer transcripts → more compactions → more uncached re-billing. Related in spirit to #37305 (local path); the remote path has its own version of the problem.
Auto-continue cadence: request triggers were 85.9% tool-output, 9.3% user, 4.0% self-continuation. The loop, not the user, decides the request count.

---

Alignment with this thread

Corroborates the 27–45% weighted-usage reduction from batching instructions, and suggests the ceiling is higher (~3.3× per tool call here) when batching is enforced by the client rather than requested via AGENTS.md — the model doesn't "forget" to batch (cf. the ~1,600 batchable vs ~100 batched observation above).
Consistent with #35618 / #26487, and one clarification from source: "Responses Lite" is a per-model request profile, not the transport. The GPT-5.6 family (gpt-5.6-sol/terra/luna) is flagged use_responses_lite in models-manager/models.json, and core/src/client.rs forces parallel_tool_calls: prompt.parallel_tool_calls && !model_info.use_responses_lite — i.e. always false for these models, over both HTTPS and the responses_websockets=2026-02-06 transport. Notably, OpenCode talks to the same model and backend endpoint over plain HTTPS Responses (its own WS transport is env-gated off by default in the tested build) without the lite profile, and there the model emits up to 13 tool calls in one response — the batching capability exists server-side; the lite client policy suppresses it. The coupling is as old as the profile itself — #26487 introduced use_responses_lite, the parallel-tools override, and forced persistent reasoning (reasoning.context: all_turns) in the same change, and #27946 notes tool namespacing was waiting on "coordination & fixes in Responses API (around collisions & return items)" — suggesting single-call-per-turn is a lite-contract limitation, not a model limitation. The exec tool description showing only single-call examples (this issue) compounds it.
Suggested direction, matching prior comments: enable parallel tool calls (or a bounded batch contract) for GPT-5.6, teach batching in the model-facing exec description with a Promise.all/allSettled example and decision boundary, and treat the "wait for all results, then one round trip" barrier as the default loop discipline rather than an opt-in prompt hack.

---

Caveats

Different workloads and durations; ratios are indicative. Requests-per-user-message being equal (~8) suggests comparable interaction granularity.
OpenCode has its own waste: 12.6% of its spend came from mid-session cache breaks, and heavy subagent fan-out drove 74% of its spend. It wins on loop shape, not on general efficiency.
OpenCode's "-fast" alias requests service_tier: "priority" but is serviced standard (originator-gated, and sent over HTTPS where priority routing reportedly doesn't engage anyway), so both sides were billed at standard Sol rates — verified via alternating throughput benchmarks (~54 tok/s both).
Credits computed from token categories × rate card; the plan's quota UI moved consistently with these computations during observation (~860 standard-rate credits per displayed percentage point on Pro).

GPT-5.6 often serializes independent Code Mode calls; explicit batching reduced weighted usage by 27–45%

What version of the Codex App are you using (From “About Codex” dialog)?

What subscription do you have?

What platform is your computer?

What issue are you seeing?

Main results

Normal-usage observation

Quality

What steps can reproduce the bug?

What is the expected behavior?

Additional information

Limitations

25 Comments

Setup

Results — synthetic 48-module repo, effort sweep (no-instruction ÷ instructed)

Results — three real repos, xhigh, 2 trials/arm each

Ultra is its own problem

Repro

Why this scope still fits the current source

Proposed PR scope

Tests I would add

Validation before calling it fixed

Independent Codex Desktop replication at medium effort

Setup

Results

Quality/output caveat

Follow-up: stronger AGENTS.md wording substantially changed normal-session batching

Historical behavior

First instruction

Hardened instruction

Observed behavior with the hardened wording

Cross-client validation: client-level batching (OpenCode) ≈ 3.3× lower credits per tool call, same account, no prompt changes

Setup

Headline

Where the credits actually go (decomposition)

Two secondary findings from the same data

Alignment with this thread

Caveats

Follow-up: stronger `AGENTS.md` wording substantially changed normal-session batching