A response.completed without a usage block silently skips all token totals and the rollout budget
What version of Codex CLI is running?
main (record_token_usage_info at codex-rs/core/src/session/mod.rs:3781)
What subscription do you have?
n/a (self-hosted)
Which model were you using?
Any (mechanism is model-independent; reachable with OpenAI-compatible providers that omit usage)
What platform is your computer?
macOS arm64
What terminal emulator and version are you using (if applicable)?
n/a
Codex doctor report
not available
What issue are you seeing?
Token usage is recorded into session accounting in exactly one place in the turn path: ResponseEvent::Completed { token_usage, .. } at core/src/session/turn.rs:2529 calls record_token_usage_info (core/src/session/mod.rs:3781-3815), which does:
if let Some(token_usage) = token_usage {
let token_info = { ... update_token_info_from_usage ... };
let budget_result = self.record_rollout_budget_usage(token_usage);
... contributor notifications ...
budget_result?;
}
When a response.completed event carries no usage block, token_usage is None (parsed from usage: Option<ResponseCompletedUsage> with #[serde(default)] at codex-api/src/sse/responses.rs:112-120), and the if let Some skips all three:
- session token totals (
update_token_info_from_usage) are not updated; - the rollout budget check (
record_rollout_budget_usage) is not run - and this is the ONLY call site in the turn path (core/src/session/rollout_budget.rs:26), so a usage-less completion fully escapes budget accounting; - token-usage contributor notifications are skipped.
The skip is silent: no warning, no fallback, no conservative treatment. A usage-less response.completed is an explicitly tolerated input (there is a test asserting token_usage.is_none() for a usage-less completed event, responses.rs:745-810), so this is not an error path - it is a designed input that produces zero accounting.
Related: response.incomplete events (max_output_tokens stops etc.) may carry usage server-side, but the parser at responses.rs:426-436 extracts only incomplete_details.reason and discards the rest, so a token-limited stop also contributes nothing.
What steps can reproduce the bug?
Static demonstration (no live run required):
- Configure codex with an OpenAI-compatible provider that may omit usage on
response.completed(or intercept the SSE stream and strip the usage block). - Run a turn; observe the completed event has no usage.
- Observe session totals unchanged and rollout budget not decremented for that turn.
- There is no log line indicating the skip.
What is the expected behavior?
A usage-less completion should at minimum (a) log that usage was absent, (b) keep budget accounting conservative (e.g. treat unknown usage as not exceeding the budget, or as consuming the remaining budget), rather than silently skipping both totals and the budget gate.
Additional information
Suggested fix options:
- In
record_token_usage_info, whentoken_usageisNone, log a warning and record a "usage unknown" marker instead of silently returning. - Keep
record_rollout_budget_usagereachable on the None path with a conservative assumption (e.g. count the turn against the budget with an estimated/unknown weight). - For
response.incomplete, extract and record the usage block when present (incomplete_detailsis parsed but the response's usage is discarded).
3 Comments
Thanks for the detailed repro and code pointers. I confirmed this path: when ResponseEvent::Completed has token_usage: None, record_token_usage_info returns early and skips record_rollout_budget_usage, so rollout accounting sees a silent zero delta. Could we treat missing usage as tracked unknown-usage, emit a warning, and still run a conservative budget gate instead of bypassing checks? That keeps enforcement intact for providers that omit usage on complete events.
Dynamic reproduction added: a cargo test (
zz_c5_usage_none_skips_totals_and_budget) drives the realSession::record_token_usage_info:record_token_usage_info(&turn_context, None)(aresponse.completedwithout usage).total_token_usageandlast_token_usageare byte-identical before and after the None call.Result: the None call is a silent no-op on totals, and
record_rollout_budget_usage(single call site at session/mod.rs:3789, inside theif let Some) is never invoked, so a usage-less completion escapes both totals and budget accounting with no warning. Test: 1 passed.Confirmed on current
main:ResponseEvent::Completed { token_usage: None }reachesSession::record_token_usage_info, and theif let Some(token_usage)skips totals,record_rollout_budget_usage, and contributor notifications with no log.I am not opening a PR.
docs/contributing.mdsays Codex does not accept external code PRs.Suggested scope if the team picks this up:
response.completedwithoutusageas a first-class input (the parser already allowsusage: Option<_>).token_usage == None, log that usage is unknown. Do not invent token counts.response.incompleteis a separate bug: the SSE handler maps it toApiError::Streamand dropsusage. That is #38831 (retryable stream failure) as well as lost accounting.A regression test can stay fully synthetic: record a measured usage, then call
record_token_usage_info(..., None)and assert totals are unchanged while a warning/unknown branch is taken.