Desktop/thread tokens_used reflects lifetime total_token_usage, easily mistaken for context window size
What version of Codex is running?
CLI/Desktop ~0.147.x (also applies to current app-server thread state)
What subscription do you have?
ChatGPT Pro (also reproduces when using a custom Responses-compatible model_provider with API-key auth)
Which model were you using?
gpt-5.6-sol / gpt-5.6-terra / gpt-5.6-luna (also occurs with stock ChatGPT-backed sessions)
What platform is your computer?
macOS (Desktop + CLI)
What issue are you seeing?
Long-running threads show extremely large tokens_used values in Desktop thread state (tens to hundreds of millions). This is easy to misread as current context window size, which makes sessions look “over context” even when the last turn fits comfortably under the model limit (~272k for GPT-5.6 family).
Root observation (from rollouts)
Each turn stores two usage objects under event_msg.payload.info:
last_token_usage— tokens for this request (real context pressure)total_token_usage— lifetime cumulative usage for the thread
Desktop/thread DB field threads.tokens_used tracks the lifetime counter (total_token_usage.total_tokens), not last_token_usage.
Example shape (illustrative magnitudes, not a secret session):
| Field | Approx. scale |
|-------|----------------|
| last_token_usage.total_tokens | ~90k–170k (fits 272k window) |
| total_token_usage.total_tokens / threads.tokens_used | tens–hundreds of millions after long work |
So the product behaved correctly for context limits, but the accounting/UX surface is misleading.
What steps can reproduce the bug?
- Run a multi-hour coding thread with many tool turns on GPT-5.6 (stock or custom Responses provider).
- Inspect thread metadata / Desktop state for
tokens_used. - Compare with the latest rollout
last_token_usagevstotal_token_usageon a recentevent_msg. - Observe: UI/state lifetime total is huge while last-turn usage remains under the model context window.
What is the expected behavior?
Either:
A (preferred): Expose both clearly in UI and docs, e.g.
- “Session usage (lifetime)”
- “Last turn context”
and avoid labeling lifetime totals as if they were current context; or
B: Keep lifetime totals only under an explicit name (lifetime_tokens_used) and show last_token_usage where users debug “is this over context?”.
Docs should state that total_token_usage is cumulative and is not the model context occupancy for the next turn.
Additional information
Related friction when diagnosing “context full” reports:
- Large on-disk rollouts (many MB) can make Desktop open/resume feel broken even when last-turn context is fine — separate from the counter naming issue, but often co-occurs in long threads.
- Custom
model_provider+ Responses path is not required to see dual counters; native path uses the same structure. The confusion is product-wide.
Happy to rephrase/redact further if needed. No proprietary infra details required to fix naming/docs/UI.
1 Comment
Confirmed from a local JSONL consumer: treating
event_msg.payload.info.last_token_usageas the per-turn signal andtotal_token_usageas cumulative produces sensible results. My parser useslast_token_usagefor input/cache/output accounting and only falls back to the cumulativetotal_tokensdelta when a token event lacks per-turn input/output fields. That distinction avoids presenting lifetime usage as current context pressure.Sanitized reference implementation: https://github.com/tylerchen0123-sudo/CODEX-Inspection-Guidelines-for-Dosage/blob/main/collector.py
If the Codex session schema changes again, I’m happy to adjust the parser and share a compatibility report.