Change `TokenCount` to not re-emit `last_token_usage` on rate-limit-only updates

Resolved 💬 3 comments Opened Mar 12, 2026 by Gusarich Closed Apr 18, 2026

Summary

EventMsg::TokenCount currently serves two different purposes:

  1. reporting a real token-usage update
  2. rebroadcasting the latest rate-limit snapshot

When only rate limits change, Codex emits a new TokenCount event with unchanged total_token_usage and the previous nonzero last_token_usage. That makes the JSONL rollout misleading for consumers that interpret last_token_usage as the incremental usage for that event.

I noticed this while comparing my own local usage counter against slopmeter and @ccusage/codex. Both over-reported on my real session logs for this reason.

Why this is a bug

last_token_usage looks like a per-event delta field. Re-emitting it unchanged on a pure rate-limit update makes the same usage chunk appear multiple times in the session log.

This is not just a tooling misunderstanding. There is a nearby comment that says this path should avoid duplicate TokenCount events, but the current implementation still emits them.

Root cause

  1. update_token_usage_info() appends new usage and emits TokenCount.
  1. update_token_info() stores that usage in TokenUsageInfo, preserving the most recent nonzero last_token_usage.
  1. update_rate_limits() updates only rate limits, but also emits TokenCount.
  1. send_token_count_event() republishes the stored token_info together with the latest rate limits.

So a rate-limit-only update can emit a fresh TokenCount event that repeats the previous nonzero last_token_usage even though total_token_usage did not advance.

There is also a direct comment / implementation mismatch here:

The comment says rate-limit handling should defer sending "to avoid duplicate TokenCount events", but the implementation immediately calls sess.update_rate_limits(&turn_context, snapshot).await, and that path emits TokenCount right away.

Impact

This affects tools that parse session JSONL rollouts for usage accounting. A reasonable implementation is to treat each token_count event as a new usage snapshot and last_token_usage as the incremental usage for that event. With the current behavior, that interpretation overcounts.

This is exactly what happened in my case:

  • my own local counter, which keyed off cumulative advancement, produced lower totals
  • slopmeter and @ccusage/codex, which trusted last_token_usage more directly, produced higher totals

Fix direction

Rate-limit-only updates should not emit a TokenCount event that repeats a stale nonzero last_token_usage.

Any of these would resolve the ambiguity:

  • do not emit TokenCount from update_rate_limits() when usage did not change
  • emit a separate rate-limit event instead
  • or ensure a rate-limit-only rebroadcast does not carry a nonzero stale last_token_usage

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗