›_ Codex Issues
Codex Issues / #40208

OTel response.completed reports total token usage as tool_token_count

Open 💬 1 comment Opened Aug 23, 2026 by hieheihei

What version of Codex CLI is running?

codex-cli 0.149.0-alpha.4.1

What subscription do you have?

ChatGPT Pro ($200/month)

Which model were you using?

gpt-5.6-sol

What platform is your computer?

Darwin 25.5.0 arm64 arm (macOS 26.5.2)

What terminal emulator and version are you using (if applicable)?

Not applicable (source-level issue)

Codex doctor report

Not run (source-level issue)

What issue are you seeing?

The response.completed OpenTelemetry event exports TokenUsage.total_tokens under the attribute tool_token_count:

tool_token_count = %usage.total_tokens

TokenUsage.total_tokens comes directly from the provider's response.completed.response.usage.total_tokens. It represents total token usage, not tool-specific token usage.

This is an OpenTelemetry attribute naming/semantic mismatch, not a token calculation error. Downstream consumers can interpret the same value as either total tokens or tool tokens.

What steps can reproduce the bug?

  1. Inspect codex-rs/otel/src/events/session_telemetry.rs on current main.
  2. Trace usage.total_tokens back to TokenUsage.total_tokens, populated from the provider's completed-response usage.
  3. Observe that the value is emitted as tool_token_count.
  4. See the existing OTel test expecting the same attribute.

Current source:

What is the expected behavior?

The OpenTelemetry attribute name should match the value's semantics. Total usage should be exposed as a total-token attribute, or the historical field should be explicitly documented with a compatibility and migration plan.

Additional information

Related but not duplicate: #33668 reports a missing codex.turn.token_usage metric.

External consumers already handle this field inconsistently: Statewright maps it to total_tokens, while ax and ai-observer treat it as tool tokens.

The naming mismatch was introduced with #2103 and remains present on current main.

View original on GitHub ↗

1 Comment

fangyuanbutinghua · 4 days ago

Proposed one-line fix (external PRs are currently limited to collaborators on this repository, so posting the change here for maintainers)

The mismatch is in codex-rs/otel/src/events/session_telemetry.rs, in sse_event_completed:

// current (line 982)
tool_token_count = %usage.total_tokens,
// proposed
total_token_count = %usage.total_tokens,

TokenUsage (defined in codex-rs/protocol/src/protocol.rs) has no tool-token field — it only carries input_tokens, cached_input_tokens, cache_write_input_tokens, output_tokens, reasoning_output_tokens, and total_tokens. So tool_token_count was mislabeled with the total-token value. Renaming the attribute to total_token_count makes the emitted semantics match the value it carries.

tool_token_count appears nowhere else in the repository and no tests or snapshots depend on it, so this single-line change is self-contained.

The change is ready on fangyuanbutinghua/codex:fix/otel-total-token-count (commit 49655e9) if a maintainer would like to apply or cherry-pick it.