core: per-turn token-count estimate serializes the full history for a trace-only log field; gate it behind trace-enabled

Open 💬 1 comment Opened Jun 20, 2026 by HamburgJ

Summary

On the normal per-turn path, Codex computes a full-history token estimate once per completed sampling result, and that value is used only to populate a trace!-level log field. The estimate iterates the entire conversation history and serde_json::to_strings each item, is recomputed every turn with no memoization of that work, and runs unconditionally — even when no trace subscriber is attached. No compaction, budgeting, or control-flow logic reads the per-turn value.

Root cause (current main)

  • codex-rs/core/src/session/turn.rs: in the run_turn.collect_post_sampling_state block, get_estimated_token_count(...) is .await-ed once per completed sampling result. Its Option<i64> result is consumed only by the estimated_token_count = ?estimated_token_count field of the trace!("post sampling token usage") call. Per-turn control flow uses the separate, already-cached token_status (e.g. token_limit_reached), not this value.
  • There is no tracing::enabled! / event_enabled! guard around the call — it is a plain unconditional .await. With tracing disabled, only the ?-field formatting is deferred by the macro; the value itself is already computed.
  • Call chain: get_estimated_token_count (session/mod.rs) -> History::estimate_token_count(turn_context) -> estimate_token_count_with_base_instructions (codex-rs/core/src/context_manager/history.rs). It folds over the whole items vec via estimate_item_token_count, which runs serde_json::to_string per non-reasoning item (encrypted reasoning / compaction items use .len()).
  • No memoization of that work: the only relevant cache (ORIGINAL_IMAGE_ESTIMATE_CACHE) memoizes solely the per-image original-detail byte estimate, not the serde serialization or the per-turn fold — so the hot path is recomputed every turn.

Impact

One full-history JSON serialization per turn, O(total history bytes), growing as the conversation accumulates — avoidable CPU and transient allocations on the hot turn path, paid even when no trace subscriber is attached. Not a correctness issue: compaction/budgeting use a separate cached counter and are unaffected.

Proposed fix

Compute the estimate only when the relevant trace level is enabled (guard the get_estimated_token_count call with tracing::enabled!(Level::TRACE) / event_enabled! before evaluating it). Scope strictly to this turn.rs call site — do not change estimate_token_count_with_base_instructions, which is legitimately load-bearing for remote compaction (compact_remote.rs) and token recompute (session/mod.rs: recompute_token_usage and the BodyAfterPrefix prefix-token path).

Happy to put up the one-line-guard PR if useful.

(Line numbers omitted as they drift; the symbol names above are the stable references against current main.)

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗