core: per-turn token-count estimate serializes the full history for a trace-only log field; gate it behind trace-enabled
Summary
On the normal per-turn path, Codex computes a full-history token estimate once per completed sampling result, and that value is used only to populate a trace!-level log field. The estimate iterates the entire conversation history and serde_json::to_strings each item, is recomputed every turn with no memoization of that work, and runs unconditionally — even when no trace subscriber is attached. No compaction, budgeting, or control-flow logic reads the per-turn value.
Root cause (current main)
codex-rs/core/src/session/turn.rs: in therun_turn.collect_post_sampling_stateblock,get_estimated_token_count(...)is.await-ed once per completed sampling result. ItsOption<i64>result is consumed only by theestimated_token_count = ?estimated_token_countfield of thetrace!("post sampling token usage")call. Per-turn control flow uses the separate, already-cachedtoken_status(e.g.token_limit_reached), not this value.- There is no
tracing::enabled!/event_enabled!guard around the call — it is a plain unconditional.await. With tracing disabled, only the?-field formatting is deferred by the macro; the value itself is already computed. - Call chain:
get_estimated_token_count(session/mod.rs) ->History::estimate_token_count(turn_context)->estimate_token_count_with_base_instructions(codex-rs/core/src/context_manager/history.rs). It folds over the wholeitemsvec viaestimate_item_token_count, which runsserde_json::to_stringper non-reasoning item (encrypted reasoning / compaction items use.len()). - No memoization of that work: the only relevant cache (
ORIGINAL_IMAGE_ESTIMATE_CACHE) memoizes solely the per-image original-detail byte estimate, not the serde serialization or the per-turn fold — so the hot path is recomputed every turn.
Impact
One full-history JSON serialization per turn, O(total history bytes), growing as the conversation accumulates — avoidable CPU and transient allocations on the hot turn path, paid even when no trace subscriber is attached. Not a correctness issue: compaction/budgeting use a separate cached counter and are unaffected.
Proposed fix
Compute the estimate only when the relevant trace level is enabled (guard the get_estimated_token_count call with tracing::enabled!(Level::TRACE) / event_enabled! before evaluating it). Scope strictly to this turn.rs call site — do not change estimate_token_count_with_base_instructions, which is legitimately load-bearing for remote compaction (compact_remote.rs) and token recompute (session/mod.rs: recompute_token_usage and the BodyAfterPrefix prefix-token path).
Happy to put up the one-line-guard PR if useful.
(Line numbers omitted as they drift; the symbol names above are the stable references against current main.)
This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗