›_ Codex Issues
Codex Issues / #20981

`codex-auto-review` in OTel token usage metrics is hard to map to official model pricing

Open 💬 3 comments Opened May 4, 2026 by chesha1
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

Summary

When exporting Codex metrics through OpenTelemetry and visualizing them in Prometheus/Grafana, some token usage metrics include model="codex-auto-review".

This is confusing for cost dashboards because codex-auto-review does not appear to be an official public OpenAI model name in the pricing documentation. The token usage metric has the token counts needed for accounting, but the model label cannot be directly joined with a price table keyed by official/billable model IDs such as gpt-5.4, gpt-5.4-mini, or gpt-5.3-codex.

Example metric

{"codex.turn.token_usage_sum",
app.version="0.128.0-alpha.1",
auth_mode="Chatgpt",
env="macos",
model="codex-auto-review",
originator="codex_vscode",
os="Mac_OS",
os_version="26.4.1",
scope.name="codex",
scope.version="unknown",
service.name="codex-app-server",
service.version="0.128.0-alpha.1",
session_source="subagent_guardian",
telemetry.sdk.language="rust",
telemetry.sdk.name="opentelemetry",
telemetry.sdk.version="0.31.0",
tmp_mem_enabled="false",
token_type="cached_input"
}

Why this matters

For a Grafana dashboard that estimates Codex spend from token metrics, the natural approach is to aggregate token usage by (model, token_type) and multiply it by a pricing table with the same labels.

That breaks when the metric uses codex-auto-review, because it is not clear which official model or billing rate should be used. As a result, dashboard authors have to either drop these samples, guess a mapping, or assign a fallback price, all of which can make cost reporting inaccurate.

Request

Could the Codex team clarify what model="codex-auto-review" means in OTel token usage metrics, especially for cost or credit accounting?

In particular, it would be helpful to know whether this label represents an internal Codex routing alias, a stable public label, or the actual billable model identity users should rely on when building usage dashboards.

View original on GitHub ↗

3 Comments

github-actions[bot] contributor · 3 months ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #19420

Powered by Codex Action

Necmttn · 2 months ago

For downstream cost dashboards, model="codex-auto-review" should probably stay as the display/origin label, but the metric also needs a billable identity. Something like model.display, model.effective, model.billable, pricing_source, and pricing_version would avoid forcing every OTel consumer to guess.

If the billable model is intentionally hidden, the metric should say that explicitly with a non-priceable status rather than looking like a public model id that failed to join a price table.

Generated with ax - https://github.com/Necmttn/ax

rebroad · 23 days ago

This is also affecting local cost accounting from rollout files.

I traced the current Codex catalog entry for codex-auto-review. It is intentionally a hidden routing alias with model: null and model_provider: null; the Codex source does not map it to a stable public model. The catalog's GPT-5.2 text is part of the prompt/instructions and is not a reliable billing identity. The backend can also route differently over time.

Suggested fix:

  • Keep codex-auto-review as the requested/display model.
  • Add an optional effective_model field to RawResponseCompleted.
  • Populate it from ResponseEvent::ServerModel for normal completions and remote compaction.
  • Expose the same effective/billable identity in OTel metrics, or add an explicit pricing_source/unknown status when the backend does not provide one.

I implemented this in a downstream fork as a concrete proposal:

This preserves exact accounting without forcing downstream tools to guess whether the route used GPT-5.4, GPT-5.6 Luna, or another backend model.