[hooks] Expose per-model-request timing in rollout for observability integrations
Feature request
Please expose stable timing for each model sampling request in the persisted rollout available through the Stop hook's transcript_path, or provide an equivalent model lifecycle hook.
For each sampling request, observability integrations need:
- model request start time
- first output time, or the existing per-request
ttft_ms - model response completion time
- a stable correlation key linking those timings to emitted
response_itemrecords, token usage, and the existingturn_id
This would allow integrations to map the lifecycle to Langfuse generation.startTime, generation.completionStartTime, and generation.endTime without inferring model timing from tool events.
Current behavior
The current HookEventName enum contains these actual variants:
PreToolUse, PermissionRequest, PostToolUse, PreCompact, PostCompact, SessionStart, SessionEnd, UserPromptSubmit, SubagentStart, SubagentStop, and Stop.
There is no model-request, response-start, first-output, or response-completed hook. Source: HookEventName.
The current StopRequest contains these actual fields:
session_id, turn_id, cwd, transcript_path, model, permission_mode, stop_hook_active, last_assistant_message, and target.
Source: StopRequest.
A real Codex rollout contains a turn-level terminal record with this field shape (values redacted; field names are unchanged):
{
"timestamp": "<redacted>",
"type": "event_msg",
"payload": {
"type": "task_complete",
"turn_id": "<redacted>",
"started_at": "<redacted>",
"completed_at": "<redacted>",
"duration_ms": "<redacted>",
"time_to_first_token_ms": "<redacted>"
}
}
The public protocol defines the same actual fields on TurnCompleteEvent, and documents time_to_first_token_ms as the duration between turn start and the first model token. Source: TurnCompleteEvent.
A turn can contain multiple sampling requests separated by tools, so this turn-scoped value cannot identify completion start for later generations.
The relevant per-request rollout boundary currently has this actual field shape (values redacted; field names are unchanged):
{
"timestamp": "<redacted>",
"type": "response_item",
"payload": {
"type": "custom_tool_call",
"id": "<redacted>",
"status": "completed",
"call_id": "<redacted>",
"name": "exec",
"internal_chat_message_metadata_passthrough": {
"turn_id": "<redacted>"
}
}
}
{
"timestamp": "<redacted>",
"type": "event_msg",
"payload": {
"type": "token_count",
"info": {
"last_token_usage": {
"input_tokens": "<redacted>",
"cached_input_tokens": "<redacted>",
"cache_write_input_tokens": "<redacted>",
"output_tokens": "<redacted>",
"reasoning_output_tokens": "<redacted>",
"total_tokens": "<redacted>"
}
}
}
}
These records do not expose the enclosing sampling request's start, first output, or completion timestamps. A post-hoc tracing integration therefore has to infer generation boundaries from response_item and token_count; in tool-calling turns that can make generation timing collapse onto, or include, tool execution.
Existing signals that could be reused
Codex already computes relevant timing internally:
TurnCompleteEvent.time_to_first_token_msis exposed, but it is turn-scoped.- PR #30883 attaches the actual per-request field
ttft_msto the existingcodex.sse_event/response.completedtelemetry record. ItemStartedEvent.started_at_msandItemCompletedEvent.started_at_ms/completed_at_msdescribe item lifecycles, not the enclosing model sampling request. Source:ItemStartedEventandItemCompletedEvent.
The requested change could reuse the existing per-request ttft_ms measurement rather than measure TTFT again. Please persist/expose it together with model-request start and response-completion timing at the hook/rollout boundary. The exact new schema is intentionally left to maintainers; all field names shown above are fields Codex already emits or defines today.
Reproduction
- Configure a
Stophook that readstranscript_path. - Run a turn with multiple model requests separated by tool calls.
- Inspect the rollout around
response_itemandevent_msg.token_count. - Attempt to reconstruct one generation observation per model request.
- The turn-level
time_to_first_token_mscannot be assigned to later requests, and no per-request model lifecycle is available.
Related
- #30883 added per-request TTFT completion telemetry.
- #33420 requests a streamed assistant-message hook, but assistant text deltas do not cover reasoning-only or tool-call responses and do not provide request start.
- #37317 discusses native OTel response signals and their volume.
- #35363 added item start times, which helps item timing but does not establish the model-request lifecycle.
3 Comments
Implementation is ready in the fork:
GitHub currently reports that this repository limits opening pull requests to collaborators, so the upstream PR could not be created from this account. The comparison is ready for a collaborator to review or open as a PR.
Implementation summary (ready for review if invited)
The proposed change is fully implemented in the fork branch:
https://github.com/yunhungo/codex/tree/codex/per-request-rollout-timing
(commit
9fb70b9e"Persist per-response model timing", based on upstream3b366654)Design (following the issue's request to reuse existing signals):
ModelResponseTimingonRawResponseCompletedEvent:request_started_at_ms: wall-clock timestamp taken immediately before the transport requestfirst_output_at_ms: wall-clock timestamp of the firstresponse.output_item.addedcompleted_at_ms: wall-clock timestamp whenresponse.completedis observedttft_ms: the existing mapped-stream-to-first-output latency, reused (not re-measured)turn_idadded toRawResponseCompletedEventas the correlation key linking timing to the enclosing turn,response_itemrecords, and token usageshould_persist_event_msgincodex-rs/rolloutnow persistsRawResponseCompletedsoStophook consumers can read it viatranscript_path(previously dropped)serde(default, skip_serializing_if)codex-apiSSE andRawResponseCompletedNotification(JSON schema + TS types + precomputed bundles), and through compact pathsTests: new end-to-end mock-server test
codex-rs/core/tests/suite/raw_response_timing.rsasserts the persisted rollout contains exactly oneRawResponseCompletedper response with a non-emptyturn_idand monotonic timing (request_started_at_ms <= first_output_at_ms <= completed_at_ms), matching the issue's reconstruction scenario.Happy to rebase onto latest
mainand address review feedback if the team would like to take this forward.Hi @yunhungo, this subagent persistence/history issue aligns with some boundary anomalies observed in multi-agent rollouts. Codex Rescue Alpha5 provides read-only lifecycle and subagent boundary diagnostics, cleanly separating historical start markers from current live execution state without altering the source rollout.
If you have access to the local session, you can run a non-destructive check:
No raw session data is required, and please redact private paths if you share any output.