[amazon-bedrock] Streaming emits cumulative snapshots as duplicate agent_message items (Mantle/Responses)

Open 💬 2 comments Opened Jun 24, 2026 by yjiseok

Summary

With the built-in amazon-bedrock provider (Mantle Responses API), a single assistant turn emits many agent_message item.completed events whose text is the monotonically-growing cumulative snapshot of the same final message. The client persists each streaming snapshot as a separate completed message item instead of coalescing deltas.

Environment

codex-cli 0.142.0 (latest stable) AND 0.143.0-alpha.9 — both affected
Provider: amazon-bedrock (built-in, Mantle Responses path)
Region: us-east-2, model openai.gpt-5.5
macOS (Apple Silicon)

Repro

codex exec --json --skip-git-repo-check "<long prompt yielding a multi-chunk response>"

Observed

A single turn emits N agent_message item.completed events; text length grows monotonically (same final message repeated as ever-longer prefixes):

0.142.0: 31 items, text length 189 -> 4049
0.143.0-alpha.9: 16 items, text length 295 -> 3386 (same pathological pattern)

Short single-chunk responses emit only 1 item, which masks the bug (a one-line "OK_ONCE" test looks fine).

In the TUI this renders the answer repeatedly/growing; the rollout jsonl also stores the duplicate message items (distinct ids) for one turn.

Expected

One agent_message item per assistant turn (output_text deltas/snapshots coalesced), as on the OpenAI-hosted Responses path.

Notes

Not reproduced on the OpenAI-hosted Responses path; appears specific to the Bedrock Mantle streaming adapter.
model_reasoning_effort (xhigh/high/medium) has no effect — duplicate count tracks response length, not effort.
The built-in amazon-bedrock provider ignores stream tuning knobs (stream_idle_timeout_ms, stream_max_retries, wire_api), so there is no user-side config workaround.

View original on GitHub ↗

2 Comments

Nicolas0315 · 2 months ago

Thanks for the detailed repro. This lines up with #30072 and likely belongs in the same failure class: completed Bedrock/Mantle stream snapshots are being treated as separate assistant items instead of being coalesced before history storage.

The narrow fix I would aim for is in the client path just before completed assistant response.output_item.done items are persisted, not in the generic SSE parser. The SSE layer currently maps each response.output_item.done directly into ResponseEvent::OutputItemDone, while session/turn.rs still has the active item / streaming UI context needed to decide whether consecutive assistant messages are snapshots of the same output.

I would keep the coalescing conservative:

same item.id: replace earlier assistant snapshot with the later one;
different IDs: only coalesce consecutive assistant text messages in the same sampling request when role/phase match, no tool/reasoning/web/image item intervenes, and the older text is a strict prefix of the newer text;
do not dedupe across retry attempts, response IDs, or after response.failed, since #27185 looks like a separate retry/reconnect lifecycle issue that should remain diagnosable.

Regression coverage I would expect: cumulative same-id snapshots collapse to one persisted final message; distinct-id monotonic snapshots like this issue collapse to one final message; non-prefix assistant messages remain separate; a tool/reasoning boundary flushes any pending assistant snapshot; and a response.failed/retry fixture proves this does not mask terminal/retry behavior. I also would not claim this fixes mid-task stop cases such as #26860 unless a trace shows the same cumulative snapshot pattern there.

niceysam · 2 months ago

Additional repro — confirms the cumulative-snapshot mechanism, and it is much worse with CJK (Korean) output

Hitting the same bug on Codex CLI 0.142.3, model_provider = amazon-bedrock, model openai.gpt-5.5. Wanted to add a clean repro that I think directly confirms the root-cause hypothesis in this thread (and likely ties #29755 ↔ #26305 together).

Environment

Codex CLI: 0.142.3
Provider: amazon-bedrock (built-in), model openai.gpt-5.5
Surface: TUI
Output language: Korean (CJK)

Symptom
A single assistant turn is rendered/persisted as ~17 separate assistant messages, each one a longer superset of the previous (the earlier text is a strict prefix of the next). Visible to the user as the same answer printed over and over, each time slightly longer.

Evidence (from the session rollout .jsonl)
Within one sampling request I see the assistant text persisted repeatedly with monotonically increasing length — i.e. cumulative stream snapshots stored as distinct agent_message items instead of being coalesced:

assistant message char-length per persisted item (same turn, in order):
109 → 137 → 186 → 240 → 257 → 428 → 492 → 678 → 719 → 839 → 973 → 1299 → 1420 → 1518 → 1522 → 1604 → 1656

Each item's text is a strict prefix of the following item's text. This matches the maintainer's description in this thread exactly: completed Bedrock/Mantle stream snapshots are mapped 1:1 from response.output_item.done into OutputItemDone and never coalesced before history storage.

CJK angle (link to #26305)
This reproduces consistently with Korean output. An equivalent English-language turn does not blow up the same way, which lines up with #26305 ("streamed Chinese/CJK output is duplicated into history → runaway token growth"). Because every duplicated snapshot is fed back into history, prompt tokens grow turn-over-turn and can exceed the model maximum.

Why it matters

User-facing: the same answer is shown many times — unusable for non-English (Korean/CJK) workflows.
Token cost / context: duplicated cumulative snapshots inflate history and can push prompt tokens past the limit within a few turns.

Happy to share a redacted slice of the rollout .jsonl (lengths + item ids, no payload) if that helps reproduce. Glad to provide further traces.