Consider caching repeated image data URLs and speeding up large output truncation

What variant of Codex are you using?

CLI

What feature would you like to see?

I would like Codex to reduce local processing overhead in two hot paths that can show up in image-heavy or output-heavy sessions:

1. Cache repeated prompt image data URLs.
2. Avoid unnecessary full-string scans during middle truncation of large outputs.

These are independent changes. If the maintainers are interested, they could be reviewed or staged separately.

Why this would help

Repeated local image attachments and view_image outputs can already reuse processed image bytes, but they still rebuild the same base64 data:image/... URL on each prompt/output path. Caching the finished data URL by image content digest and image mode should reduce repeated CPU work and allocation pressure when the same screenshot or local image is replayed through context.

Large shell, MCP, and error outputs frequently pass through middle truncation before being displayed or sent back to the model. The current truncation path can scan character data more than needed, and ASCII logs pay per-character splitting costs even though byte boundaries are valid character boundaries for ASCII.

Prototype results

I have local prototype patches for both changes.

For repeated prompt image data URLs:

• Small repeated PNG: about 20.91 us with cached data URLs vs about 35.39 us for the old byte-cache-plus-base64 path.
• Large repeated PNG: about 71.57 us with cached data URLs vs about 171.9 us for the old byte-cache-plus-base64 path.

For large output truncation:

• ASCII bytes, 1 MiB to 8 KiB: about 670 us to 9.606 us.
• ASCII token budget, 1 MiB to 2k tokens: about 506.2 us to 8.498 us.
• UTF-8 bytes, 512 KiB to 8 KiB: about 275.5 us to 244.1 us.
• UTF-8 token budget, 512 KiB to 2k tokens: about 272.5 us to 256.2 us.

The image change primarily helps repeated image use. First-time image processing should be essentially unchanged except for a small extra cache lookup/insert. The truncation change primarily helps large ASCII output such as build logs, compiler output, and test logs.

Expected behavior and compatibility

The intended behavior is unchanged:

• Image prompt output should keep the same image_url, detail mode, resize/original behavior, MIME handling, and invalid-image error behavior.
• Image cache keys are content-based, so updated file contents should not go stale.
• Truncation should preserve the same prefix/suffix shape, marker text, token approximation, and UTF-8 boundary handling.

Potential tradeoffs:

• Caching finished image data URLs adds another bounded in-memory LRU cache for hot image outputs.
• The image cache is effective only inside a Tokio runtime, matching the existing image byte cache behavior.
• Adding truncation benchmarks introduces a dev-only benchmark dependency.

Related context checked

I searched related PRs, issues, and discussions before prototyping. The closest areas I found were image-heavy history or compaction reports and general context/truncation discussions, but I did not find an existing issue that covered these exact local processing hot paths.

If this direction aligns with the Codex team's plans, I can share the exact patches or open an invited PR.

AI assistance

Drafted with Codex.