Bug: Chinese UTF-8 characters garbled in handoff context serialization
Open 💬 0 comments Opened Jul 6, 2026 by cuikaipp
Bug: UTF-8 Chinese garbled in inter-conversation handoff summaries
Description
When a conversation containing Chinese text is summarized and handed off to a new conversation, the Chinese characters in the handoff summary become garbled (Mojibake).
Examples
| Original | Garbled |
|----------|--------|
| \u2705 (checkmark) | random CJK chars |
| Chinese text | Mojibake gibberish |
Analysis
All persistent files (memory.json, mcp.json, session_index.jsonl, SQLite databases) are correctly UTF-8 encoded. The corruption only appears in the in-memory handoff summary passed between agent sessions.
Root Cause (suspected)
Double-encoding in the IPC/serialization pipeline:
- UTF-8 encoded Chinese bytes being decoded as Latin-1 or Windows-1252 at a pipe/JSON boundary
- Possible missing
ensure_ascii=Falseinjson.dumps()during handoff serialization - Or pipe encoding between processes not set to UTF-8
Environment
- OS: Windows 11 (Chinese Simplified, code page 936)
- Codex: Codex++ desktop app / Codex CLI
- Python: 3.12
- Terminal: PowerShell 5.x
Steps to Reproduce
- Have a conversation with Chinese content
- Use handoff or context inheritance to start a new conversation
- Observe garbled Chinese characters in the context summary
Expected
Chinese characters preserved correctly in handoff summaries.
Actual
Chinese characters corrupted (UTF-8 bytes interpreted as Latin-1).