Bug: Chinese UTF-8 characters garbled in handoff context serialization

Open 💬 0 comments Opened Jul 6, 2026 by cuikaipp

Bug: UTF-8 Chinese garbled in inter-conversation handoff summaries

Description

When a conversation containing Chinese text is summarized and handed off to a new conversation, the Chinese characters in the handoff summary become garbled (Mojibake).

Examples

| Original | Garbled |
|----------|--------|
| \u2705 (checkmark) | random CJK chars |
| Chinese text | Mojibake gibberish |

Analysis

All persistent files (memory.json, mcp.json, session_index.jsonl, SQLite databases) are correctly UTF-8 encoded. The corruption only appears in the in-memory handoff summary passed between agent sessions.

Root Cause (suspected)

Double-encoding in the IPC/serialization pipeline:

  • UTF-8 encoded Chinese bytes being decoded as Latin-1 or Windows-1252 at a pipe/JSON boundary
  • Possible missing ensure_ascii=False in json.dumps() during handoff serialization
  • Or pipe encoding between processes not set to UTF-8

Environment

  • OS: Windows 11 (Chinese Simplified, code page 936)
  • Codex: Codex++ desktop app / Codex CLI
  • Python: 3.12
  • Terminal: PowerShell 5.x

Steps to Reproduce

  1. Have a conversation with Chinese content
  2. Use handoff or context inheritance to start a new conversation
  3. Observe garbled Chinese characters in the context summary

Expected

Chinese characters preserved correctly in handoff summaries.

Actual

Chinese characters corrupted (UTF-8 bytes interpreted as Latin-1).

View original on GitHub ↗