plugin-creator sample script crashes on non-ASCII marketplace.json under non-UTF-8 locale (missing encoding="utf-8")

Open 💬 0 comments Opened Aug 27, 2026 by xu-kai-quan

Description

codex-rs/skills/src/assets/samples/plugin-creator/scripts/create_basic_plugin.py opens files without specifying encoding="utf-8":

def load_json(path: Path) -> dict[str, Any]:
    with path.open() as handle:
        return json.load(handle)

(also the two path.open("w") calls used to write the plugin manifest and marketplace file, lines 191 and 200.)

Without an explicit encoding, Python (below the PEP 686 UTF-8-by-default cutover) uses locale.getpreferredencoding() for text I/O. On a non-UTF-8-locale Windows install (e.g. Chinese/Japanese/Korean Windows, where the system locale is commonly cp936/cp932/cp949), this is not UTF-8.

load_json() reads ~/.agents/plugins/marketplace.json, which is the file the real Rust product maintains (codex-rs/core-plugins/src/marketplace.rs:1079, via serde_json::to_string_pretty, which emits raw UTF-8 for non-ASCII strings, not \uXXXX-escaped). As soon as any plugin's displayName (or any other field) contains a non-ASCII character — an accented name, CJK text, an emoji — running this sample script on a non-UTF-8-locale Windows machine crashes with UnicodeDecodeError instead of working normally.

Steps to reproduce

import json, tempfile
from pathlib import Path
from create_basic_plugin import load_json

p = Path(tempfile.mkdtemp()) / "marketplace.json"
# Simulate a marketplace.json as written by the real Rust app (raw UTF-8, not \uXXXX-escaped)
p.write_text(
    json.dumps({"name": "personal", "interface": {"displayName": "café-tools ☕"}}, ensure_ascii=False),
    encoding="utf-8",
)
load_json(p)

On a machine whose locale encoding is cp936 (verified via locale.getencoding()):

UnicodeDecodeError: 'gbk' codec can't decode byte 0x95 in position 65: illegal multibyte sequence

Expected behavior

Reading/writing a UTF-8 JSON file (which per RFC 8259 is what all JSON is) should work regardless of the host's locale.

Actual behavior

UnicodeDecodeError on non-UTF-8-locale systems as soon as the JSON contains non-ASCII text.

Environment

  • Commit: 7c37479 (main, 2026-08-27), Python 3.14, Windows 11, locale cp936
  • File: codex-rs/skills/src/assets/samples/plugin-creator/scripts/create_basic_plugin.py:100,191,200
  • Note: codex-rs/skills/tests/test_plugin_creator.py exists and writes fixtures with encoding="utf-8", but always via json.dumps(payload) (default ensure_ascii=True), which produces pure-ASCII output and therefore never exercises this path — that's likely why the existing test suite hasn't caught it.

Suggested fix

 def load_json(path: Path) -> dict[str, Any]:
-    with path.open() as handle:
+    with path.open(encoding="utf-8") as handle:
         return json.load(handle)

and the same encoding="utf-8" addition to the two path.open("w") calls at lines 191 and 200.

View original on GitHub ↗