Skill Creator scripts fail to read UTF-8 SKILL.md files on Windows non-UTF-8 locales
What issue are you seeing?
The bundled Skill Creator scripts rely on the platform default text encoding when reading or writing UTF-8 skill files. On Windows systems using a non-UTF-8 locale, quick_validate.py can fail with UnicodeDecodeError when a valid UTF-8 SKILL.md contains characters such as smart quotes, an em dash, or CJK text.
Observed error:
Traceback (most recent call last):
File ".../skill-creator/scripts/quick_validate.py", line 99, in <module>
valid, message = validate_skill(sys.argv[1])
File ".../skill-creator/scripts/quick_validate.py", line 23, in validate_skill
content = skill_md.read_text()
File ".../pathlib.py", line 1028, in read_text
return f.read()
UnicodeDecodeError: 'gbk' codec can't decode byte 0x9d in position 2788: illegal multibyte sequence
The current implementation uses:
content = skill_md.read_text()
generate_openai_yaml.py has the same implicit-encoding read and also calls write_text() without an encoding, so it appears susceptible to the same locale-dependent behavior.
What steps can reproduce the bug?
Environment used:
OS: Windows NT 10.0.26200.0
System culture: zh-CN
Python: 3.13.9
locale.getencoding(): cp936
Codex CLI: 0.139.0
PyYAML: 6.0.3
- Create a valid skill whose
SKILL.mdis encoded as UTF-8 and contains a character not decodable under CP936/GBK, for example:
```markdown
---
name: utf8-validation-test
description: Validate a skill containing UTF-8 punctuation.
---
# UTF-8 validation test
Do not ask “should I continue?” during uninterrupted execution.
```
- On a Chinese Windows locale, run the bundled validator normally:
``powershell``
python path\to\skill-creator\scripts\quick_validate.py path\to\utf8-validation-test
- Observe
UnicodeDecodeErrorfromskill_md.read_text().
- Run the same command with Python UTF-8 mode enabled:
``powershell``
python -X utf8 path\to\skill-creator\scripts\quick_validate.py path\to\utf8-validation-test
The skill validates successfully.
What is the expected behavior?
Skill Creator scripts should read and write skill files consistently as UTF-8 on every supported platform, independent of the user's Windows locale.
A narrow fix would be to specify UTF-8 explicitly:
content = skill_md.read_text(encoding="utf-8")
and:
output_path.write_text(content, encoding="utf-8")
A regression test containing non-ASCII UTF-8 text would prevent the platform default encoding from being reintroduced.
Additional information
Current affected sources:
codex-rs/skills/src/assets/samples/skill-creator/scripts/quick_validate.pycodex-rs/skills/src/assets/samples/skill-creator/scripts/generate_openai_yaml.py
I searched open issues for quick_validate, skill creator encoding, GBK, and UnicodeDecodeError and did not find an issue covering this behavior. Issue #25548 also concerns quick_validate.py, but it addresses frontmatter guidance rather than file encoding.
I can prepare the small fix and regression coverage if a maintainer would like to invite a pull request under the repository's external contribution policy.
2 Comments
I have also run into this kind of Windows-only failure with agent/skill files that are valid UTF-8 but contain smart quotes, em dashes, or non-Latin text. The file is fine; the script fails because it falls back to the Windows locale codec.
The durable fix is to make every skill/template read and write explicit:
Using
PYTHONUTF8=1orchcp 65001can work as a local workaround, but I don’t think validation tools should depend on the user’s console/session encoding. Explicit UTF-8 in the scripts is safer and makes the same skill package behave consistently on Windows/macOS/Linux.One extra thing I’d check: any generator script that copies or rewrites
SKILL.md/YAML should also specifyencoding="utf-8"on both read and write, otherwise the validator may be fixed while the generator still reproduces the same locale-dependent bug.Confirmed this is still reproducible in a current bundled Skill Creator update, and this is the second time an update has restored the locale-dependent implementation after a local UTF-8 patch.
Current bundled
quick_validate.pySHA-256:5347a0a09cfb546bba1c0d1a30dae0a233d9a05f57bd4e7877155c588bcdabf7.On Windows with Python 3.11 using the default GBK/CP936 locale, a valid UTF-8
SKILL.mdcontaining CJK text fails atskill_md.read_text()withUnicodeDecodeError. NoPYTHONUTF8orPYTHONIOENCODINGoverride is involved. Replacing only that call withskill_md.read_text(encoding="utf-8")makes the same validation succeed under the default code page.This recurrence raises the priority of fixing the versioned upstream source and adding a non-ASCII regression fixture; otherwise each Codex update can silently reintroduce the defect.