Skill Creator scripts fail to read UTF-8 SKILL.md files on Windows non-UTF-8 locales

Open 💬 2 comments Opened Jul 14, 2026 by WilliamClifton-dev

What issue are you seeing?

The bundled Skill Creator scripts rely on the platform default text encoding when reading or writing UTF-8 skill files. On Windows systems using a non-UTF-8 locale, quick_validate.py can fail with UnicodeDecodeError when a valid UTF-8 SKILL.md contains characters such as smart quotes, an em dash, or CJK text.

Observed error:

Traceback (most recent call last):
  File ".../skill-creator/scripts/quick_validate.py", line 99, in <module>
    valid, message = validate_skill(sys.argv[1])
  File ".../skill-creator/scripts/quick_validate.py", line 23, in validate_skill
    content = skill_md.read_text()
  File ".../pathlib.py", line 1028, in read_text
    return f.read()
UnicodeDecodeError: 'gbk' codec can't decode byte 0x9d in position 2788: illegal multibyte sequence

The current implementation uses:

content = skill_md.read_text()

generate_openai_yaml.py has the same implicit-encoding read and also calls write_text() without an encoding, so it appears susceptible to the same locale-dependent behavior.

What steps can reproduce the bug?

Environment used:

OS: Windows NT 10.0.26200.0
System culture: zh-CN
Python: 3.13.9
locale.getencoding(): cp936
Codex CLI: 0.139.0
PyYAML: 6.0.3
  1. Create a valid skill whose SKILL.md is encoded as UTF-8 and contains a character not decodable under CP936/GBK, for example:

```markdown
---
name: utf8-validation-test
description: Validate a skill containing UTF-8 punctuation.
---

# UTF-8 validation test

Do not ask “should I continue?” during uninterrupted execution.
```

  1. On a Chinese Windows locale, run the bundled validator normally:

``powershell
python path\to\skill-creator\scripts\quick_validate.py path\to\utf8-validation-test
``

  1. Observe UnicodeDecodeError from skill_md.read_text().
  1. Run the same command with Python UTF-8 mode enabled:

``powershell
python -X utf8 path\to\skill-creator\scripts\quick_validate.py path\to\utf8-validation-test
``

The skill validates successfully.

What is the expected behavior?

Skill Creator scripts should read and write skill files consistently as UTF-8 on every supported platform, independent of the user's Windows locale.

A narrow fix would be to specify UTF-8 explicitly:

content = skill_md.read_text(encoding="utf-8")

and:

output_path.write_text(content, encoding="utf-8")

A regression test containing non-ASCII UTF-8 text would prevent the platform default encoding from being reintroduced.

Additional information

Current affected sources:

  • codex-rs/skills/src/assets/samples/skill-creator/scripts/quick_validate.py
  • codex-rs/skills/src/assets/samples/skill-creator/scripts/generate_openai_yaml.py

I searched open issues for quick_validate, skill creator encoding, GBK, and UnicodeDecodeError and did not find an issue covering this behavior. Issue #25548 also concerns quick_validate.py, but it addresses frontmatter guidance rather than file encoding.

I can prepare the small fix and regression coverage if a maintainer would like to invite a pull request under the repository's external contribution policy.

View original on GitHub ↗

2 Comments

PsYcGoD · 1 month ago

I have also run into this kind of Windows-only failure with agent/skill files that are valid UTF-8 but contain smart quotes, em dashes, or non-Latin text. The file is fine; the script fails because it falls back to the Windows locale codec.

The durable fix is to make every skill/template read and write explicit:

content = skill_md.read_text(encoding="utf-8")
skill_md.write_text(content, encoding="utf-8", newline="\n")

Using PYTHONUTF8=1 or chcp 65001 can work as a local workaround, but I don’t think validation tools should depend on the user’s console/session encoding. Explicit UTF-8 in the scripts is safer and makes the same skill package behave consistently on Windows/macOS/Linux.

One extra thing I’d check: any generator script that copies or rewrites SKILL.md/YAML should also specify encoding="utf-8" on both read and write, otherwise the validator may be fixed while the generator still reproduces the same locale-dependent bug.

chmxyz · 13 days ago

Confirmed this is still reproducible in a current bundled Skill Creator update, and this is the second time an update has restored the locale-dependent implementation after a local UTF-8 patch.

Current bundled quick_validate.py SHA-256: 5347a0a09cfb546bba1c0d1a30dae0a233d9a05f57bd4e7877155c588bcdabf7.

On Windows with Python 3.11 using the default GBK/CP936 locale, a valid UTF-8 SKILL.md containing CJK text fails at skill_md.read_text() with UnicodeDecodeError. No PYTHONUTF8 or PYTHONIOENCODING override is involved. Replacing only that call with skill_md.read_text(encoding="utf-8") makes the same validation succeed under the default code page.

This recurrence raises the priority of fixing the versioned upstream source and adding a non-ASCII regression fixture; otherwise each Codex update can silently reintroduce the defect.