asciicheck.py flags CRLF line endings as invalid characters (false positive on Windows checkouts)
Description
scripts/asciicheck.py (run in CI via .github/workflows/repo-checks.yml as ./scripts/asciicheck.py README.md) reads a file, decodes it as UTF-8, and flags every character outside 0x20-0x7E (plus a small allowlist) as an error:
for lineno, line in enumerate(text.splitlines(keepends=True), 1):
for colno, char in enumerate(line, 1):
codepoint = ord(char)
if char == "\n":
continue
if (
not (0x20 <= codepoint <= 0x7E)
and codepoint not in allowed_unicode_codepoints
):
errors.append((lineno, colno, char, codepoint))
Only \n is exempted — \r (U+000D) is not. If the file being checked has CRLF line endings (e.g. checked out on Windows with core.autocrlf=true, a common Git-for-Windows default), every single line is reported as containing an "invalid character". --fix doesn't help either: \r isn't in the substitutions map, so the rewritten file still contains \r and the script still exits non-zero.
The repo has no .gitattributes line forcing eol=lf for the files this script checks, so a Windows contributor who checks out the repo with the common core.autocrlf=true setting and runs this script locally against README.md gets a wall of false-positive errors, even though the intent of the script (per its own docstring) is to catch non-breaking spaces/smart quotes/dashes, not legitimate line endings.
Steps to reproduce
from pathlib import Path
p = Path("crlf_test.md")
p.write_bytes(b"hello world\r\nsecond line\r\n")
$ python scripts/asciicheck.py crlf_test.md
Invalid character at line 1, column 12: U+000D (\r)
Invalid character at line 2, column 12: U+000D (\r)
$ echo $?
1
Expected behavior
CRLF line endings should not be flagged as invalid characters.
Actual behavior
Every line of a CRLF file produces a spurious "Invalid character ... U+000D (\r)" error, and --fix cannot resolve it since \r isn't in the substitution table.
Environment
- Commit:
7c37479(main, 2026-08-27), Python 3.14, Windows 11 - File:
scripts/asciicheck.py:93-94 - No existing test file covers this script.
Suggested fix
- if char == "\n":
+ if char in ("\n", "\r"):
continue