`gitDiffToRemote` silently omits untracked files with non-ASCII filenames (quotepath C-quoted paths used verbatim)

What issue are you seeing?

codex app-server's gitDiffToRemote silently omits every untracked file whose name contains a non-ASCII character (or ", \, control chars) from the returned diff. There is no error and no warning — the diff simply arrives incomplete, so any client that attaches the local diff (e.g. IDE-extension / cloud-task flows) works from code that is missing those files entirely.

With git's default core.quotepath=true, this affects every filename containing accented or CJK characters (héllo.txt, 笔记.md, …), which is an everyday case for non-English-speaking users.

Reproduced on codex-cli 0.147.0 (npm), macOS arm64, git 2.50.1. The relevant code is unchanged at HEAD 646f7c0a91b8e327d263335da68ae8ef212895ce; the behavior is platform-independent (it comes from git's quoting, not the OS).

What steps can reproduce the bug?

Setup (offline, no sign-in needed):

mkdir -p /tmp/codex-diff-repro && cd /tmp/codex-diff-repro
git init --bare origin.git
git clone origin.git work && cd work
git commit --allow-empty -m init
git push origin HEAD
printf 'unicode-named file\n' > 'héllo.txt'
printf 'ascii-named file\n'   > normal.txt

End-to-end via the app-server:

(printf '{"method":"initialize","id":1,"params":{"clientInfo":{"name":"repro","title":"repro","version":"0.0.1"}}}\n'
 sleep 2
 printf '{"method":"initialized"}\n'
 sleep 1
 printf '{"method":"gitDiffToRemote","id":2,"params":{"cwd":"/tmp/codex-diff-repro/work"}}\n'
 sleep 5) | codex app-server

Observed response (0.147.0) — the diff contains only normal.txt; héllo.txt is gone:

{"id":2,"result":{"sha":"…","diff":"diff --git a/normal.txt b/normal.txt\nnew file mode 100644\nindex 0000000..dae9dac\n--- /dev/null\n+++ b/normal.txt\n@@ -0,0 +1 @@\n+ascii-named file\n"}}

Root cause is visible with plain git, replaying what the code does per file:

$ git ls-files --others --exclude-standard        # parsed line-by-line by info.rs
"h\303\251llo.txt"                                # C-quoted literal, quotes included
normal.txt

$ git diff --no-textconv --no-ext-diff --binary --no-index -- /dev/null '"h\303\251llo.txt"'
error: Could not access '"h\303\251llo.txt"'      # exit code 1, empty stdout

What is the expected behavior?

Untracked files appear in the gitDiffToRemote diff regardless of what characters their names contain — or, failing that, an error is surfaced instead of silently returning an incomplete diff.

Additional information

Root-cause analysis (paths/lines at HEAD 646f7c0):

1. codex-rs/git-utils/src/info.rs:723-737 — diff_against_sha collects untracked files with git ls-files --others --exclude-standard and parses stdout with .lines(). Neither -z nor a core.quotepath override is used (run_git_command_with_timeout_from at info.rs:410-427 only injects safe.bareRepository, core.hooksPath, and fsmonitor config).
2. With git's default core.quotepath=true, any filename byte > 0x7F — i.e. every non-ASCII name — plus ", \, and control characters is emitted as a C-quoted string including the surrounding double quotes (e.g. "h\303\251llo.txt").
3. info.rs:742-756 — that quoted literal is passed verbatim as a path to git diff --no-textconv --no-ext-diff --binary --no-index -- /dev/null <name>. No file exists under the literal quoted name, so git prints error: Could not access … to stderr and exits 1 with empty stdout.
4. info.rs:758-763 — exit code 1 must be accepted here (--no-index exits 1 when a diff exists), stdout is appended, and stderr is never inspected — so the failure is indistinguishable from "empty diff" and the file silently disappears.

Fix outline:

• Use git ls-files -z --others --exclude-standard and split on NUL. This is robust for all names — including ones containing ", \, or newlines, which core.quotepath=off alone would still quote.
• Optionally harden step 4: treat a --no-index run with empty stdout and non-empty stderr as a failure rather than appending nothing, so future errors in this path can't be silently swallowed.

The existing tests only cover ASCII untracked names (core/src/git_info_tests.rs), which is why this never tripped CI. Happy to provide more detail if useful.