Windows: shell wrapper via `powershell.exe -Command` mojibakes UTF-8 files on non-English locales (PowerShell 5.1 `Get-Content` defaults to ANSI codepage)

I have a tested fix ready in my fork for this PowerShell UTF-8 mojibake issue.

Branch:

• https://github.com/gianlucafarias/codex/tree/fix/powershell-utf8-defaults

What it changes:

• Expands the PowerShell UTF-8 prefix to set:
• Console Input/OutputEncoding
• $OutputEncoding
• $PSDefaultParameterValues for Get-Content/Set-Content/Add-Content/Out-File

Validation run locally:

• cargo test -p codex-shell-command (136 passed)
• cargo test -p codex-shell-command powershell::tests:: (6 passed)
• cargo clippy -p codex-shell-command --tests -- -D warnings
• cargo fmt --all -- --check

Could a maintainer invite me to open a PR for this fix?
I can also attach a patch file if preferred.

Adding a few additional observations from a separate reproduction on Windows with GPT-5.5 in high effort mode. Filing on this issue rather than opening a new one, because the root cause appears to be the same as described in the body.

1. The bug does not only produce wrong answers — it also causes Codex to write corrupted content back to the file. When Codex reads a file with the broken encoding, it sometimes treats the garbled output as a real defect and overwrites the file with its own interpretation, effectively writing ASCII or garbled content over the original. This is the variant that causes actual data loss.

1. Frequency impression: on GPT-5.5 in high effort mode this has become a recurring issue across multiple sessions. I have the impression it became more frequent relatively recently — it was not happening at first — but I have not bisected a specific build, so I am presenting this as an observation rather than a regression claim.

1. Cross-referencing #13755 for visibility — it tracks the same underlying read-encoding root cause from the false-positive-warning angle and is worth linking here.

I can reproduce this on the current Codex Desktop build on Japanese Windows, and the attachment write path is not corrupting the file.

Environment

• Codex Desktop 26.707.3748.0
• bundled/session CLI 0.144.0-alpha.4
• Windows 10.0.26200 x64
• Windows PowerShell 5.1.26100.8655
• system locale ja-JP, active ANSI code page 932

Evidence

Codex Desktop created a long-paste attachment as pasted-text.txt. I am not publishing the original because it contains private infrastructure details, but I verified the file independently:

• 17,630 bytes
• no BOM
• strict UTF-8 decode succeeds
• decoding and re-encoding as UTF-8 reproduces the original bytes exactly
• no replacement characters or pre-existing mojibake markers

The rollout then invoked the equivalent of:

Get-Content -Raw -LiteralPath '<attachment>\pasted-text.txt'

with no -Encoding. Japanese text was mojibaked in the tool output. The controls are deterministic:

strict UTF-8 == Get-Content default        -> false
strict UTF-8 == Get-Content -Encoding UTF8 -> true

This isolates the failure to the Windows PowerShell 5.1 read boundary, not clipboard capture, attachment persistence, console rendering, or the model.

Current main at 9e552e9d15ba52bed7077d5357f3e18e330f8f38 still only sets [Console]::OutputEncoding; that cannot repair a string already decoded through CP932.

I prepared a current-main commit that preserves the existing best-effort console prefix and adds a separate best-effort default only for legacy PowerShell:

if ($PSVersionTable.PSVersion.Major -lt 6) {
    $PSDefaultParameterValues['Get-Content:Encoding'] = 'utf8'
}

The change intentionally does not set *:Encoding or write-cmdlet defaults, because Windows PowerShell 5.1 utf8 writes a BOM and would broaden behavior unnecessarily. An explicit user -Encoding still wins. The patch includes a Windows regression test that writes BOM-less Japanese UTF-8 and asserts an exact read through the prefixed powershell.exe command.

Behavioral verification passes on PS 5.1 / CP932, ConstrainedLanguage, and PowerShell 7.6.3. Rust formatting and diff checks pass. The full local Cargo test build is still pending because Windows Smart App Control blocks Cargo-generated unsigned build scripts on this machine (Code Integrity 3077), so CI validation would still be required.

This is the same fallback path discussed in #29085. Per the invitation-only contribution policy, I have not opened an unsolicited PR. If a maintainer would like this focused current-main patch, please invite the PR and I will push it.

Another locale confirmation (Hebrew, ACP 1255), plus two measurements that I think
change the shape of the proposed fix. Filing here rather than opening a new issue since
the root cause is the same one described in the body.

TL;DR: the existing [Console]::OutputEncoding prefix does not actually work, and
PowerShell 7 alone does not fix this. Both are load-bearing assumptions above.

1. The existing prefix is silently a no-op

The body says the current helper "fixes the stdout-write side". On my machine it does not,
because of how it is wrapped. This is the command Codex builds, quoted verbatim from its
own error output:

pwsh.exe -NoProfile -Command "try { [Console]::OutputEncoding=[System.Text.Encoding]::UTF8 } catch {}
Get-Content -Raw -LiteralPath .\sample.mjs"

The [Console]::OutputEncoding setter throws when no console is attached, which is the
case for a child spawned with piped stdio and windowsHide. The catch {} discards it and
the shell falls back to the inherited console codepage.

Verified by asking Codex to report its own state after that prefix has run:

Active code page: 862
[Console]::OutputEncoding.CodePage  ->  862
[Console]::InputEncoding.CodePage   ->  862
$PSVersionTable.PSVersion           ->  7.6.5

So the write side is unprotected too, not just the read side. Worth wrapping the fix in
something that surfaces failure rather than catch {} — this was invisible for months
precisely because that exception never reached anyone.

2. PowerShell 7 is not sufficient on its own

The body notes PS7 "defaults to UTF-8 across cmdlets", which implies moving to pwsh 7
resolves this. It does not. Controlled experiment, spawning each shell exactly the way
Codex does (Node spawn, piped stdio, windowsHide,
-NoLogo -NoProfile -NonInteractive -EncodedCommand), reading the same UTF-8 file:

| shell | console codepage | result |
|---|---|---|
| Windows PowerShell 5.1 | inherited (862) | corrupt |
| Windows PowerShell 5.1 | 65001 | corrupt — reproduces the exact bytes found in my rollout files |
| pwsh 7.6.5 | inherited (862) | corrupt (different signature: ✓ best-fits to √ via cp862) |
| pwsh 7.6.5 | 65001 | clean |

Row 2 is the useful one: it maps the lab back to the field. Row 3 is the one that matters
for the fix — pwsh 7 fixes the read default but still encodes its output to the
inherited console codepage, so $PSDefaultParameterValues alone will not close this. The
console codepage has to be set explicitly (SetConsoleOutputCP/SetConsoleCP at process
creation, or chcp 65001 as the first statement).

3. It also produces hard apply_patch failures, not only wrong answers

The body frames this as "confident wrong answer". On this machine it additionally breaks
editing outright, because the model builds patch context from the mangled text:

ERROR codex_core::tools::router: error=apply_patch verification failed:
Failed to find expected lines in <path>\<file>.mjs:
console.log('ג“ visual windows preserve overlapping narration context across cut boundaries');

The file contains ✓ there. ג + U+009C + “ is precisely the cp1255 decode of
E2 9C 93, stored in the rollout as c2 9c e2 80 9c.

Scale on one machine, across 349 rollout files: 155 apply_patch verification failed
occurrences, and 25,739 instances of a single signature (ג€, U+2014 through cp1255)
across 205 of 343 sessions.

Confirming what @claell reported in the second comment about write-back: Codex's own
~/.codex/config.toml accumulated corrupted Hebrew [projects.*] keys, so the corruption
reaches Codex's own config, not just user files.

4. New-ish failure mode: fabricated patch context when the shell is fully broken

While the shell was failing outright, Codex issued four consecutive speculative
apply_patch calls against a file it had never successfully read, inventing "pending",
"unverified", 'pending', "needs review" as the file's current contents. A failed read
degrading into invented patch context seems worth guarding independently of the encoding
fix.

Workaround, for anyone hitting this before the fix lands

Launch codex through a wrapper that sets the codepage first. It works because the console
is inherited down the whole chain (cmd -> codex.exe -> pwsh.exe):

@echo off
chcp 65001 >nul
"C:\path\to\codex.exe" %*

Verified end to end afterwards: shell read clean, apply_patch succeeded, and the bytes
written to disk were correct (e2 9c 93 and e2 80 94 preserved). Note this needs real
PowerShell 7 present as well — see below.

Caution for anyone installing PowerShell 7 to work around this

winget install --id Microsoft.PowerShell installs the MSIX/Store build (that manifest
is MSIX-only). Its app-execution alias in WindowsApps is a 0-byte reparse point, which
[windows] sandbox = "unelevated" cannot launch:

windows sandbox: CreateProcessAsUserW failed: 5 (Access is denied.)
| cmd=C:\Users\<user>\AppData\Local\Microsoft\WindowsApps\pwsh.exe -NoProfile -Command ...

Codex prefers that binary from PATH over Windows PowerShell 5.1, so installing PowerShell 7
the most obvious way takes the shell tool from "corrupts text" to "every command fails at
process creation". The MSI from GitHub releases installs a real executable and works. This
overlaps with #39276; might be worth skipping reparse-point binaries during shell discovery.

Environment: codex-cli 0.150.0-alpha.8 (also reproduced on 0.147.0-alpha.6.6 and
0.149.0-alpha.4.3), Windows 11 26200, he-IL, ACP 1255 / OEMCP 862.