False Turkish encoding corruption warnings during code review on Windows PowerShell
What version of the IDE extension are you using?
26.304.20706
What subscription do you have?
Plus
Which IDE are you using?
VS Code
What platform is your computer?
Microsoft Windows NT 10.0.26200.0 x64
What issue are you seeing?
On Windows PowerShell, Codex sometimes tells me Turkish characters are broken (for example İ, ş, ç) even though the file is actually UTF-8 and looks correct in my IDE. It seems like Codex is reading terminal rendered text and treating that as real file corruption. I need Codex to validate encoding more reliably before warning about text issues. Right now, I get incorrect encoding warnings while reviewing or editing .tsx files that contain Turkish text, and this leads to confusion and extra work.
What steps can reproduce the bug?
You can reproduce this by opening a .tsx file that contains Turkish UTF-8 text, then reading that file through Windows PowerShell output. In some cases, PowerShell shows mojibake characters (for example İ, ş, ç) and Codex treats that terminal rendering as if the source file itself is corrupted even though the file is valid UTF-8 and displays correctly in the IDE.
Example source:
example.tsx:
const label = "Hızlı Geçiş Sistemi";
Then run this in powershell:
Get-Content .\src\app\routes\pages\example.tsx
Codex may read the console output and incorrectly report Turkish character corruption as a real file issue.
What is the expected behavior?
Codex should distinguish between terminal display encoding issues and actual file encoding issues. If the source file is valid UTF-8 and renders correctly in the IDE Codex should not report character corruption. Before flagging text corruption Codex should verify file content using a reliable validation instead of relying only on PowerShell rendered output.
Additional information
Shortly: IDE text rendering is correct but the issue appears in terminal rendered output interpretation.
7 Comments
Potential duplicates detected. Please review them and close your issue if it is a duplicate.
Powered by Codex Action
Arabic encording corruption as well
Use PowerShell 7.5.4.
@ok-oai any one from openai its urgent
Thanks, I tested this and can confirm that on PowerShell 7.5.5, the same file is displayed correctly without any issues.
So switching to PowerShell 7 does resolve the problem.
However, I believe this still represents a usability issue. Most Windows users are still on PowerShell 5.1 by default and may not be aware of encoding differences or the need to upgrade.
I investigated this while reproducing #15422 / #13755 on Windows PowerShell 5.1.
Root-cause hypothesis: the shell-command wrapper currently pins
[Console]::OutputEncoding, but Windows PowerShell can still decode BOM-less UTF-8 Markdown through the active ANSI code page unless the surrounding PowerShell process is consistently forced to UTF-8. In practice, commands likeGet-Content/Select-Stringcan return mojibake for Turkish characters, which then makes generated patches mismatch the file bytes.I prepared a focused branch with a potential fix here:
https://github.com/evrenonur/codex/tree/codex/powershell-utf8-text-io
Summary of the approach:
chcp.com 65001in the Windows PowerShell startup prefix[Console]::InputEncoding,[Console]::OutputEncoding, and$OutputEncodingto UTF-8 without BOMGet-ContentandSelect-Stringto UTF-8 for the tool-call shell processValidation I ran locally:
cargo fmt --package codex-shell-commandcargo test -p codex-shell-command powershell --libI noticed the current contributing guide says external PRs are invitation-only, so I am leaving this as analysis and a concrete patch outline in the issue thread rather than relying on an unsolicited PR.
I'm using PowerShell 7.5.4, and while it fixed the mojibake issue for me for some time, I've restarted encountering the same problem recently. Codex sometimes doesn't preserve Turkish characters correctly when generating or editing text, even though the same file already contains correctly written Turkish text.
When I read the file with PowerShell 7.5.4, the existing Turkish text is displayed correctly, so this no longer seems to be a PowerShell rendering issue. However, Codex still generates new text with incorrect Turkish characters. With PowerShell 5.x, at least the existing text was displayed incorrectly.
It seems to ignore the existing context unless the prompt explicitly tells it to preserve the Turkish alphabet.