PowerShell encoding issue on Windows: Chinese characters display as garbled text when using stdio with codex-app-server

What version of Codex is running?

0.43.0-alpha.5

Which model were you using?

etc

What platform is your computer?

x64

What steps can reproduce the bug?

Context and Reproduction Flow
Communication Method: stdio-based interaction with codex-app-server.exe
Tool Invocation: LLM calls PowerShell tools through the application server
Command Example: pwsh.exe with commands like ls -R
Problem Manifestation: Chinese directory and file names display as garbled characters like "Ѹ������" instead of proper Chinese text

What is the expected behavior?

_No response_

What do you see instead?

_No response_

Additional information

GitHub Issue Template

Title: PowerShell encoding issue on Windows: Chinese characters display as garbled text when using stdio with codex-app-server

Description:

Problem

When using stdio communication with codex-app-server.exe, and the LLM invokes PowerShell tools (specifically pwsh.exe with commands like ls), Chinese characters in the command output appear as garbled text (mojibake). This occurs specifically in the interaction flow between the application server and PowerShell tool execution.

Context and Reproduction Flow

1. Communication Method: stdio-based interaction with codex-app-server.exe
2. Tool Invocation: LLM calls PowerShell tools through the application server
3. Command Example: pwsh.exe with commands like ls -R
4. Problem Manifestation: Chinese directory and file names display as garbled characters like "Ѹ" instead of proper Chinese text

Root Cause Analysis

The issue is located in codex-rs/core/src/shell.rs:

1. Hardcoded -NoProfile parameter: The PowerShell command construction in format_default_shell_invocation() function (lines 78-82) hardcodes the -NoProfile parameter.

1. Profile loading bypass: The -NoProfile parameter prevents PowerShell from loading user profiles that contain UTF-8 encoding configurations.

1. Encoding mismatch: Windows PowerShell defaults to GB2312 encoding (code page 936), while the application expects UTF-8. When users configure UTF-8 in their PowerShell profiles, the -NoProfile parameter bypasses these settings.

1. Configuration override ineffective: Even when shell_environment_policy.use_profile is set to true, the hardcoded -NoProfile parameter still takes precedence in the tool execution path.

Technical Details

• Affected Component: Tool execution pipeline in codex-app-server
• Communication Protocol: stdio between client and app-server
• Tool Type: PowerShell tool invocations
• Encoding Conflict: GB2312 (Windows default) vs UTF-8 (application expectation)

Expected Behavior

PowerShell tool outputs should display Chinese characters correctly when:

• Users have configured UTF-8 encoding in their PowerShell profiles
• The tool is invoked through stdio communication with codex-app-server
• The use_profile configuration is set to true

Actual Behavior

Chinese characters in PowerShell command outputs appear as garbled text in the application interface.

Steps to Reproduce

1. Configure PowerShell to use UTF-8 encoding in user profile
2. Set shell_environment_policy.use_profile = true in config.toml
3. Start codex-app-server with stdio communication
4. Use LLM to invoke a PowerShell command that outputs Chinese text (e.g., ls in a directory with Chinese names)
5. Observe garbled characters in the command output

Proposed Solutions

Option 1 (Recommended): Respect the use_profile configuration in tool execution

• When shell_environment_policy.use_profile = true, remove the -NoProfile parameter from PowerShell invocations
• Allow PowerShell to load user profiles and encoding settings during tool execution

Option 2: Add tool-specific encoding configuration

• Add encoding configuration options specifically for PowerShell tool execution
• Allow explicit UTF-8 encoding specification for stdio-based tool interactions

Option 3: Implement encoding detection and conversion

• Detect the encoding of PowerShell outputs and convert to UTF-8
• Use libraries like encoding_rs for robust encoding handling

Environment

• OS: Windows 11 Enterprise
• PowerShell: PowerShell 7 (pwsh.exe)
• Codex Version: 0.43.0-alpha.5
• Communication: stdio with codex-app-server.exe
• Default Encoding: GB2312 (code page 936)
• Desired Encoding: UTF-8 (code page 65001)

Code References

The problematic code is in the tool execution path:

• shell.rs line 78-82: PowerShell command formatting with hardcoded -NoProfile
• Tool execution pipeline that doesn't respect use_profile configuration for encoding

Additional Notes

This issue affects all Windows users who work with non-ASCII characters, particularly in Chinese, Japanese, Korean, and other non-Latin scripts. The problem is especially critical for users in regions where non-Latin scripts are commonly used.
Title: PowerShell encoding issue on Windows: Chinese characters display as garbled text when using stdio with codex-app-server

Description:

Problem

When using stdio communication with codex-app-server.exe, and the LLM invokes PowerShell tools (specifically pwsh.exe with commands like ls), Chinese characters in the command output appear as garbled text (mojibake). This occurs specifically in the interaction flow between the application server and PowerShell tool execution.

Context and Reproduction Flow

1. Communication Method: stdio-based interaction with codex-app-server.exe
2. Tool Invocation: LLM calls PowerShell tools through the application server
3. Command Example: pwsh.exe with commands like ls -R
4. Problem Manifestation: Chinese directory and file names display as garbled characters like "Ѹ" instead of proper Chinese text

Root Cause Analysis

The issue is located in codex-rs/core/src/shell.rs:

1. Hardcoded -NoProfile parameter: The PowerShell command construction in format_default_shell_invocation() function (lines 78-82) hardcodes the -NoProfile parameter.

1. Profile loading bypass: The -NoProfile parameter prevents PowerShell from loading user profiles that contain UTF-8 encoding configurations.

1. Encoding mismatch: Windows PowerShell defaults to GB2312 encoding (code page 936), while the application expects UTF-8. When users configure UTF-8 in their PowerShell profiles, the -NoProfile parameter bypasses these settings.

1. Configuration override ineffective: Even when shell_environment_policy.use_profile is set to true, the hardcoded -NoProfile parameter still takes precedence in the tool execution path.

Technical Details

• Affected Component: Tool execution pipeline in codex-app-server
• Communication Protocol: stdio between client and app-server
• Tool Type: PowerShell tool invocations
• Encoding Conflict: GB2312 (Windows default) vs UTF-8 (application expectation)

Expected Behavior

PowerShell tool outputs should display Chinese characters correctly when:

• Users have configured UTF-8 encoding in their PowerShell profiles
• The tool is invoked through stdio communication with codex-app-server
• The use_profile configuration is set to true

Actual Behavior

Chinese characters in PowerShell command outputs appear as garbled text in the application interface.

Steps to Reproduce

1. Configure PowerShell to use UTF-8 encoding in user profile
2. Set shell_environment_policy.use_profile = true in config.toml
3. Start codex-app-server with stdio communication
4. Use LLM to invoke a PowerShell command that outputs Chinese text (e.g., ls in a directory with Chinese names)
5. Observe garbled characters in the command output

Proposed Solutions

Option 1 (Recommended): Respect the use_profile configuration in tool execution

• When shell_environment_policy.use_profile = true, remove the -NoProfile parameter from PowerShell invocations
• Allow PowerShell to load user profiles and encoding settings during tool execution

Option 2: Add tool-specific encoding configuration

• Add encoding configuration options specifically for PowerShell tool execution
• Allow explicit UTF-8 encoding specification for stdio-based tool interactions

Option 3: Implement encoding detection and conversion

• Detect the encoding of PowerShell outputs and convert to UTF-8
• Use libraries like encoding_rs for robust encoding handling

Environment

• OS: Windows 11 Enterprise
• PowerShell: PowerShell 7 (pwsh.exe)
• Codex Version: 0.43.0-alpha.5
• Communication: stdio with codex-app-server.exe
• Default Encoding: GB2312 (code page 936)
• Desired Encoding: UTF-8 (code page 65001)

Code References

The problematic code is in the tool execution path:

• shell.rs line 78-82: PowerShell command formatting with hardcoded -NoProfile
• Tool execution pipeline that doesn't respect use_profile configuration for encoding

Additional Notes

This issue affects all Windows users who work with non-ASCII characters, particularly in Chinese, Japanese, Korean, and other non-Latin scripts. The problem is especially critical for users in regions where non-Latin scripts are commonly used.