›_ Codex Issues
Codex Issues / #27784

apply_patch fails on UTF-8 BOM files

Open 💬 2 comments Opened Jun 12, 2026 by wangyauneli

Description
The \^Gpply_patch\ tool fails to match context when the target file has a UTF-8 BOM (bytes \EF BB BF\). It returns \Failed to find context\ without indicating the BOM is the cause, which is misleading and causes the agent to retry endlessly.

Steps to reproduce

  1. Create a JS file with UTF-8 BOM encoding (e.g. saved by Windows Notepad or WeChat DevTools)
  2. Ask the agent to modify a line in that file using \^Gpply_patch\
  3. The tool reports \Failed to find context\ even though the context lines are present in the file

Expected behavior
The tool should detect and strip BOM before matching context, then preserve the original encoding when writing back.

Impact
Any .js / .json / other files saved with BOM by Windows editors cannot be patched, causing the agent to waste many retry attempts without understanding the root cause. This is a common scenario on Windows since Notepad adds BOM by default.

View original on GitHub ↗

2 Comments

wgu9 · 2 months ago

I reproduced this against current main with the standalone apply_patch binary.

Minimal repro:

mkdir -p /tmp/codex-bom-repro
printf '\357\273\277const value = 1;\n' > /tmp/codex-bom-repro/bom.js
cargo run -q -p codex-apply-patch --bin apply_patch -- '*** Begin Patch
*** Update File: /tmp/codex-bom-repro/bom.js
@@
-const value = 1;
+const value = 2;
*** End Patch
'

Actual result:

Failed to find expected lines in /tmp/codex-bom-repro/bom.js:
const value = 1;

The likely owner is codex-rs/apply-patch/src/lib.rs::derive_new_contents_from_chunks.

Current behavior:

  • fs.read_file_text(...) returns the file as text, including the leading UTF-8 BOM as \u{FEFF}.
  • derive_new_contents_from_chunks immediately splits it with original_contents.split('\n').
  • Therefore the first logical line is "\u{FEFF}const value = 1;", not "const value = 1;".
  • compute_replacements(...) / seek_sequence(...) then cannot match ordinary patch context for the first line and reports a missing context line.

Suggested fix shape:

  • Detect a leading UTF-8 BOM after reading the file text.
  • Strip it only for patch matching/replacement.
  • Preserve the BOM when writing the updated file back if the original file had one.

Suggested regression:

  • Add an apply-patch test with a BOM-prefixed file where the first line is updated using normal patch context.
  • Assert the patch succeeds, the first line is changed, and the output file still begins with the UTF-8 BOM bytes.

Per docs/contributing.md, I am not opening a PR unless invited, but this should be a small focused patch in the apply-patch crate.

MovGP0 · 2 months ago

I provided a patch file with BOM detection in a patch file in this comment:

https://github.com/openai/codex/issues/13918#issuecomment-4808867609

The same mechanism can be used for detecting the BOM before patching. Can't provide the fix myself, since contributions are limited to OAI employees.