apply_patch: seek_sequence ignores its documented EOF fallback, failing valid "*** End of File" hunks
What issue are you seeing?
codex-rs/apply-patch/src/seek_sequence.rs documents a two-stage search when
eof is true, but only implements the first stage — the code contradicts its own
doc comment.
The doc comment (lines 1–6) states:
> When eof is true, we first try starting at the end-of-file (so that patterns
> intended to match file endings are applied at the end), and fall back to
> searching from start if needed.
The implementation replaces the search origin instead of falling back to it:
``rustfor i in search_start..=lines.len()-pattern.len()
let search_start = if eof && lines.len() >= pattern.len() {
lines.len() - pattern.len() // only ever the tail offset
} else {
start
};
// every matching pass loops `
eof
When is true, all four matching passes (exact, rstrip, trim, unicode-normalize)start
only test the single end-of-file offset. There is no fallback to , so if theseek_sequence
pattern is not literally the final lines of the file, returns None`
even though the pattern exists earlier.
Impact: compute_replacements (lib.rs:767, 779) calls this with
eof = chunk.is_end_of_file, i.e. for every update hunk ending in *** End of File.
If such a hunk's old_lines are not exactly the file's last lines (trailing lines
follow the edited region, or a slight miscount), the whole patch fails with
Failed to find expected lines — instead of matching the region that does exist. The
documented leniency that would rescue it is simply absent, and no existing test
covers the eof-fallback path.
Observed on current main (commit b8b61bc69).
What steps can reproduce the bug?
seek_sequence is pub(crate), so the precise reproduction is a unit test added to
its tests module (in codex-rs/apply-patch/src/seek_sequence.rs). The pattern
exists at the start of the file but not at the end:
``rust``
#[test]
fn test_eof_falls_back_to_start_when_not_anchored_to_end() {
let lines = to_vec(&["alpha", "beta", "gamma", "delta"]);
let pattern = to_vec(&["alpha", "beta"]);
assert_eq!(
seek_sequence(&lines, &pattern, /*start*/ 0, /*eof*/ true),
Some(0)
);
}
Run:
````
cargo nextest run -p codex-apply-patch -E 'test(test_eof_falls_back_to_start)'
Actual result on unmodified main:
``left == right
thread '...test_eof_falls_back_to_start_when_not_anchored_to_end' panicked at
apply-patch/src/seek_sequence.rs:
assertion failed``
left: None
right: Some(0)
test result: FAILED. 0 passed; 1 failed
For comparison, the same lookup with eof = false returns Some(0) as expected, so
the failure is specific to the eof path.
What is the expected behavior?
Per the function's own documentation, an eof search should try the end-of-file
position first and then fall back to a full search from start. So the assertion
above should return Some(0), while a repeated pattern should still prefer its
end-of-file occurrence.
A minimal fix — fall back to a full search only after the eof-anchored passes fail,
which preserves EOF-preference:
// immediately before the final `None`:
if eof && search_start > start {
return seek_sequence(lines, pattern, start, /*eof*/ false);
}
```
With this change the full crate suite passes, including two added regression tests
(the fallback case above, plus an EOF-preference case asserting a repeated pattern
still matches the last occurrence under `eof = true`):
```
cargo nextest run -p codex-apply-patch
Summary 88 tests run: 88 passed, 0 skipped
```
### Additional information
Additional information
Root cause is a documentation/implementation mismatch introduced by the `eof`
anchoring optimization: it narrows the search to the tail offset without preserving
the documented fallback, and no test exercised the fallback path so the regression
went unnoticed.
I have a branch-ready fix plus the two regression tests prepared and verified locally
(fails before, 88/88 pass after). Happy to open a PR if the team would like — noting
docs/contributing.md makes external code contributions invitation-only, so I'm filing
this as a bug report first.
2 Comments
I dug into this and can confirm the bug on current
main, with a root cause and a minimal fix. Sharing the analysis here first per the contributing guidelines.Root cause
In
codex-rs/apply-patch/src/seek_sequence.rs, the doc comment promises a two-stage search wheneofis true (try the end-of-file offset, then fall back tostart), but the implementation only sets a single search origin and never falls back:When
eofis true,search_startis the tail offset, so the loop runs for exactly onei. All four decreasing-strictness passes (exact, rstrip, trim, unicode-normalize) test only that single end-of-file position. The documented fallback tostartis absent.compute_replacementscalls this witheof = chunk.is_end_of_filefor every hunk ending in*** End of File. If such a hunk'sold_linesare not literally the file's last lines, the whole patch fails withFailed to find expected lines, even though the region exists earlier.Reproduction
Unit level (the pattern is at the start but not the end):
End-to-end — a real
*** End of Filehunk whose edited region isn't the file tail (thirdtrails it):Against
first\nsecond\nthird\n, this fails today withFailed to find expected lines in tail.txt:\nfirst\nsecond.Proposed fix (outline)
Extract the four search passes into a
search_from(lines, pattern, search_start)helper, and for theeofcase try the anchored end-of-file match first, then fall back to a normal search fromstart:This makes the code match its documented behavior. The anchored tail match still wins whenever it exists, so there is no behavior change for hunks that already match at the file tail — the change only adds matches that previously failed. I traced every branch for panic-safety (the
eof_startsubtraction is guarded by the existingpattern.len() > lines.len()early return; the inclusive range is iterator-only, so asearch_startpast the end simply yields zero iterations).I have a branch with this fix plus tests (fallback-to-start, tail-preference-on-repeat, fuzzy fallback, absent-pattern, and the end-to-end regression test above — which fails before the change and passes after). Happy to open a PR if the team would like one.
Thanks for the thorough confirmation, @hassaanch23 — good to see it reproduces cleanly for you too, and your search_from helper extraction is exactly the right shape.
As the issue author I also have a branch-ready fix, and it converges on the same structure: pull the four decreasing-strictness passes into a search_from(lines, pattern, search_start) helper, then in the eof case try the anchored end-of-file position first and fall back to search_from(.., start) — so the tail match still wins whenever it exists (no behavior change for hunks that already match at the file end) and we only add the matches that previously failed.
I've verified it end to end on current main: the regression test for the non-anchored *** End of File case fails before the change and passes after, the tail-preference-on-repeat case is covered, and the full codex-apply-patch suite is green (88/88) with cargo fmt --check clean.
cc @md-oai (you added the unicode-normalisation pass in this file) — happy to open the PR whenever it fits the contribution policy; there are now two of us with a ready fix, so just let me know how you'd like to handle it.