apply_patch: seek_sequence ignores its documented EOF fallback, failing valid "*** End of File" hunks

Open 💬 2 comments Opened Jul 18, 2026 by SEPURI-SAI-KRISHNA

What issue are you seeing?

codex-rs/apply-patch/src/seek_sequence.rs documents a two-stage search when
eof is true, but only implements the first stage — the code contradicts its own
doc comment.

The doc comment (lines 1–6) states:

> When eof is true, we first try starting at the end-of-file (so that patterns
> intended to match file endings are applied at the end), and fall back to
> searching from start if needed.

The implementation replaces the search origin instead of falling back to it:

``rust
let search_start = if eof && lines.len() >= pattern.len() {
lines.len() - pattern.len() // only ever the tail offset
} else {
start
};
// every matching pass loops
for i in search_start..=lines.len()-pattern.len()
`
When
eof is true, all four matching passes (exact, rstrip, trim, unicode-normalize)
only test the single end-of-file offset. There is no fallback to
start, so if the
pattern is not literally the final lines of the file,
seek_sequence returns None`
even though the pattern exists earlier.

Impact: compute_replacements (lib.rs:767, 779) calls this with
eof = chunk.is_end_of_file, i.e. for every update hunk ending in *** End of File.
If such a hunk's old_lines are not exactly the file's last lines (trailing lines
follow the edited region, or a slight miscount), the whole patch fails with
Failed to find expected lines — instead of matching the region that does exist. The
documented leniency that would rescue it is simply absent, and no existing test
covers the eof-fallback path.

Observed on current main (commit b8b61bc69).

What steps can reproduce the bug?

seek_sequence is pub(crate), so the precise reproduction is a unit test added to
its tests module (in codex-rs/apply-patch/src/seek_sequence.rs). The pattern
exists at the start of the file but not at the end:

``rust
#[test]
fn test_eof_falls_back_to_start_when_not_anchored_to_end() {
let lines = to_vec(&["alpha", "beta", "gamma", "delta"]);
let pattern = to_vec(&["alpha", "beta"]);
assert_eq!(
seek_sequence(&lines, &pattern, /*start*/ 0, /*eof*/ true),
Some(0)
);
}
``
Run:

``
cargo nextest run -p codex-apply-patch -E 'test(test_eof_falls_back_to_start)'
``

Actual result on unmodified main:

``
thread '...test_eof_falls_back_to_start_when_not_anchored_to_end' panicked at
apply-patch/src/seek_sequence.rs:
assertion
left == right failed
left: None
right: Some(0)
test result: FAILED. 0 passed; 1 failed
``

For comparison, the same lookup with eof = false returns Some(0) as expected, so
the failure is specific to the eof path.

What is the expected behavior?

Per the function's own documentation, an eof search should try the end-of-file
position first and then fall back to a full search from start. So the assertion
above should return Some(0), while a repeated pattern should still prefer its
end-of-file occurrence.

A minimal fix — fall back to a full search only after the eof-anchored passes fail,
which preserves EOF-preference:

  // immediately before the final `None`:
  if eof && search_start > start {
      return seek_sequence(lines, pattern, start, /*eof*/ false);
  }
  ```

  With this change the full crate suite passes, including two added regression tests
  (the fallback case above, plus an EOF-preference case asserting a repeated pattern
  still matches the last occurrence under `eof = true`):

  ```
  cargo nextest run -p codex-apply-patch
  Summary  88 tests run: 88 passed, 0 skipped
  ```

### Additional information

 Additional information

  Root cause is a documentation/implementation mismatch introduced by the `eof`
  anchoring optimization: it narrows the search to the tail offset without preserving
  the documented fallback, and no test exercised the fallback path so the regression
  went unnoticed.

  I have a branch-ready fix plus the two regression tests prepared and verified locally
  (fails before, 88/88 pass after). Happy to open a PR if the team would like — noting
  docs/contributing.md makes external code contributions invitation-only, so I'm filing
  this as a bug report first.

View original on GitHub ↗

2 Comments

hassaanch23 · 1 month ago

I dug into this and can confirm the bug on current main, with a root cause and a minimal fix. Sharing the analysis here first per the contributing guidelines.

Root cause

In codex-rs/apply-patch/src/seek_sequence.rs, the doc comment promises a two-stage search when eof is true (try the end-of-file offset, then fall back to start), but the implementation only sets a single search origin and never falls back:

let search_start = if eof && lines.len() >= pattern.len() {
    lines.len() - pattern.len()   // tail offset only
} else {
    start
};
// all four passes loop `for i in search_start..=lines.len()-pattern.len()`

When eof is true, search_start is the tail offset, so the loop runs for exactly one i. All four decreasing-strictness passes (exact, rstrip, trim, unicode-normalize) test only that single end-of-file position. The documented fallback to start is absent.

compute_replacements calls this with eof = chunk.is_end_of_file for every hunk ending in *** End of File. If such a hunk's old_lines are not literally the file's last lines, the whole patch fails with Failed to find expected lines, even though the region exists earlier.

Reproduction

Unit level (the pattern is at the start but not the end):

let lines = to_vec(&["alpha", "beta", "gamma", "delta"]);
let pattern = to_vec(&["alpha", "beta"]);
// eof = false -> Some(0); eof = true -> None (bug; should be Some(0))
assert_eq!(seek_sequence(&lines, &pattern, 0, /*eof*/ true), Some(0));

End-to-end — a real *** End of File hunk whose edited region isn't the file tail (third trails it):

*** Begin Patch
*** Update File: tail.txt
@@
 first
-second
+second updated
*** End of File
*** End Patch

Against first\nsecond\nthird\n, this fails today with Failed to find expected lines in tail.txt:\nfirst\nsecond.

Proposed fix (outline)

Extract the four search passes into a search_from(lines, pattern, search_start) helper, and for the eof case try the anchored end-of-file match first, then fall back to a normal search from start:

if eof {
    let eof_start = lines.len() - pattern.len();
    if let Some(idx) = search_from(lines, pattern, eof_start) {
        return Some(idx);
    }
}
search_from(lines, pattern, start)

This makes the code match its documented behavior. The anchored tail match still wins whenever it exists, so there is no behavior change for hunks that already match at the file tail — the change only adds matches that previously failed. I traced every branch for panic-safety (the eof_start subtraction is guarded by the existing pattern.len() > lines.len() early return; the inclusive range is iterator-only, so a search_start past the end simply yields zero iterations).

I have a branch with this fix plus tests (fallback-to-start, tail-preference-on-repeat, fuzzy fallback, absent-pattern, and the end-to-end regression test above — which fails before the change and passes after). Happy to open a PR if the team would like one.

SEPURI-SAI-KRISHNA · 1 month ago

Thanks for the thorough confirmation, @hassaanch23 — good to see it reproduces cleanly for you too, and your search_from helper extraction is exactly the right shape.

As the issue author I also have a branch-ready fix, and it converges on the same structure: pull the four decreasing-strictness passes into a search_from(lines, pattern, search_start) helper, then in the eof case try the anchored end-of-file position first and fall back to search_from(.., start) — so the tail match still wins whenever it exists (no behavior change for hunks that already match at the file end) and we only add the matches that previously failed.

I've verified it end to end on current main: the regression test for the non-anchored *** End of File case fails before the change and passes after, the tail-preference-on-repeat case is covered, and the full codex-apply-patch suite is green (88/88) with cargo fmt --check clean.

cc @md-oai (you added the unicode-normalisation pass in this file) — happy to open the PR whenever it fits the contribution policy; there are now two of us with a ready fix, so just let me know how you'd like to handle it.