Thread metadata updates and unarchive can bypass the per-thread writer lock

Open 💬 2 comments Opened Aug 4, 2026 by luvs01

What version of Codex CLI is running?

openai/codex main at 9873cba8ce6d14e650e12cdc0dddd159ae6613d7 (source-level codex-thread-store regression tests).

What subscription do you have?

ChatGPT Pro. This is local thread-store coordination and is not plan-dependent.

Which model were you using?

Not model-dependent.

What platform is your computer?

Windows 11 x64. The affected thread-store code is cross-platform; Windows makes the rename/handle ordering especially important.

What terminal emulator and version are you using (if applicable)?

PowerShell 7. The reproducer calls the local thread-store API directly.

Codex doctor report

{
  "status": "not_applicable",
  "reason": "source-level thread-store lock regression; no auth, provider, MCP, or network dependency"
}

What issue are you seeing?

#36389 added per-thread cross-process writer ownership for create/resume and for archive/delete, but two mutation paths still bypass that ownership boundary:

  1. LocalThreadStore::update_thread_metadata can update SQLite/name indexes and append compatibility SessionMeta records without acquiring the per-thread OS writer lock. A second process can therefore rename a thread, change memory/Git metadata, or append to the same legacy JSONL while the owning process has a live recorder.
  1. LocalThreadStore::unarchive_thread acquires only its process-local lifecycle lock. Another process can have the archived rollout resumed and open for append while this process renames that file back into sessions/. On Unix the original writer can continue appending to the old file descriptor after the canonical path moved; on Windows the rename may fail late. Either outcome violates the intended fail-closed single-writer invariant.

Both paths can reach durable mutation before checking whether another process owns the thread.

What steps can reproduce the bug?

Metadata update

  1. Create a legacy or paginated rollout for thread T.
  2. In store/process A, acquire or retain T's per-thread writer ownership.
  3. In an independent store/process B using the same CODEX_HOME, call update_thread_metadata for T (for example, set name, memory_mode, or Git metadata).
  4. Current main does not acquire WriterLockCoordinator in this path, so the update is not rejected as a writer conflict.

A focused regression can hold A's writer guard, call the metadata update through B, and assert ThreadStoreError::Conflict plus byte-for-byte unchanged rollout content.

Unarchive

  1. Create an archived legacy or paginated rollout for thread T.
  2. In store/process A, resume or otherwise hold T's per-thread writer ownership.
  3. In independent store/process B, call unarchive_thread(T).
  4. Current main takes only B's process-local lifecycle lock, then proceeds toward the rename without checking A's OS writer lock.

A focused regression can assert Conflict, the archived path and content remain unchanged, and unarchive succeeds after A releases ownership.

What is the expected behavior?

Every mutation of a thread's canonical JSONL path or compatibility metadata should share the same ownership boundary introduced by #36389.

  • Metadata updates should acquire the process-local writer mutex and the cross-process writer lock before any SQLite, JSONL, or name-index mutation.
  • A same-process live recorder should reuse its existing OS ownership without recursively locking the Tokio writer mutex.
  • Unarchive should use the established order: lifecycle lock → process-local writer lock → reject a local live recorder → cross-process writer lock.
  • A competing owner should produce ThreadStoreError::Conflict before any partial mutation.

Additional information

I searched current issues, PRs, and the latest main; I did not find an exact report or newer fix for these two paths.

I have a focused local patch that:

  • adds an explicit “persist while writer guard is already held” helper, avoiding recursive mutex acquisition;
  • guards the whole metadata transaction before its first mutation;
  • guards unarchive through path lookup, rename, timestamp update, SQLite update, and readback;
  • adds legacy and paginated conflict regressions for both paths.

The new regressions pass, as do the existing live metadata, unarchive, and cross-process create/resume tests. Scoped Clippy and formatting checks pass. The change is 131 additions and 2 deletions across three codex-thread-store files, with no protocol or data migration.

This is a narrow follow-up to #36389 and does not claim to solve the separate optional rollout-compression ownership boundary. Per the invitation-only contribution policy, I can submit the validated patch as a PR if a maintainer confirms that this follow-up is wanted.

View original on GitHub ↗

2 Comments

ded-furby · 24 days ago

I reproduced this from code review on main and agree with the lock ordering diagnosis.

In update_thread_metadata (codex-rs/thread-store/src/local/update_thread_metadata.rs), metadata writes happen before any writer lock is held: the SQLite patch path and compatibility reconciliation can mutate rollout path/title/memory-mode while another process has an active live recorder.

In unarchive_thread (codex-rs/thread-store/src/local/unarchive_thread.rs), we currently take lock_lifecycle(thread_id) only, then rename the archived file and update sqlite. That allows a race with a concurrently-held live writer before rename/metadata update.

Suggested sequence (in line with 36389 intent):
1) lifecycle lock
2) process-local writer lock
3) fail fast if live recorder exists
4) cross-process writer lock (reuse existing writer helper if available)
5) perform path lookup + rename + sqlite updates atomically after lock acquisition

A regression test for each operation (active recorder from owner store should return ThreadStoreError::Conflict with no partial path changes) would lock this down.

If this is still welcome for review, I can prepare a scoped PR with just those ordering changes and tests in update_thread_metadata + unarchive_thread.

ebergin-petm · 7 days ago

I found a related Windows Desktop behavior that may be distinct from the writer-lock issue described here.

When opening an older Codex thread or switching back into its context, the local threads.rollout_path value is rewritten from a normal path such as C:\...\sessions\...jsonl to an extended path beginning with \\?\C:\..., even when no new message is sent. The rollout file still exists.

Newly created GPT Agent threads do not reproduce this. After the legacy path is normalized, the normal archive operation succeeds without restarting Codex; reopening the thread reintroduces the prefix.

Before normalization, archive fails with The system cannot find the file specified (os error 2).

This may be a context rehydration/path-persistence issue related to the archive/unarchive behavior here. Should it remain under this issue or be tracked separately?