Marketplace upgrade/add leaks uncleaned `.staging/` and `marketplace-backup-*` dirs in `$CODEX_HOME/.tmp/marketplaces` (unbounded disk growth)

Open 💬 2 comments Opened Jul 10, 2026 by soichisumi
💡 Likely answer: A maintainer (github-actions[bot], contributor) responded on this thread — see the highlighted reply below.

Marketplace upgrade/add leaks uncleaned .staging/ and marketplace-backup-* dirs in $CODEX_HOME/.tmp/marketplaces (unbounded disk growth)

Summary

Plugin-marketplace upgrade and add operations can leave scratch directories under $CODEX_HOME/.tmp/marketplaces/ that are never garbage-collected. Over time these grow without bound — in one environment they reached tens of GB (100+ leftover directories) and filled the disk.

Two categories leak:

  • $CODEX_HOME/.tmp/marketplaces/.staging/marketplace-upgrade-* and .../.staging/marketplace-add-* — per-operation git clones (each can be hundreds of MB to >1 GB).
  • $CODEX_HOME/.tmp/marketplaces/marketplace-backup-* — pre-upgrade rollback snapshots.

Environment

  • @openai/codex 0.144.0
  • Code references below are against main as of 2026-07-10 (line numbers may drift).

Root cause (from reading codex-rs/core-plugins/src/)

  • add path leaks even on ordinary failures. .staging/marketplace-add-* is created via tempfile and then TempDir::keep() (marketplace_add.rs:156-174), which disables the Drop-based cleanup. As a result any subsequent failure — clone, validation, destination move, or rollback — leaves the directory behind (not only abnormal process exit). The add-path git clone also has no timeout, so a stuck clone can be interrupted and stranded.
  • upgrade path leaks on abnormal exit. .staging/marketplace-upgrade-* is created every upgrade via tempfile::Builder::…prefix("marketplace-upgrade-").tempdir_in(&staging_parent) (marketplace_upgrade.rs:226-241) and relies on TempDir's Drop. If the process is killed/interrupted before the guard drops, the directory is left behind.
  • backups are intentionally retained on rollback failure. marketplace-backup-* is created in activate_marketplace_root() (marketplace_upgrade/activation.rs:76-129): the live dir is renamed into backup_dir/root as rollback insurance. On success the TempDir drops; but the double-failure paths intentionally keep the backup (it may be the only surviving copy of the live root), and abnormal exit also leaves it.
  • No sweep exists for either category. A repo-wide search for marketplace-backup matches only the creator. There is an age-based cleanup — remove_stale_curated_repo_temp_dirs (startup_sync.rs:402-485, 10-minute max age, called at :388) — but it targets a different temp category (.tmp/plugins-clone-*) and never touches .staging/* or marketplace-backup-*.

Why it accumulates in practice

Hosts that spawn codex frequently as short-lived subprocesses and terminate them (before the Rust TempDir Drop runs, where Drop is even still in effect) never get the implicit cleanup. Combined with the add path's keep() disabling Drop entirely, each interrupted or failed upgrade/add can strand a full clone, and nothing sweeps them later, so they pile up indefinitely.

Does external deletion trigger a re-download?

The upgrade skip-vs-clone decision (marketplace_upgrade.rs:202-282) decides to skip cloning based only on: (1) the live destination dir existing and validating, (2) last_revision from config.toml matching the remote HEAD, and (3) <destination>/.codex-marketplace-install.json matching. It never reads .staging/* or marketplace-backup-*. So removing an abandoned scratch/backup directory does not by itself cause a re-clone.

Two caveats make blind external deletion unsafe, though:

  • Deleting an in-flight .staging/* directory breaks the operation that owns it.
  • A marketplace-backup-* left behind by a rollback failure may be the only recoverable copy of the live root; deleting it before the live root is restored can lose that state.

Proposed fix

Primary (structural, in codex-rs/core-plugins):

  • Drop the add-path TempDir::keep() (marketplace_add.rs:172) and keep the TempDir guard alive through the rename/activation, so ordinary failures clean up automatically.
  • Serialize add / upgrade / cleanup for a given marketplace with a cross-process lock, and sweep .staging/* only while holding that lock (so a sweep can never race an in-flight operation — a directory's top-level mtime does not reliably prove inactivity).
  • For marketplace-backup-*, reconcile rather than blindly age-delete: restore or retain the backup when the live root is absent/invalid, and delete only once a healthy live root is established and no operation is active.

Interim external workaround (for affected users, until the above lands): a periodic job that deletes $CODEX_HOME/.tmp/marketplaces/.staging/marketplace-{upgrade,add}-* and .../marketplace-backup-*, filtering by name prefix so live marketplace dirs are never touched, and gating each candidate on an age threshold plus an open-file-handle check so an in-flight operation is skipped (since mtime alone is not sufficient). For git-backed marketplaces a deleted backup only costs a re-clone.

View original on GitHub ↗

2 Comments

github-actions[bot] contributor · 1 month ago

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

  • #30794
  • #30620

Powered by Codex Action

wey-gu · 1 month ago

Independent confirmation: one installation accumulated 8,448 marketplace-upgrade-* directories and more than 1 TiB over a month; another had 2,126 directories / about 98 GiB.

One additional contributor is that ConfiguredMarketplaceUpgradeState::in_flight is process-local, so separate app-server processes can clone concurrently. I prepared an upgrade-only patch that:

  • serializes configured upgrades with an OS-backed lock under CODEX_HOME/.tmp
  • sweeps only marketplace-upgrade-* entries older than ten minutes while holding that lock
  • re-reads on-disk config after lock acquisition so waiters skip stale snapshots
  • preserves recent, unrelated, backup, and symlink entries

Tested implementation: https://github.com/wey-gu/codex/commit/1bb3ae37f

just test -p codex-core-plugins: 319/319 passed with an isolated home, including a two-thread concurrent-upgrade test. just fix -p codex-core-plugins also passed. This patch intentionally does not delete marketplace-backup-*; those need the reconciliation behavior described in this issue.