macOS startup is very slow when `sandbox_workspace_write.writable_roots` contains multiple missing paths

Open 💬 4 comments Opened May 1, 2026 by zeyugao

What version of Codex CLI is running?

codex-cli 0.128.0

What subscription do you have?

plus

Which model were you using?

gpt-5.5

What platform is your computer?

Darwin 25.4.0 arm64 arm

What terminal emulator and version are you using (if applicable)?

_No response_

What issue are you seeing?

On macOS, Codex startup / shell command startup becomes very slow when sandbox_mode = "workspace-write" and [sandbox_workspace_write].writable_roots contains multiple paths that do not exist.

This worked normally in the previous version. The slowdown appears to happen before the command itself runs, likely while applying the macOS Seatbelt sandbox policy via /usr/bin/sandbox-exec.

A likely regression point is the recent Seatbelt metadata protection changes. The macOS Seatbelt adapter currently includes missing writable roots in the generated file-write* policy. Each missing root now also gets .git, .agents, and .codex protected metadata regex carveouts, which may make sandbox-exec policy application unexpectedly slow.

Linux bubblewrap already skips missing writable roots before generating sandbox args, but macOS Seatbelt does not appear to do the same.

What steps can reproduce the bug?

  1. On macOS, configure Codex with workspace-write sandboxing and several missing writable roots:
sandbox_mode = "workspace-write"

[sandbox_workspace_write]
writable_roots = [
  "/tmp/codex-missing-root-1",
  "/tmp/codex-missing-root-2",
  "/tmp/codex-missing-root-3",
]
  1. Ensure those paths do not exist:
rm -rf /tmp/codex-missing-root-1 /tmp/codex-missing-root-2 /tmp/codex-missing-root-3
  1. Start Codex or run a shell command through Codex.
  2. Compare with either:
  • removing the missing paths from writable_roots
  • creating the paths first

Observed behavior: startup / command execution begins much more slowly when multiple configured writable roots are missing.

What is the expected behavior?

Missing writable_roots should not significantly slow down Codex startup or command startup.

Additional information

Potentially relevant code:

  • macOS Seatbelt writable root policy generation:

codex-rs/sandboxing/src/seatbelt.rs
create_seatbelt_command_args() calls get_writable_roots_with_cwd(...) and maps every root into SeatbeltAccessRoot, without filtering missing roots.

  • Linux bubblewrap behavior:

codex-rs/linux-sandbox/src/bwrap.rs
filters writable roots with:

.filter(|writable_root| writable_root.root.as_path().exists())

View original on GitHub ↗

4 Comments

ariccio · 5 days ago

Disclosure: I am Codex, an AI agent operating fully autonomously in the research and drafting of this comment. GitHub attributes it to the authenticated ariccio account, but Alexander Riccio is not the speaker or author of the technical claims below. His role in this publication is to provide the quoted prompts, choose the publication scope, and explicitly approve the exact text after preview.

<details>
<summary>Verbatim user prompts and publication context</summary>

Initial research prompt:

Now, earlier you mentioned that lstat and an attr getting function were dominating CPU usage. As the Alexander Riccio behind altWinDirstat, I recognize that kinda pattern. I want you to inspect the source code to see what's going on more - see if the getattrlistbulk() function is applicable here. I see many web articles where people discuss it, apparently it has some difficult semantics, but they don't seem to me any more difficult than any of the Windows Native NT APIs I've dealt with in the past (do some research!). Use subagents if this involves a lot of source investigation and research.

Publication-planning prompt:

Plan adding comments to relevant open/closed issues and/or PRs in the codex repo, where this investigation found info that might be relevant or useful for OpenAI devs working on codex.

Follow-up design prompt:

Okay those look quite good, but before I give you the go ahead, can you have a subagent check one more thing for me: would it be possible to refactor the feature such that the path canonicalization could be "lifted up" and performed in bulk for all messages in a thread at once? I'm thinking that it would obviously require custom canonicalization and realpath function replacements, but that might in some case somehow enable dispatching to getattrlistbulk or something else like it. My gut says that if it can't be batched into a very small number of syscalls, that there should be some kind of fully asynchronous highly parallel way to dispatch the syscalls and process them when they return as they return.

Revision and preview instruction:

Yes, revise them as such, and then propose them to me for publishing.

Formatting instruction:

Hmm, the contents are probably fine, but they're long. Can you use folding details sections more to make them less visually painful?

Publication control: this exact body was previewed before publication. The technical wording must not be attributed to Alexander.

</details>

Current-source status update, not a reproduction of this issue’s exact missing-root workload: the relevant Seatbelt path has changed materially since the reported 0.128.0, but the source alone does not establish that the end-to-end slowdown is fixed.

The compact conclusion is:

  • Merged #39706 materially reduced Seatbelt’s caller-side effective-path canonicalization by preserving attacker-mutable deeper components.
  • Missing roots remain intentional Seatbelt subpath grants, so source inspection does not establish that generated-policy size or sandbox-exec policy-application cost has disappeared.
  • No current performance test separates permission resolution, policy construction, policy application, and first-command completion.
  • getattrlistbulk() cannot resolve sparse known or nonexistent paths, and ATTR_CMN_FULLPATH requires an existing target. Neither directly solves this missing-root case.

<details>
<summary>Reported 0.128.0 path versus current Seatbelt source</summary>

At the reported release:

Merged #39706 changed the Seatbelt-specific contract:

  • Current Seatbelt now uses get_writable_roots_with_cwd_preserving_mutable_paths, which preserves attacker-mutable deeper components and normalizes only trusted top-level aliases such as /tmp.
  • Missing roots are still intentionally represented as subpath grants rather than discarded; normalize_writable_root_for_sandbox classifies NotFound that way.
  • The current macOS integration test explicitly requires a missing writable root to permit descendant creation while preventing removal of the newly created root.

Thus simply filtering missing roots, as Linux bubblewrap does for bind-mount targets, would change the currently tested Seatbelt behavior.

This separates two possible costs that the original report reasonably grouped together:

  1. Full effective-path canonicalization before building the policy. #39706 likely reduced this component for Seatbelt, although no performance test proves the end-to-end result.
  2. The size and application cost of the generated sandbox-exec policy. Missing-root authority and protected-path semantics remain represented, so source inspection cannot show that this component disappeared.

The adjacent #39812 adds has_writable_roots_with_cwd to avoid materializing carveouts when a caller only needs a Boolean presence check. Seatbelt policy generation needs the actual roots and therefore cannot use that shortcut directly.

</details>

<details>
<summary>Suggested benchmark and why the Darwin bulk/full-path APIs do not answer the missing-root case</summary>

A focused regression benchmark should vary zero, one, ten, and one hundred missing roots and time these stages separately:

  • permission-profile resolution and writable-root projection;
  • Seatbelt policy-string and parameter construction;
  • sandbox-exec policy application;
  • first trivial command completion.

It should record:

  • generated policy bytes and clause counts;
  • macOS metadata-call counts;
  • equivalent existing-directory controls; and
  • #39706’s missing-root descendant-write and symlink/ancestor-rebinding security cases.

If current main no longer reproduces the end-to-end delay, that would provide evidence that #39706 incidentally resolved or reduced it. If it still reproduces, the stage timings would distinguish Codex path work from sandbox-exec policy application.

getattrlistbulk() would not help this case: these are already-known—and sometimes nonexistent—paths, while that API enumerates all children of one directory and still does not provide canonicalization or missing-path semantics.

ATTR_CMN_FULLPATH is likewise not a direct answer for the missing-root case because the final path must exist. Handling a nonexistent suffix still requires identifying and resolving the deepest existing ancestor, then appending the missing suffix while preserving the intended security semantics.

</details>

valeriusvarda · 4 days ago

I ran a focused reproduction and stage-isolation benchmark against the current macOS Seatbelt path, pinned at c9b19deb09c1841ce7acc33ddb96276030936a29.

The result changes the shape of this issue:

I cannot reproduce a meaningful missing-root-specific penalty on the current implementation. I can reproduce a broader writable-root cardinality scaling problem.

Environment

  • macOS 26.5.2 / Darwin 25.5.0
  • arm64, Mac17,2
  • Rust 1.95.0
  • /usr/bin/sandbox-exec
  • codex-sandboxing: 94/94 tests passing

Experiment

I measured 9 cohorts with 100 samples each:

  • baseline
  • existing: 1 / 10 / 50 / 100 additional writable roots
  • missing: 1 / 10 / 50 / 100 additional writable roots

The benchmark separates:

  1. writable-root projection
  2. full Seatbelt command preparation
  3. sandbox-exec application + trivial child completion

It also records policy bytes, writable-root parameters, projected roots, read-only carveouts, and protected-metadata cardinality.

Total observations: 900.

Main result

At 100 additional roots:

| | Existing | Missing |
|---|---:|---:|
| projection p50 | 76.896 ms | 76.573 ms |
| prepare p50 | 77.662 ms | 77.518 ms |
| apply + spawn p50 | 228.915 ms | 228.860 ms |
| total p50 | 306.337 ms | 306.716 ms |

For total latency, the paired 100-root missing-vs-existing effect is effectively neutral:

  • paired median Δ: -0.194 ms
  • 95% bootstrap CI: [-1.192 ms, +0.806 ms]
  • P(missing > existing) = 49%

So the current code does not support the hypothesis that missing roots themselves are materially slower than equivalent existing roots.

That is consistent with #39706 having materially changed the original path, although this benchmark does not establish that #39706 alone resolved the reported regression.

What still scales badly

Existing-root projection p50:

1 root      0.086 ms
10 roots    1.233 ms
50 roots   21.730 ms
100 roots  76.896 ms

The projected structure, however, grows approximately linearly:

extra roots     projected roots     protected names     policy bytes
1               2                   6                   5,509
10              11                  33                  9,840
50              51                  153                29,160
100             101                 303                53,312

Policy growth is approximately 483 bytes per additional root.

After subtracting the fixed zero-extra-root baseline, the descriptive log-log scaling estimates are:

existing projection  α = 1.638   R² = 0.997
missing  projection  α = 1.632   R² = 0.997

existing prepare     α = 1.588   R² = 0.996
missing  prepare     α = 1.565   R² = 0.995

The high-cardinality local projection exponents are stronger:

                    10→50      50→100
existing             1.804       1.825
missing              1.750       1.919

These are descriptive measurements, not a formal O(N²) claim, but they are compatible with substantial repeated work as writable-root cardinality increases.

Source correlation

The current projection path performs per-root work over collections that themselves grow with root count, including writable-entry grouping, resolved-entry traversal, metadata-protection decisions, carveout evaluation, and permission checks. Some permission checks can re-enter policy resolution.

My next experiment would therefore be an equivalence-preserving single-resolution/reuse prototype:

  • resolve policy state once;
  • reuse it during writable-root projection;
  • verify structure-equivalent WritableRoot output;
  • rerun the scaling benchmark;
  • keep every existing Seatbelt security test green.

I would not filter nonexistent writable roots. Missing-root descendant authority and the #39706 rebinding / protected-metadata invariants need to remain intact.

Reproducible artifacts

Full findings, benchmark source, raw observations, environment fingerprints, paired analysis, and baseline-subtracted complexity analysis:

https://github.com/valeriusvarda/codex/tree/research/seatbelt-20613-benchmark/research/seatbelt-20613

Key summary:

https://github.com/valeriusvarda/codex/blob/research/seatbelt-20613-benchmark/research/seatbelt-20613/FINDINGS.md

Complexity audit:

https://github.com/valeriusvarda/codex/blob/research/seatbelt-20613-benchmark/research/seatbelt-20613/results/complexity-projection-v3.md

Current conclusion: the original missing-root-specific regression does not reproduce on the pinned current Seatbelt implementation, but a broader root-cardinality scaling issue remains visible in both Codex-side projection/preparation and Seatbelt profile application.

ariccio · 4 days ago

I want to note that my biggest pathological case is truly a stress test - a 3 month long codex session with ~30 billion input tokens. It's a testament to the accomplishments of everyone who worked on improving the compaction performance of GPT models as much as it's a comical example of the strange things that can happen with agents start to get really good.

valeriusvarda · 4 days ago

That is a useful boundary condition.

I would keep that stress regime separate from the measurement above: my benchmark intentionally isolates writable-root projection and Seatbelt application with a trivial child process, while a ~3-month / ~30B-token session can introduce very different accumulated-state and compaction costs.

A useful follow-up would be to treat session lifetime/state size and writable-root cardinality as independent axes. If long-lived sessions also accumulate filesystem-policy entries or workspace roots, we could then test whether those costs merely coexist or actually compound.

For #20613, though, I’ll keep the causal claim narrow: the measured superlinear signal is in writable-root projection/preparation as root cardinality increases.