Login-shell tests rely on non-hermetic startup assumptions
What issue are you seeing?
On v0.149.0-alpha.3, several Linux codex-core tests fail when a login shell sources startup configuration that emits diagnostics or attempts filesystem writes unavailable inside the sandbox. The extra startup output becomes part of the captured command result, causing exact-output and remote-compaction-parity assertions to fail.
Affected categories include request-permission tests, remote-compaction-parity tests, and a login-shell test that verifies the shell home.
What steps can reproduce the bug?
- Use a Linux environment with non-trivial login-shell startup configuration.
- Run the affected
codex-coretests under a sandbox whose filesystem permissions differ from the host shell environment. - Observe that startup diagnostics are appended to the command output and cause the assertions to fail.
What is the expected behavior?
Tests that exercise login shells should be hermetic. Their expected command output should not depend on ambient user startup files, host-specific hooks, or whether paths used by those hooks are writable inside the sandbox.
Additional information
The failure exposes several assumptions that do not hold for sandboxed login shells:
- isolating
HOMEdoes not necessarily isolate every startup-file or shell-initialization source; - host startup configuration may run in a filesystem namespace with different permissions;
- command-output and parity tests assume that startup hooks produce no observable output.
Suggested fixes:
- run these tests with a fully controlled temporary home and startup-file set;
- explicitly suppress or override ambient startup hooks while retaining dedicated coverage for login-shell behavior; and
- separate shell diagnostics from the command output captured by the test harness.
A related host-environment test-isolation issue is #38906.
1 Comment
Commit
9ec70ca31ccontrols shell startup inputs at the child-process boundary: each test shell receives an isolated HOME plus controlled BASH_ENV and ENV values pointing to an empty generated startup file. The process-wide environment is not mutated, and tests that intentionally provide shell variables can still override the defaults. The login-shell and remote-compaction-parity tests pass with this approach.The request-permission failures observed in the same runner are separate: those commands fail before execution because the runner cannot create the sandbox namespace.