Test mode for evaluating AGENTS.md changes

What feature would you like to see?

I'd like a test mode which can rerun Codex N times with a given starting prompt / AGENTS.md / starting directory and record the distribution of responses it gives to that starting prompt.

My motivation is I found Codex CLI responds non-deterministically / stochastically to changes in my AGENTS.md which makes it really difficult to assess whether changes to AGENTS.md are making things better or worse overall - I need to see a distribution of responses or almost like an A/B test of two different AGENTS.md to see if changes in AGENTS.md context give better prompt adherence.

I initially updated AGENTS.md just on intuition and gut feeling. More recently I started altering AGENTS.md and manually restarting Codex CLI over and over with the same prompt and count how often it did or didn't follow my new AGENTS.md instructions (it's this onerous later process I'd like to be more automated).

In terms of output if the results could either be saved to a file or a pretty TUI table showing the full response, both text and (hypothetical) shell commands that would've been run and what timeouts were used (some of my AGENTS.md instructions just relate to setting command timeouts) and whether a permission elevation request would've been made (some of my AGENTS.md instructions relate to what to do elevation requests on) - showing the unique responses and a count of how many of the N times each response was generated that'd be immensely useful for helping fine tune my AGENTS.md files for Codex.

Additional information

_No response_