Add GUI interaction support on Linux

What variant of Codex are you using?

CLI

What feature would you like to see?

Warning: this prototype is written entirely by GPT-5.5 with no manual code review (but we all know that GPT-5.5 hat PHD level intelligence, so criticizing this code will only out you as too stupid for AI) https://github.com/NoNaeAbC/codex/tree/codex/gui-wayland-capture-support

My worst codex-cli usage experience is when it comes to debugging computer graphics. Currently codex is not able to autonomously work on GUIs thus this should be added to codex.

This prototype while having a solid foundation (by what I told GPT to do, I did not read the code myself) has several issues that need to be worked on:

First the overall goal is that there is a single JS console in the background and the model interacts with the console. There are several reasons including:

• Only one tool instead of dozens of tools like press button, release button.
• It lets the model create precise actions and events like "press the button for exactly 5 seconds" or "press the button every 10 seconds"

Most events aren't implemented because once the JS path exists it should be trivial.
One could thing about adding custom protocols for the future so that the model can decine custom debug interactions with the application, but that would require partial redesign because the protocols that are supported must be known at compile time.

I am not happy how the model handheld zwp_linux_dmabuf_feedback_v1 because it can happen that a window is not able to be captured because this code doesn't support a specific format or layout.

Proper colorspace support is non-existent, one reason is that one can't simply tell the model to "handle colorspace correctly" and expect it to do so, even if one provides the h.273 pdf. The second reason is that I found no reference to gpt-5.5 colorspace handling itself and thus don't know how to do tone mapping.

I did not do extensive testing, but I know that chrome and firefox work on my machine. But I don't think that this should support every kiosk WM under the sun. That means there will always be machines that this will not support. Most of the testing that is required will not cover the cases where everything works, but when the app spawned by GPT is not behaving, getting unresponsive and cleaning up resources.

I did not review the sandbox code, in theory this code should have no negative impact on the sandbox, in fact even a positive one, because this code is supposed to only whitelist /dev/dri and a single unix socket. This might be a improvement because a lot of code that currently must be ran outside the sandbox, could stay in there. Note that at one moment in time at least the GPT-5.5 decided it was best to disable the entire sandbox, I hope that this was only intermediate. Sandbox escape requires kernel vulnerabilities to exist.

I do not have any headless support, but adding this should be trivial, I simply don't have any interest because I want to see what the model does.

Another issue that I can't solve is that GPT-5.5 is not trained to identify rendering artifacts in images, given that this is the primary reason this code exists (debugging rendering issues without me having to paste screenshots), there is real danger that this code will convince the model even more that the code is correct because GPT-5.5 doesn't have my description.

Note that this should not replace specialized tools like chrome MCP, it is focused on debugging computer graphics tasks and definitely not "book me a flight on that website" or any other agentic computer usage.

Additional information

_No response_