Windows Computer Use can pair lock-screen pixels with target-app accessibility state
What version of the Codex App are you using?
Codex Desktop 26.715.3651.0
Bundled Computer Use plugin: 26.715.31251
Bundled @oai/sky: 0.4.20
What subscription do you have?
Not relevant / not verified for this local runtime behavior.
What platform is your computer?
Windows x64, OS build 26100.
What issue are you seeing?
When the Windows desktop is locked, the supported bundled Computer Use wrapper initializes and can still enumerate or launch target applications, but window activation and state capture do not surface a reliable structured locked-session result.
In a locked session:
activate_windowfailed with the generic errorfailed to activate captured window.get_window_state({ include_screenshot: true, include_text: true })returned lock-screen pixels while the returned window identity still referred to the requested application.- For Paint, the same state response included a detailed Paint accessibility tree even though the screenshot showed the Windows lock screen.
- For Calculator, the screenshot showed the lock screen while accessibility was null.
This cross-modal mismatch makes it possible for an agent to see target-app accessibility semantics paired with pixels from a different security surface. The bundled guidance correctly says to stop when the desktop is locked, but the API did not identify that condition directly; it had to be inferred visually.
No UI input was sent while the desktop was locked.
What steps can reproduce the bug?
- Lock the Windows desktop.
- In Codex Desktop, initialize Computer Use through the supported bundled wrapper:
<plugin root>/scripts/computer-use-client.mjs.
- Read the bundled
guidance,api, andconfirmationsdocumentation. - Call
list_windows()/list_apps(). - Launch Calculator or Paint and select exactly one returned target window.
- Call
activate_window({ window }). - Call
get_window_state({ window, include_screenshot: true, include_text: true }). - Compare the screenshot surface with the returned window identity and accessibility tree.
What is the expected behavior?
One of the following should happen before any input is possible:
- Return a structured
desktop_lockedor equivalent error. - Expose a first-class session/health state such as
get_session_state()orhealth_check()with desktop lock, interactive-session, capture, and UI Automation readiness. - Refuse to return a target-app accessibility tree when the captured pixels belong to the lock screen.
- At minimum, label the capture surface so screenshot/accessibility disagreement is explicit.
activate_window should also distinguish locked desktop, stale window, and permission/activation failures instead of returning only failed to activate captured window.
Additional information
After manually unlocking the desktop, the same bundled runtime successfully:
- Launched a fresh Calculator window.
- Activated and captured matching screenshot/accessibility state.
- Clicked
2 + 3 =using fresh accessibility state before each action. - Verified the display value
5. - Closed only the owned Calculator window and verified it was gone.
That isolates the failure to locked-session detection/capture behavior rather than general plugin initialization or Calculator control.
Related: #26528 includes intermittent failed to activate captured window behavior, but does not appear to cover this reproducible lock-screen screenshot / target-app accessibility mismatch.