bug(computer-use): Return key is not delivered to Pygame windows

Open 💬 1 comment Opened Jul 26, 2026 by Orinks

Summary

Computer Use can navigate a Pygame menu with arrow keys, but its key-injection call reports success for activation keys while Pygame receives no corresponding event.

Reproduction

  1. Open a simple Pygame window that records keyboard events.
  2. Target the window with Computer Use.
  3. Send the Return key with the desktop-control key API.

Expected behavior

Pygame receives a KEYDOWN/KEYUP event for Return, allowing Enter to activate a menu selection.

Actual behavior

Computer Use reports the key was sent, but the Pygame event log records no Return event. Arrow-key navigation works.

Additional observation

The desktop focus step produced an unexpected left-Alt event in the Pygame probe. A hidden probe could not accept input because Computer Use reported that the foreground window had no process identity.

Impact

This blocks automated desktop verification of keyboard-driven Pygame workflows. Manual testing works normally; this appears limited to the Computer Use input/focus layer.

View original on GitHub ↗

1 Comment

brady-bbrr · 27 days ago

Related broader reproduction: this affects more than Pygame’s Return key.

Feedback session: 019fbae0-5e34-75f1-9bc8-adbed9b80a2d

Environment:

  • Windows 11 Pro x64, build 26200
  • Computer Use plugin 26.727.40816
  • Satisfactory 1.2.3.1 CL#495413 (Unreal Engine)
  • Game running Windowed at 1920×1080
  • Steam, the game launcher, and the shipping executable all verified Elevated=False
  • No RUNASADMIN compatibility flags

Reproduction:

  1. Activate and click the Satisfactory window.
  2. Confirm Computer Use mouse clicks and camera movement work.
  3. Call press_key for Tab, M, or B.
  4. Each call reports success, but the game receives no corresponding action.
  5. type_text likewise does not invoke game hotkeys.

Control tests from the same Computer Use runtime succeed:

  • Notepad receives literal text and Ctrl+A.
  • Calculator receives 123+456= and produces 579.
  • Calculator receives Escape and clears its display.

Satisfactory exposes no Raw Input toggle in its current Controls menu, and its saved configuration contains no RawInput or bUseRaw setting. The failure was already reproduced outside exclusive fullscreen and with matching non-elevated integrity levels.

This suggests the issue is broader than one Pygame key: applications using game-oriented input paths may reject the current synthetic keyboard backend while still accepting Computer Use mouse input.

Expected behavior:

  • Deliver compatible key-down/key-up events—potentially using a scancode-compatible backend—or
  • Return an explicit unsupported-target/failure result instead of reporting success when no keyboard event reaches the application.