Computer Use plugin: all action tools fail with 'not active' due to framework session model conflict

Open 💬 3 comments Opened Jun 7, 2026 by gainaixiaobing

Environment

  • Codex Desktop on macOS
  • Plugin: computer-use v1.0.809 (openai-bundled)
  • MCP Server: SkyComputerUseClient.app via stdio

Summary

All Computer Use action tools (click, type_text, scroll, drag, set_value, perform_secondary_action) fail with:

"Computer Use is not active for '{app}'. You first must call get_app_state to get the latest state before doing other Computer Use actions."

get_app_state and list_apps work correctly every time — they return valid screenshots and accessibility trees. The problem is strictly with action tools that depend on a session established by get_app_state.

Root Cause

get_app_state's description states: "This must be called once per assistant turn before interacting with the app."

This creates a framework-level conflict with the current tool-calling model:

  1. Separate batches (sequential turns): Call get_app_state in batch 1, receive result, call click in batch 2. The click fails because each batch is treated as a separate "assistant turn", and the session state from batch 1 does not persist to batch 2.
  1. Same batch (parallel execution): Call get_app_state and click together in one function_calls block. The click fails because parallel execution offers no ordering guarantee — click may execute before or concurrently with get_app_state, before the session is established.
  1. Coordinate-based clicks: Same issue — the session check fails regardless of whether element_index or x,y coordinates are used.

The plugin requires sequential execution within a single turn (get_app_state first, then action), but the framework only supports either parallel (no ordering) or cross-turn sequential (no session persistence).

Steps to Reproduce

  1. Open Calculator.app on macOS
  2. In Codex desktop, invoke get_app_state for Calculator — succeeds, returns screenshot + UI tree
  3. Immediately invoke click with element_index: "6" (AllClear button) — fails with "not active" error
  4. Try calling get_app_state and click in the same parallel batch — click still fails (race condition)

Expected Behavior

Action tools should work after get_app_state is called in the same turn. The framework should either:

  • Support sequential tool execution within a single turn (execute get_app_state to completion before executing click)
  • Persist the Computer Use session across turns within a conversation

Impact

Complete feature unavailability. The Computer Use plugin is entirely non-functional for any action that requires UI interaction. Only read-only operations (get_app_state, list_apps) work. This affects all Computer Use use cases: app automation, UI testing, accessibility-based control, etc.

Suggested Fix

The Codex desktop framework needs to support one of:

  1. Ordered sequential calls within a turn: Allow declaring tool call dependencies so get_app_state completes before dependent action tools execute
  2. Persistent session across turns: Let the Computer Use MCP server maintain its session state across assistant turns within the same conversation
  3. Deferred session activation: Let action tools implicitly trigger get_app_state if no session is active

View original on GitHub ↗

3 Comments

piazzatron · 2 months ago

Ran into this one as well today. Huge blocker for autonomous workflows.

pmxi · 2 months ago

I had a similar issue. I deleted ~/.codex/plugins/cache (which deletes plugins) and reinstalled computer-use, which worked.

pro-dg · 1 month ago

Reproduced a closely related failure on macOS with the current openai-bundled computer-use plugin 1.0.1000451.

The Node REPL wrapper now allows ordered calls in one mcp__node_repl__js invocation, but that did not resolve the failure for Cursor (com.todesktop.230313mzl4w4u92):

  1. get_app_state({ app: "Cursor" }) can succeed and return the accessibility tree and screenshot.
  2. A following click({ app: "Cursor", ... }) reports: Computer Use is not active for '/Applications/Cursor.app'.
  3. Putting await get_app_state(...) and then await click(...) sequentially in the same Node REPL invocation instead ends with: Sky Computer Use native pipe closed before response.
  4. Calling get_app_state with the exact bundle id or /Applications/Cursor.app also closes the native pipe, while list_apps() continues to work and reports Cursor as running.

This suggests there is still an app-target normalization/session-key mismatch (Cursor display name versus resolved app path), plus a native helper crash/exit on the exact-path path. A stable canonical target key across get_app_state and action calls—and an error containing the helper exit status/crash-log path—would make this debuggable and restore action calls.