Computer Use plugin: all action tools fail with 'not active' due to framework session model conflict

Open 💬 2 comments Opened Jun 7, 2026 by gainaixiaobing

Environment

  • Codex Desktop on macOS
  • Plugin: computer-use v1.0.809 (openai-bundled)
  • MCP Server: SkyComputerUseClient.app via stdio

Summary

All Computer Use action tools (click, type_text, scroll, drag, set_value, perform_secondary_action) fail with:

"Computer Use is not active for '{app}'. You first must call get_app_state to get the latest state before doing other Computer Use actions."

get_app_state and list_apps work correctly every time — they return valid screenshots and accessibility trees. The problem is strictly with action tools that depend on a session established by get_app_state.

Root Cause

get_app_state's description states: "This must be called once per assistant turn before interacting with the app."

This creates a framework-level conflict with the current tool-calling model:

  1. Separate batches (sequential turns): Call get_app_state in batch 1, receive result, call click in batch 2. The click fails because each batch is treated as a separate "assistant turn", and the session state from batch 1 does not persist to batch 2.
  1. Same batch (parallel execution): Call get_app_state and click together in one function_calls block. The click fails because parallel execution offers no ordering guarantee — click may execute before or concurrently with get_app_state, before the session is established.
  1. Coordinate-based clicks: Same issue — the session check fails regardless of whether element_index or x,y coordinates are used.

The plugin requires sequential execution within a single turn (get_app_state first, then action), but the framework only supports either parallel (no ordering) or cross-turn sequential (no session persistence).

Steps to Reproduce

  1. Open Calculator.app on macOS
  2. In Codex desktop, invoke get_app_state for Calculator — succeeds, returns screenshot + UI tree
  3. Immediately invoke click with element_index: "6" (AllClear button) — fails with "not active" error
  4. Try calling get_app_state and click in the same parallel batch — click still fails (race condition)

Expected Behavior

Action tools should work after get_app_state is called in the same turn. The framework should either:

  • Support sequential tool execution within a single turn (execute get_app_state to completion before executing click)
  • Persist the Computer Use session across turns within a conversation

Impact

Complete feature unavailability. The Computer Use plugin is entirely non-functional for any action that requires UI interaction. Only read-only operations (get_app_state, list_apps) work. This affects all Computer Use use cases: app automation, UI testing, accessibility-based control, etc.

Suggested Fix

The Codex desktop framework needs to support one of:

  1. Ordered sequential calls within a turn: Allow declaring tool call dependencies so get_app_state completes before dependent action tools execute
  2. Persistent session across turns: Let the Computer Use MCP server maintain its session state across assistant turns within the same conversation
  3. Deferred session activation: Let action tools implicitly trigger get_app_state if no session is active

View original on GitHub ↗

This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗