Computer Use screenshots can capture unrelated windows instead of the target window

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• #22892

Powered by Codex Action

This suggested duplicate does not appear to be related to this report.

The linked issue (#22892) is about Windows Codex app XHigh verification quality and native WinUI UI fixes. This issue is specifically about Codex Computer Use screenshot capture on macOS: Computer Use captured unrelated windows instead of the intended target/browser window, which creates unnecessary token usage and a privacy/safety risk.

So I do not think this issue should be closed as a duplicate of #22892.

Solution: Fix Computer Use Screenshots Capturing Unrelated Windows

Issue: openai/codex #24301
Labels: bug, app, computer-use
Component: Computer Use Screenshot Capture
Severity: HIGH (Privacy & Safety)

---

1. Problem Statement

Codex Computer Use takes screenshots that capture unrelated windows instead of the target application window. In the reported case:

• Computer Use was investigating logs in Chrome
• Screenshots captured ChatGPT web page and Codex app window instead
• This is a privacy and safety issue — unrelated user content may be included in screenshots sent to the AI model

---

2. Root Cause Analysis

2.1 Screenshot Capture Flow

On macOS, screenshot capture typically uses one of these mechanisms:

1. CGWindowListCreateImage — captures all windows or windows by option
2. ScreenCaptureKit (macOS 12.3+) — captures specific windows/apps
3. Accessibility API — captures window bounds via AXUIElement

The Bug: The screenshot capture is likely using a "capture all visible windows" or "capture primary display" mode instead of targeting the specific window identified by Computer Use.

2.2 Likely Root Cause

Window Targeting Logic Missing or Incorrect

The Computer Use module identifies a target window (e.g., Chrome tab showing logs), but the screenshot capture function doesn't use this window reference. Instead, it captures:

• The entire screen, OR
• All windows at the current mouse position, OR
• The frontmost window (which may have changed since target identification)

Race Condition: Between identifying the target window and capturing the screenshot, the window z-order may have changed, causing a different window to be captured.

---

3. Proposed Fix

3.1 Primary Fix: Window-Targeted Screenshot Capture

File: src/computer-use/screenshot.ts (or equivalent)

interface ScreenshotOptions {
  targetWindow?: WindowRef;  // NEW: explicit window target
  captureMode: 'window' | 'screen' | 'region';
}

async function captureScreenshot(options: ScreenshotOptions): Promise<Screenshot> {
  if (options.captureMode === 'window' && options.targetWindow) {
    // Use ScreenCaptureKit (macOS 12.3+) or CGWindowListCreateImage with window ID
    const windowId = options.targetWindow.cgWindowID;
    
    // ScreenCaptureKit approach (preferred on macOS 12.3+)
    const content = await SCShareableContent.create();
    const window = content.windows.find(w => w.windowID === windowId);
    if (!window) {
      throw new Error(`Window ${windowId} not found`);
    }
    
    const capture = await SCStream.captureFrame(window);
    return { image: capture.image, sourceWindow: window.title };
  }
  
  // Fallback: CGWindowListCreateImage with specific window ID
  const image = CGWindowListCreateImage(
    CGRectZero,
    CGWindowListOption.includeOnlyWindowID,  // KEY FIX: only capture specific window
    options.targetWindow?.cgWindowID ?? 0,
    CGWindowImageOption.default
  );
  
  return { image, sourceWindow: options.targetWindow?.title };
}

3.2 Secondary Fix: Window Validation Before Capture

Add validation that the target window is still valid before capturing:

async function captureTargetedScreenshot(targetWindow: WindowRef): Promise<Screenshot> {
  // 1. Verify window still exists
  if (!isWindowValid(targetWindow.cgWindowID)) {
    throw new Error(`Target window ${targetWindow.title} no longer exists`);
  }
  
  // 2. Verify window is still on screen (not minimized/hidden)
  const windowInfo = getWindowInfo(targetWindow.cgWindowID);
  if (windowInfo.isOnScreen === false) {
    throw new Error(`Target window ${targetWindow.title} is not visible on screen`);
  }
  
  // 3. Capture ONLY this window (not the whole screen)
  return captureScreenshot({
    targetWindow,
    captureMode: 'window'
  });
}

3.3 Tertiary Fix: Add Source Attribution to Screenshots

Every screenshot should include metadata about what was captured:

interface Screenshot {
  image: Buffer;
  sourceWindow: string;      // Window title
  sourceApp: string;         // Application name
  sourceWindowID: number;    // CGWindowID for verification
  captureMode: 'window' | 'screen' | 'region';
  timestamp: number;
}

This allows the Computer Use agent to verify that the correct window was captured before processing.

---

4. Testing Strategy

Unit Tests

1. Capture specific window → screenshot contains only that window's content
2. Window minimized during capture → error thrown, not a screenshot of wrong content
3. Window closed during capture → error thrown
4. Multiple windows same app → correct window captured by ID, not by name

Integration Tests

1. Computer Use identifies Chrome window → screenshot shows Chrome content only
2. User switches to another app mid-task → Computer Use detects mismatch and retries
3. Multiple monitors → correct window on correct monitor captured

Manual Testing

1. Open Chrome with logs, open ChatGPT in another window
2. Start Computer Use task on Chrome
3. Verify screenshots only contain Chrome content
4. Switch to ChatGPT during task → verify Computer Use detects and handles

---

5. Impact Assessment

• Severity: HIGH — Privacy leak of unrelated user content to AI model
• User Impact: Users' private content (emails, chats, documents) may be visible to AI
• Risk of Fix: Low — adding window targeting is additive and more restrictive
• Breaking Changes: None — existing screen capture mode still available as fallback

---

6. Files to Modify

| File | Change |
|------|--------|
| src/computer-use/screenshot.ts | Add window-targeted capture mode |
| src/computer-use/window-utils.ts | Add window validation helpers |
| src/computer-use/computer-use-agent.ts | Use window-targeted capture |
| src/types/screenshot.ts | Add source attribution fields |
| tests/computer-use/screenshot.test.ts | Add regression tests |

---

7. Estimated Effort

• Implementation: 4-6 hours
• Testing: 2-3 hours
• Review: 1 hour
• Total: 7-10 hours

Additional Windows reproduction showing that the selected-window metadata and returned bitmap can disagree.

Observed

1. Computer Use window discovery returned exactly one window matching the target application.
2. The accessibility readback for that selection identified the correct target window and executable.
3. A screenshot requested from the same selected window rendered an unrelated Codex task window instead.
4. No screenshot content is attached or quoted here.

This is more specific than an accidental full-screen capture: the accessibility tree identified the intended window, while the screenshot payload belonged to another window. Protected unrelated task content entered the model context, so coordinate actions were stopped immediately.

A fresh selection/activation again produced the correct accessibility tree, but target-safe input methods did not change the selected app state; no coordinate fallback was attempted. The isolated test app was then closed cleanly.

Expected

• Bind the screenshot to a stable window identity (for example HWND + process identity/creation time), not only the latest focused surface.
• Return source-window identity with the bitmap and compare it with the requested target before exposing the image to the model.
• Fail closed when accessibility metadata and screenshot provenance disagree.

This is both a functional targeting failure and a privacy boundary issue. No unrelated screenshot, task text, local path, window ID, or user data is included in this report.