Chrome + Computer use struggle with Chrome-hosted remote VMs, causing slow and expensive retry loops

Codex app version

26.527.60818 (3437)

Platform

• macOS Tahoe 26.5
• Apple Silicon MacBook Pro
• Pro plan

Related feedback ID

019e6d9e-e292-7fc3-b38e-de4fa40de6f0

Summary

I am reporting a usability and performance issue with Codex when using both the Chrome plugin and Computer use to operate a remote virtual machine through a Chrome browser window.

This is not only a Computer use issue. In real Codex workflows, the Chrome plugin and Computer use are often used together: Chrome provides the browser context, while Computer use handles the visual interaction, clicking, typing, scrolling, and navigating inside the page. When the page itself contains a remote VM viewport, both tools need to handle a nested and latency-sensitive visual environment.

Currently, Codex is very slow and unreliable when controlling a VM displayed inside Chrome. The model often appears unfamiliar with this type of nested VM interface, makes frequent incorrect operations, and then has to undo or redo multiple steps. This wastes a large amount of time and tokens.

Observed behavior

When working inside a remote VM through Chrome:

• Codex moves and acts much more slowly than when controlling a normal browser page.
• It frequently performs the wrong operation or clicks/types in the wrong place.
• Several steps often need to be restarted or redone from scratch.
• The total task time increases by multiple times.
• Token consumption becomes extremely high compared with normal browser operation or standard computer-use tasks.
• The Chrome plugin and Computer use do not seem to fully adapt to the latency and interaction delay of a remote VM viewport embedded inside Chrome.

This is especially painful because the mistakes are not always due to task difficulty. Many of them seem related to Codex not being sufficiently skilled at operating a VM viewport embedded inside Chrome, especially when input latency, rendering delay, focus changes, or nested coordinate mapping are involved.

Why this matters

For some workflows, especially during early VM setup or when a VM does not yet have SSH or other terminal access configured, users cannot simply use SSH or direct terminal commands. They must operate the remote virtual machine visually through the browser window.

In those situations, Codex needs to be able to reliably operate the Chrome-hosted VM interface. Otherwise, tasks become much slower and much more expensive in tokens than they should be.

Expected behavior

Codex should be more proficient when using the Chrome plugin together with Computer use to operate a remote VM displayed inside the browser. It should be able to:

• Recognize that it is interacting with a nested VM viewport inside Chrome.
• Adapt to VM display/input latency instead of acting as if it were a normal low-latency web page.
• Click, type, scroll, and navigate inside the VM more accurately.
• Avoid repeated incorrect actions and unnecessary resets.
• Complete VM setup and configuration workflows with time/token costs closer to normal Chrome/computer-use tasks.

Actual behavior

Operating a Chrome-hosted remote VM currently takes far longer than expected, often with repeated mistakes and rollbacks. The total time and token usage can become much higher than both ordinary Chrome browsing tasks and ordinary computer-use tasks.

Request

Please improve Codex's ability to visually operate remote virtual machines inside Chrome using the combined Chrome plugin + Computer use workflow. This likely requires more training/evaluation on nested browser-based VM environments, including initial setup workflows where SSH or direct terminal access is not yet available.

Please also optimize how the Chrome and Computer use tools handle remote VM latency. The current behavior does not perfectly adapt to delayed rendering, delayed input feedback, nested focus, and remote-desktop-style interaction inside a Chrome page.

This would make Codex much more practical for real-world VM setup, cloud console, lab environment, and browser-hosted remote desktop workflows.