Expose custom interaction hooks and APIs for Codex desktop pets

+1

+1

I’d like to add a related use case that would benefit from a small plugin-published pet state API, not just read-only pet/status observation.

I’m building a local Codex plugin for a Steam-focused Windows setup. The plugin can already detect useful external state locally, for example:

• Steam client open/closed
• Steam Big Picture/controller UI active
• installed Steam app/game process running
• launch or validate action requested from a local dashboard
• recent Steam app update/patch activity

Today the plugin can render a separate dashboard companion using the same Codex pet spritesheet contract, but it cannot drive the native floating Codex pet. That means the official pet and the plugin companion duplicate each other.

A minimal API that would solve this cleanly:

codex.pet.setState({
  source: "steam-command-center",
  animation: "running", // existing pet row id
  title: "Crimson Desert",
  detail: "Game running",
  ttlMs: 5000,
  priority: 40
})

codex.pet.clearState("steam-command-center")

Suggested constraints:

• Only allow existing pet animation row IDs, e.g. idle, waving, jumping, failed, waiting, running, review, etc.
• Require explicit user permission per plugin/source, e.g. “Allow Steam Command Center to temporarily control pet status.”
• Require TTL expiry so stale external state does not leave the pet stuck.
• Include source attribution so users know what is controlling the pet.
• Let first-party Codex work states win over lower-priority plugin states, especially waiting-for-input, approval, failure, and ready-for-review states.
• Avoid arbitrary code execution or arbitrary animation injection; this should be a small state handoff, not a general desktop automation surface.

The broader design ask: let trusted plugins publish ephemeral, permissioned pet state to the native floating pet overlay, while Codex remains the arbiter of priority and safety. This would let local integrations such as Steam, OBS, CI monitors, build dashboards, or hardware monitors use the official pet as the shared status surface instead of each building their own overlay.

I want to add a related product-design point from the CLI user side.

For my workflow, the main value of pets is not that they appear inside the current terminal surface. The value is that they are an ambient, always-visible status surface for Codex work: running, waiting for input or approval, ready for review, failed, idle, etc. If the pet is confined to a terminal pane, it only works while that terminal is visible and focused enough to notice. That makes it much less useful as a status indicator.

This is also why terminal pets feel like the wrong primary design direction to me, independent of tmux / Sixel / Kitty compatibility. Terminal rendering is useful as a fallback for SSH, headless, or no-Desktop environments, but the primary local experience should be:

• CLI sessions publish lifecycle/status events locally.
• Codex Desktop floating pet overlay consumes those events.
• The Desktop pet stays above other apps and reflects active CLI work in the same way it reflects Desktop-created work.
• Terminal pets remain an optional fallback, not the main status surface.

This is related to #22625, but I do not think the important framing is “the terminal pet does not work” or “tmux does not support images.” The more important issue is that pets are fundamentally ambient status UI, and ambient status UI should live outside the terminal container. A small permissioned state API like the one discussed above (codex.pet.setState(...), or a first-party CLI -> Desktop event bridge) would solve this cleanly without requiring every integration to build its own overlay.

+1 for a documented, read-only pet/task status API. I have a concrete external-monitoring use case that is not currently covered by hooks or by connecting to a separately managed app-server.

I want to run a small local background service that can reflect Codex activity while I continue using Codex normally across the Desktop app, CLI, and IDE. The minimum useful state model would be:

• idle: no active turn
• running: Codex is reasoning, calling tools, editing files, or streaming a reply
• waiting_for_approval: the active turn is waiting for command/file/network/tool approval
• error: the turn failed or the thread entered a system-error state

A minimal API could expose both a current snapshot and a stream of transitions, for example over a user-scoped local socket:

• task/thread id
• active turn id, when present
• state
• source surface (desktop, cli, ide)
• timestamp
• optional redacted error category

Important lifecycle semantics would include explicit approval-requested and approval-resolved transitions, plus turn-failed/turn-completed events. Prompt text, assistant messages, commands, and file contents should be excluded by default.

Today the available mechanisms leave gaps:

• Global hooks provide useful events such as UserPromptSubmit, PermissionRequest, and Stop, but there is no complete approval-resolved or turn-error lifecycle for an external status service.
• App-server exposes the right structured events, but Desktop/IDE clients own their app-server connection, often over private stdio, so an unrelated local process cannot subscribe to the same stream.
• Parsing persisted transcripts would be brittle because that format is not a stable public interface.

The built-in floating pet already derives equivalent product states (Running, Needs input, Ready, Blocked). Exposing a privacy-preserving read-only view of that same derived state would let menu-bar apps, accessibility tools, hardware indicators, and custom desktop companions integrate without parsing conversations or forcing every Codex client through a custom shared app-server.

I opened #34145 as an implementation-focused child proposal for the interaction contract discussed here. It narrows the first deliverable to manifest-defined, permissioned bindings for actions already advertised by Codex or an installed app, with an illustrative music-control use case, a backwards-compatible pet.json shape, dispatcher pseudocode, security boundaries, staged implementation cost, and MVP acceptance criteria. It is intended to complement—not replace—this broader API request.