Codex CLI Multi-Agent Workflows (v0.105.0)
What is the type of issue?
Documentation is missing
What is the issue?
Codex CLI Multi-Agent Workflows (v0.105.0)
Source: openai/codex rust-v0.105.0
Date researched: 2026-02-25
Documentation status: Undocumented by OpenAI — the official changelog stops at v0.99.0 and none of these features appear in developers.openai.com.
---
Overview
Codex CLI v0.105.0 introduced a significant batch-processing primitive called spawn_agents_on_csv alongside a suite of TUI improvements for managing sub-agents. Together, these features turn Codex from a single-agent coding tool into an orchestrator capable of map-reduce-style workflows over arbitrarily large datasets.
The work spans six PRs:
| PR | Title | Author | Impact |
|----|-------|--------|--------|
| #10935 | Agent jobs (spawn_agents_on_csv) + progress UI | @daveaitel-openai | +3370 lines — core feature |
| #12320 | Add nickname to sub-agents | @jif-oai | +1125 lines |
| #12327 | Cleaner TUI for sub-agents | @jif-oai | +1734 lines |
| #12332 | Better agent picker in TUI | @jif-oai | +1828 lines |
| #12570 | Keep dead agents in the agent picker | @jif-oai | +189 lines |
| #12767 | Display pending child-thread approvals in TUI | @jif-oai | +419 lines |
---
1. spawn_agents_on_csv — The Core Primitive
What it does
spawn_agents_on_csv is a tool the model can call that reads a CSV file, creates one sub-agent per row, fans them out with configurable concurrency, collects results back into a new CSV, and provides real-time progress with ETA. The entire lifecycle — spawn, run, collect, export — happens in a single deterministic tool call.
This is the map-reduce pattern for LLM agents: the CSV is the input manifest, each row is a work item, each sub-agent is a mapper, and the output CSV is the reduction.
Tool parameters
The tool exposes these parameters (from codex-rs/core/src/tools/spec.rs):
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| csv_path | string | yes | Path to the input CSV file |
| instruction | string | yes | Template instruction for each worker. Supports {column_name} placeholders filled from row data |
| id_column | string | no | Column name to use as stable item ID (defaults to row-0, row-1, etc.) |
| output_csv_path | string | no | Where to write results (defaults to <input_stem>.agent-job-<suffix>.csv) |
| max_concurrency | integer | no | Max parallel workers (capped by agents.max_threads config) |
| max_workers | integer | no | Alias for max_concurrency. Set to 1 for sequential |
| max_runtime_seconds | integer | no | Per-item timeout before a worker is marked failed |
| output_schema | object | no | JSON schema to enforce on worker results (planned structured output support) |
There is a companion tool called report_agent_job_result that workers call to report their results back:
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| job_id | string | yes | ID of the parent job |
| item_id | string | yes | ID of the specific work item |
| result | object | yes | JSON result object |
| stop | boolean | no | If true, cancels all remaining items |
How it works internally
The implementation lives in codex-rs/core/src/tools/handlers/agent_jobs.rs (1,227 lines). Here's the lifecycle:
1. CSV parsing and item creation
When the model calls spawn_agents_on_csv, the handler:
- Reads and parses the CSV (handles BOM, flexible column counts, empty-row skipping)
- Validates headers are unique
- Generates stable item IDs from the
id_columnor falls back torow-Nformat - Deduplicates IDs automatically — if two rows share the same ID, it appends
-2,-3, etc. - Creates a job record and all item records in SQLite
2. Template instruction rendering
The instruction field supports {column_name} placeholders. For each row, the handler substitutes column values into the template:
instruction: "Review the file at {path} and check for {area} issues"
Given a CSV row path=src/auth.rs, area=security, the worker receives:
Review the file at src/auth.rs and check for security issues
Double braces {{ and }} are treated as literal braces (escaped), matching the Mustache convention.
3. Worker prompt construction
Each spawned sub-agent receives a structured prompt built by build_worker_prompt() that includes:
- The rendered instruction (with placeholders filled)
- The
job_idanditem_idso the worker knows how to report back - The output schema (if provided) for structured results
- Instructions to call
report_agent_job_resultwhen done
4. Concurrency-controlled execution loop
The run_agent_job_loop() function orchestrates everything:
while items remain:
check for cancellation
fill available concurrency slots with pending items
for each slot:
spawn a sub-agent with the worker prompt
mark item as "running" with assigned thread ID
reap stale items (exceeded max_runtime_seconds)
find finished threads
finalize completed items (collect results, update status)
emit progress update
Key behaviors:
- Concurrency is bounded:
max_concurrencyis capped byagents.max_threadsfrom config (default varies by setup) - Stale worker reaping: Workers that exceed
max_runtime_secondsare force-failed and their thread is shut down - Crash recovery: If a worker's thread dies without reporting, the item is detected via
find_finished_threads()and marked appropriately - Cancellation: A worker can set
stop: truein itsreport_agent_job_resultcall to cancel all remaining items - Auto-export: On completion, the output CSV is written automatically
5. Progress and ETA
A JobProgressEmitter periodically emits progress updates as background events. The calculation:
rate = completed_items / elapsed_seconds
eta_seconds = remaining_items / rate
Progress events are emitted at a fixed interval (defined by PROGRESS_EMIT_INTERVAL), formatted as agent_job_progress:<json> background notifications. In codex exec mode, this drives a stable single-line progress bar that redraws in place (no scrolling).
SQLite schema
Jobs are persisted in SQLite across two tables (migration 0014_agent_jobs.sql):
CREATE TABLE agent_jobs (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
status TEXT NOT NULL, -- pending | running | completed | failed | cancelled
instruction TEXT NOT NULL,
output_schema_json TEXT,
input_headers_json TEXT NOT NULL,
input_csv_path TEXT NOT NULL,
output_csv_path TEXT NOT NULL,
auto_export INTEGER NOT NULL DEFAULT 1,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
started_at INTEGER,
completed_at INTEGER,
last_error TEXT,
max_runtime_seconds INTEGER -- added in migration 0015
);
CREATE TABLE agent_job_items (
job_id TEXT NOT NULL,
item_id TEXT NOT NULL,
row_index INTEGER NOT NULL,
source_id TEXT,
row_json TEXT NOT NULL, -- full row as JSON object
status TEXT NOT NULL, -- pending | running | completed | failed
assigned_thread_id TEXT, -- which sub-agent thread is working on this
attempt_count INTEGER NOT NULL DEFAULT 0,
result_json TEXT, -- worker's reported result
last_error TEXT,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
completed_at INTEGER,
reported_at INTEGER,
PRIMARY KEY (job_id, item_id),
FOREIGN KEY(job_id) REFERENCES agent_jobs(id) ON DELETE CASCADE
);
Output CSV format
The output CSV includes all original columns plus metadata columns appended:
<original_columns>, job_id, item_id, row_index, source_id, status, attempt_count, last_error, result_json, reported_at, completed_at
Configuration knobs
These can be set in config.toml under [agents]:
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| agents.max_threads | integer | varies | Global cap on concurrent sub-agent threads |
| agents.max_depth | integer | default set internally | Maximum nesting depth for sub-agents |
| agent_job_max_runtime_seconds | integer | internal default | Per-item timeout for batch jobs |
CLI overrides via -c:
codex exec -c agents.max_threads=16 ...
Feature gate
spawn_agents_on_csv requires the collab and sqlite experimental features to be enabled:
codex exec --enable collab --enable sqlite --full-auto ...
Demo usage
From the PR description — this is the canonical example:
codex exec \
--enable collab \
--enable sqlite \
--full-auto \
--progress-cursor \
-c agents.max_threads=16 \
-C /path/to/your/repo \
- <<'PROMPT'
Create /tmp/agent_job_progress_demo.csv with columns: path,area and 30 rows:
path = item-01..item-30, area = test.
Then call spawn_agents_on_csv with:
- csv_path: /tmp/agent_job_progress_demo.csv
- instruction: "Run `python - <<'PY'` to sleep a random 0.3-1.2s, then output JSON with keys: path, score (int). Set score = 1."
- output_csv_path: /tmp/agent_job_progress_demo_out.csv
PROMPT
The --progress-cursor flag enables the stable single-line progress bar.
Tests
Three integration tests exist in codex-rs/core/tests/suite/agent_jobs.rs:
spawn_agents_on_csv_runs_and_exports— verifies the full lifecycle: CSV in, workers run, results collected, output CSV exported with correct columnsspawn_agents_on_csv_dedupes_item_ids— verifies that duplicate item IDs (e.g., two rows with the sameid_columnvalue "foo") are automatically deduplicated to "foo" and "foo-2"spawn_agents_on_csv_stop_halts_future_items— verifies that when a worker reportsstop: true, remaining items stay pending and the job is marked cancelled. Asserts only 1 worker was spawned out of 3 items.
---
2. Sub-Agent Nicknames
What it does
Every sub-agent spawned by Codex now receives a random tree-themed nickname from a curated list. These names appear in the TUI, the agent picker, event logs, and are surfaced to the model itself.
The name list
The names are embedded at compile time from codex-rs/core/src/agent/agent_names.txt (100 entries):
Ash, Elm, Yew, Fir, Oak, Pine, Spruce, Cedar, Birch, Maple,
Beech, Alder, Willow, Poplar, Aspen, Larch, Juniper, ...
All tree names — short, memorable, easy to distinguish at a glance. The list is loaded via Rust's include_str!() macro.
How nicknames are assigned
When AgentControl::spawn_agent() is called:
- A spawn slot is reserved via
reserve_spawn_slot()(respectsagents.max_threads) - A nickname is picked from the list via
reserve_agent_nickname()— names are drawn without replacement within a session, so no two active agents share a name - The nickname is stored in the
SessionSource::SubAgent(SubAgentSource::ThreadSpawn { agent_nickname, ... })metadata - The nickname is persisted in SQLite via migration
0013_threads_agent_nickname.sql
Where nicknames appear
- Agent picker TUI: The
format_agent_picker_item_name()function renders names as: Ash [reviewer]— nickname with roleMaple— nickname alone[reviewer]— role without nickname (fallback)Agent— no metadata at allMain [default]— the primary thread- Event history: Spawn and interaction events show
"Ash spawned","Interaction with Birch completed" - Model context: Nicknames are surfaced to the model (PR #12575) so the model can refer to specific agents by name
Persistence
Migration 0013_threads_agent_nickname.sql adds nickname storage to the thread metadata table, so names survive across session restarts.
---
3. Cleaner TUI for Sub-Agents
What changed
PR #12327 rewrote codex-rs/tui/src/multi_agents.rs (+410/-173 lines) to present a cleaner view of sub-agent activity in the terminal UI.
New protocol types
Several new TypeScript protocol types were added for the app-server communication layer:
| Type | Purpose |
|------|---------|
| CollabAgentRef | Reference to a sub-agent (thread ID + nickname + role) |
| CollabAgentStatusEntry | Status of a specific agent (running, completed, etc.) |
| CollabAgentSpawnEndEvent | Emitted when a sub-agent spawn completes |
| CollabAgentInteractionEndEvent | Emitted when an interaction with a sub-agent finishes |
| CollabCloseEndEvent | Emitted when a sub-agent thread is closed |
| CollabWaitingBeginEvent / CollabWaitingEndEvent | Emitted when the parent is waiting on sub-agents |
| CollabResumeBeginEvent / CollabResumeEndEvent | Emitted when resuming a sub-agent |
History rendering
The TUI now renders sub-agent events as structured history cells with:
- Agent nickname and role in the title line
- Thread ID for traceability
- Prompt text that was sent to the agent
- Color-coded status indicators
The spawn_end() and interaction_end() functions build these cells with a title_with_agent() helper that incorporates nickname and role.
---
4. Better Agent Picker in TUI
What changed
PR #12332 added a proper agent picker accessible from the TUI. The picker is a selection popup that lists all known sub-agent threads.
How it works
The picker is opened via open_agent_picker() which:
- Iterates all known thread event channels
- For each thread, fetches metadata (nickname, role, open/closed status) from the server
- Calls
upsert_agent_picker_thread()to build the list - Sorts threads via
sort_agent_picker_threads()— open agents first, then by thread ID - Renders each entry with a status dot and formatted name
Status dots
agent_picker_status_dot_spans() renders visual indicators:
- Green
•— agent is alive/running - Dark gray
•— agent is closed/completed
Entry format
Each picker item shows:
• Ash [reviewer] (019bbb20...)
Where:
•is the status dot (green or gray)Ashis the nickname[reviewer]is the role(019bbb20...)is the abbreviated thread UUID
The picker supports search — typing filters by both name and UUID.
Snapshot test
A snapshot test agent_picker_item_name.snap verifies the rendering format.
---
5. Dead Agents in the Picker
What changed
PR #12570 (+189/-54 lines) ensures that completed/failed sub-agents remain visible in the agent picker rather than disappearing.
Why this matters
Before this PR, when a sub-agent finished its work and its thread closed, it vanished from the picker. This made it impossible to review what completed agents had done. Now:
mark_agent_picker_thread_closed()setsis_closed = trueon the entry instead of removing it- The sort order puts closed agents at the bottom (open agents first via
.cmp(&right.is_closed)) - Closed agents show a gray
•dot instead of green - You can still select a closed agent to view its transcript
Files changed
The changes are concentrated in:
codex-rs/tui/src/app.rs(+124/-47) — main picker logiccodex-rs/tui/src/multi_agents.rs(+45) — sort and status helperscodex-rs/tui/src/bottom_pane/— minor updates to selection popup infrastructure
---
6. Pending Child-Thread Approval Prompts
What changed
PR #12767 surfaces pending approval requests from sub-agent threads in the parent's TUI. This is critical for non-full-auto approval policies where sub-agents need human approval for shell commands or patches.
The problem it solves
Before this PR, if you were viewing the main thread and a sub-agent hit an approval gate (e.g., wants to run a shell command), you had no way of knowing. The sub-agent would silently block, waiting for approval that you couldn't see. You'd have to manually switch to each sub-agent thread to check.
How it works
Approval propagation
The PR also changed approval policy propagation (codex-rs/core/src/tools/handlers/multi_agents.rs, +24/-10). Previously, sub-agents were force-set to approval-policy: never. Now the parent's approval policy is inherited, so sub-agents respect the caller's requested approval level.
Pending approval tracking
A new PendingThreadApprovals widget (codex-rs/tui/src/bottom_pane/pending_thread_approvals.rs, 147 lines) tracks and renders approval states:
pub(crate) struct PendingThreadApprovals {
threads: Vec<String>, // thread IDs with pending approvals
}
The widget is placed above the composer for inactive threads and displays:
! Approval needed in Ash [reviewer]
! Approval needed in Birch [tester]
(switch to thread to review)
If more than 3 threads have pending approvals, it truncates with ....
State management
The approval state is managed through pending_interactive_replay.rs which tracks:
exec_approval_call_ids— pending shell command approvalspatch_approval_call_ids— pending patch approvalsrequest_user_input_call_ids— pending user input requestselicitation_requests— pending MCP elicitation requests
Events that can change pending state include: exec approval requests, patch approval requests, turn completions, user input answers, and elicitation events. The state refreshes whenever events arrive or the active thread changes.
---
Architecture Summary
┌─────────────────────────────────────────────────────┐
│ Codex CLI TUI │
│ ┌──────────────┐ ┌───────────────────────────────┐│
│ │ Agent Picker │ │ Chat / Transcript ││
│ │ • Ash [rev] │ │ (active thread view) ││
│ │ • Birch [test] │ │ ││
│ │ ○ Cedar │ │ ││
│ └──────────────┘ │ ! Approval needed in Birch ││
│ │ (switch to thread to review) ││
│ │ ┌─────────────────────────┐ ││
│ │ │ [composer input area] │ ││
│ │ └─────────────────────────┘ ││
│ └───────────────────────────────┘│
└─────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────┐
│ spawn_agents_on_csv │
│ │
│ CSV Input ──► Parse & Create Items ──► SQLite DB │
│ │
│ Job Runner Loop: │
│ ┌─────────────────────────────────────────────┐ │
│ │ while items remain: │ │
│ │ fill concurrency slots (max_concurrency) │ │
│ │ spawn sub-agents with templated prompts │ │
│ │ reap stale workers (timeout) │ │
│ │ collect finished results │ │
│ │ emit progress (rate, ETA) │ │
│ └─────────────────────────────────────────────┘ │
│ │
│ ──► Auto-export to Output CSV │
│ ──► Final status: completed | failed | cancelled │
└─────────────────────────────────────────────────────┘
---
Key Source Files
| File | Lines | Purpose |
|------|-------|---------|
| codex-rs/core/src/tools/handlers/agent_jobs.rs | ~1,227 | Main spawn_agents_on_csv and report_agent_job_result handlers, job runner loop, CSV parsing, template rendering |
| codex-rs/core/src/tools/spec.rs | ~257 added | Tool specifications for spawn_agents_on_csv, report_agent_job_result |
| codex-rs/core/src/agent/agent_names.txt | 100 | Tree-themed nickname list |
| codex-rs/core/src/agent/control.rs | ~256 added | Agent nickname reservation, spawn slot management |
| codex-rs/core/src/agent/guards.rs | ~172 added | Spawn depth limits, concurrency guards |
| codex-rs/state/migrations/0014_agent_jobs.sql | 38 | SQLite schema for jobs and items |
| codex-rs/state/src/model/agent_job.rs | 256 | Data model types (AgentJob, AgentJobItem, status enums) |
| codex-rs/state/src/runtime.rs | ~567 added | SQLite runtime operations for job CRUD |
| codex-rs/tui/src/multi_agents.rs | ~410 rewritten | Sub-agent TUI rendering, picker helpers |
| codex-rs/tui/src/app.rs | ~348 added | Agent picker, dead agent tracking, approval integration |
| codex-rs/tui/src/bottom_pane/pending_thread_approvals.rs | 147 | Pending approval widget |
| codex-rs/exec/src/event_processor_with_human_output.rs | 244 | Progress bar for codex exec mode |
| codex-rs/core/tests/suite/agent_jobs.rs | 424 | Integration tests |
---
Design Decisions Worth Noting
- Deterministic, single-call design: Earlier iterations had separate
run,resume,get-status, andexporttools. Review feedback led to collapsing everything into onespawn_agents_on_csvcall that runs to completion. This is simpler for the model to use and avoids non-deterministic monitoring loops.
- SQLite as the state backend: Job state is persisted in SQLite, which means jobs survive crashes. The
recover_running_items()function on startup reaps stale items from a previous run and marks them for retry.
- Nicknames as tree names: Short, memorable, thematically consistent. The "no repeats within a session" guarantee means you can always tell agents apart even with many concurrent workers.
- Approval policy inheritance: The decision to propagate the parent's approval policy to sub-agents (rather than forcing
never) is significant. It meansspawn_agents_on_csvrespects safety boundaries — if you're running inon-requestmode, workers will also ask for approval, and the TUI will surface those requests.
- Dead agent persistence in picker: A small but important UX decision. Completed agents aren't garbage-collected from the UI because their transcripts contain valuable information about what was done.
---
What's NOT Documented by OpenAI
As of 2026-02-25, none of this is in the official docs:
- The
developers.openai.com/codex/changelogstops at v0.99.0 - No mention of
spawn_agents_on_csvanywhere in the docs mirror - No mention of agent nicknames, the improved agent picker, or pending approval prompts
- The only multi-agent documentation is the Agents SDK + MCP guide (
codex/guides/agents-sdk.md), which covers the external orchestration pattern (Python + Agents SDK), not the built-in CLI primitive
The existing multi-agent docs describe how to use Codex as an MCP server called by external code. The spawn_agents_on_csv feature is the opposite: it's Codex itself orchestrating batch work internally, without any external SDK.
Where did you find it?
...
This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗