[RFC] Towards Self-Evolving Agents: Interactive Instruction Distillation (/learn) and Rule Metabolism for AGENTS.md

[RFC] Towards Self-Evolving Agents: Interactive Instruction Distillation (/learn) and Rule Metabolism for AGENTS.md

1. Executive Summary & Core Philosophy

As coding agents tackle increasingly complex, multi-week software projects, they face a fundamental evolutionary bottleneck: The Amnesia-vs-Accumulation Dilemma.

1. Amnesia (The Groundhog Day Problem): Without durable learning, each new session starts from scratch. Developers find themselves repeatedly steering the agent on the same repository quirks, local sandbox boundaries, tool arguments, and testing gates.
2. Accumulation (The Memory Swamp & Unchecked Degradation): Naive instruction systems that only accumulate text without an explicit promotion/retirement lifecycle risk Instruction Drift and Semantic Bloat—obsolete workarounds, conflicting rules, and unverified AI speculation can gradually degrade reasoning quality. Recent empirical research (e.g., Microsoft Research's SkillOpt, arXiv:2605.23904) demonstrates that ungated, unsupervised self-edit loops can rapidly degrade model performance (collapsing from 0.554 to 0.026 in single-seed benchmarks) by overfitting to superficial patterns.

The Guiding Principle: Instruction Metabolism & Rules as Hypotheses

**True agent self-evolution is not about remembering everything; it is about metabolism—treating every rule as a falsifiable hypothesis, enforcing selective extraction, progressive refinement, and the active retirement of superseded rules under human consensus.**

Codex already has a sophisticated two-phase memory architecture (codex-rs/memories) with consolidation, pruning, deduplication, and stale-evidence cleanup. This RFC proposes extending that general philosophy into an explicit, user-visible, auditable lifecycle for authoritative instructions, bridging passive memories with developer workflows and AGENTS.md.

---

2. Analysis of Codex's Current Architecture (codex-rs)

OpenAI Codex already possesses a solid foundational memory harness:

• Phase 1 (Rollout Extraction): Asynchronously extracts raw_memory, rollout_summary, and rollout_slug into the local SQLite state DB.
• Phase 2 (Global Consolidation): An internal background agent uses a Git-backed workspace under ~/.codex/memories/ to maintain MEMORY.md, memory_summary.md, and local skills/.

The existing consolidation path already performs meaningful lifecycle work: incremental updates, no-op decisions, stale-input pruning, deduplication, and removal or rewriting of memory no longer supported by evidence. The gap is therefore not “Codex lacks forgetting.” The gap is that this lifecycle is largely passive and memory-scoped, while authoritative operating instructions remain outside an equivalent user-directed promotion/revision flow.

The Critical Structural Gaps

1. Isolation from Authoritative Instructions (AGENTS.md)

• codex-memories-write updates the memory workspace, but does not intentionally promote validated session learnings into global ~/.codex/AGENTS.md or repository-level ./AGENTS.md.
• Project constraints, architectural invariants, and team boundaries therefore remain outside the current memory consolidation lifecycle unless a developer edits them separately.

1. Absence of In-Session Human-Agent Consensus (/learn)

• Memory consolidation is primarily asynchronous and startup-triggered.
• Developers cannot intentionally run a distillation pass immediately after a difficult debugging session, repeated correction, or architectural decision and review the proposed durable rule changes before they are applied.

1. Lack of an Explicit, User-Visible Lifecycle for Authoritative Instructions

• AGENTS.md rules do not currently have a first-class interactive lifecycle such as ADD, NARROW, REPLACE, or RETIRE driven by session evidence and confirmed by the user.
• Without such a lifecycle, teams must manually reconcile stale, overlapping, or overly broad instructions over time.

---

3. Proposed Architecture & System Design

                     ┌────────────────────────────────┐
                     │    Active Codex CLI Session    │
                     │  (User corrections & debug)    │
                     └───────────────┬────────────────┘
                                     │
                 User calls `/learn` │ (or Agent suggests at turn completion)
                                     ▼
        ┌────────────────────────────────────────────────────────┐
        │        Interactive Distillation Sub-Agent              │
        │   (memory/instruction distillation pass)               │
        └────────────────────────────┬───────────────────────────┘
                                     │
                  ┌──────────────────┴──────────────────┐
                  ▼                                     ▼
     [ Rule Metabolism Engine ]            [ Epistemic Layering ]
     - Add: Net-new invariant              - Workspace: `./AGENTS.md`
     - Narrow: Restrict broad rule          - Global: `~/.codex/AGENTS.md`
     - Replace: Supersede old pattern      - Skill: `skills/<name>/SKILL.md`
     - Retire: Remove obsolete rule        - Memory: `MEMORY.md`
                  │
                  ▼
   ┌─────────────────────────────────────────────────────────────┐
   │            Human-in-the-Loop Review & Staged Diff           │
   │  "Proposed update to ./AGENTS.md (-2 lines, +3 lines): [Y/n]"│
   └──────────────────────────────┬──────────────────────────────┘
                                  │ (Approved)
                                  ▼
   ┌─────────────────────────────────────────────────────────────┐
   │         Atomic Mutation to Local/Global Instruction File    │
   └─────────────────────────────────────────────────────────────┘

3.1 Interactive Distillation Protocol (/learn / /distill)

• Trigger: Invocable by the developer via /learn (or suggested by Codex upon resolving a multi-turn error).
• Scope Extraction: Inspects the active turn transcript and execution diff for:
• Explicit user corrections and workflow preferences.
• Validated failure shields (Symptom → Root Cause → Fix → Verification).
• Platform/environment constraints (e.g. TCC permissions, required flags, build timeouts).
• Architectural invariants or contribution rules that have become stable enough to deserve promotion.

The command defaults to a review-first flow: distill candidates, show their proposed destination, lifecycle operation, and review criteria, and only write after explicit user approval.

3.2 The Four Operations of Rule Metabolism & Rules as Hypotheses

Every authoritative-rule candidate is treated as a falsifiable hypothesis rather than an unquestioned permanent decree, and is classified into one of four discrete lifecycle operations:

1. ADD: Introduce a verified, net-new constraint or workflow invariant. (Optionally tagged with a falsifiable_condition and review_after timestamp).
2. NARROW: Restrict the scope of an existing rule found to be overly broad.
3. REPLACE: Atomically supersede an older, sub-optimal rule with a validated newer approach.
4. RETIRE: Remove an obsolete rule whose underlying conditions no longer apply (e.g., when an upstream CLI bug is fixed or a library migration completes).

These operations are intended as an auditable interaction model, not necessarily a permanent on-disk schema. The important property is that Codex must distinguish “add another instruction” from “this evidence changes, restricts, or invalidates an existing instruction.”

3.3 Epistemic Layering & Fenced Machine Sections

Not every useful lesson belongs in AGENTS.md, and machine-distilled text should not corrupt human-authored architectural prose.

• Ephemeral Episode → Kept in local session context only; discarded on session close.
• Durable but Non-Authoritative Knowledge → MEMORY.md / memory artifacts.
• Workspace Invariant / Team Operating Rule → ./AGENTS.md (version-controlled with project code).
• Fenced Block Isolation: Distilled instructions can land in a dedicated, clearly delimited block (e.g., <!-- CODEX-LEARNED:START --> ... <!-- CODEX-LEARNED:END -->) subordinate to hand-authored rules to preserve clean structural boundaries.
• Global Developer Preference / Operating Rule → ~/.codex/AGENTS.md.
• Reusable Multi-Step Procedure → Synthesized into a dedicated skills/<name>/SKILL.md.

3.4 Variance-Aware Evaluation vs. Deterministic Constraints

• Deterministic Operational Constraints (e.g., specific CLI flags --network-timeout 60, required path prefixes, formatters): Verified directly by the immediate tool execution/test pass during the session.
• Probabilistic / Behavioral Strategy Rules (e.g., prompt reframing, high-level reasoning heuristics): Treated as empirical hypotheses; when evaluated, should be checked across multiple runs ($k=5$) against a golden regression suite rather than single-run stochastic noise.

3.5 Standing Surface Budget & Adherence-Probed Capacity Gating

Every review of an instruction file inherently suffers from build-bias: rules feel like assets, leading humans and agents to continuously accumulate instructions while rarely retiring older ones. However, as the standing instruction surface expands, model adherence to any individual rule steadily degrades.

To turn retirement into an unavoidable, first-class outcome, the architecture incorporates three complementary mechanics:

1. The Forced-Choice Moment (Trade-Off Friction):

• When a newly proposed ADD would breach the active standing surface budget, the distillation flow halts silent addition and enforces an explicit trade-off prompt:

> "Standing surface budget exceeded. Promotion requires an explicit RETIRE or MERGE of existing rules to make room for ADD."

• This friction breaks the hoarding instinct and ensures retirement is a required operational step rather than an optional chore.

1. Adherence-Probed Capacity Gating (Empirical Evidence over Task Breadth):

• Rather than allowing an agent to arbitrarily self-expand its budget based on workspace task breadth (which conflates what a user wants loaded with what the model actually follows, effectively letting the auditor grade its own homework), the capacity ceiling is bounded by measured adherence evidence.
• The workspace probes multi-rule compliance with all standing rules loaded jointly. If empirical probes demonstrate quiet rule-dropping under load, the budget ceiling is hard-locked, mandating the pruning of mediocre rules before new invariants can land.

1. Path-Scoped & Skill Offloading:

• Domain-specific constraints (e.g., prose styling, database migrations, specific framework protocols) are offloaded to directory-scoped rules or on-demand skills, keeping the root AGENTS.md strictly minimal regardless of user versatility.

3.6 Fail-Closed Anchor Guard & Interactive Semantic Re-Anchoring

A subtle failure mode in real-world human-agent co-editing is Anchor Drift leading to Zombie Rules:

• Developers frequently hand-edit AGENTS.md between sessions (rephrasing rules, moving lines, or tweaking formatting).
• When /learn subsequently attempts a REPLACE or RETIRE operation, the verbatim target text no longer matches.
• A naive implementation might silently fall back to ADD into the fenced machine section to avoid a hard error. This reports "Applied", but leaves the mutated old rule intact alongside the new rule, creating contradictory duplicates and stealthily inflating the standing surface.

To prevent this silent degradation, the engine enforces three safeguards:

1. Strict Fail-Closed Invariant: REPLACE and RETIRE must never silently degrade into ADD. If a target rule cannot be anchored with certainty, the mutation must fail-fast rather than creating a duplicate.
2. Interactive Semantic Re-Anchoring: When an exact verbatim match fails, the engine runs a fuzzy/semantic similarity search across AGENTS.md and surfaces drifted candidate lines for explicit user confirmation (Supersede this drifted line? [Y/n/select/abort]).
3. Machine-Block Stable Identifiers: Rules generated in the machine-managed fenced block carry lightweight stable identifiers (e.g., <!-- id: py-formatter -->), ensuring unambiguous targeting even if explanatory text inside the block has been lightly modified.

---

4. Proposed User Experience (CLI Walkthrough)

$ codex
> [User]: Fix the network timeout when calling the internal build cluster on macOS.
> [Codex]: ... (triages sandbox flags, discovers missing `--network-timeout 60` and TCC bypass, fixes issue and verifies) ...
> [User]: /learn

[Codex Distillation Engine]
Analyzing session transcript (Turn 1 to Turn 7)...
Checking standing surface budget (Current: 38/40 rules, 4.8KB / 5.0KB)...

Proposed Rule Updates:

1. [Project Rule -> ./AGENTS.md] (ADD)
   Operation: ADD (Fenced Block)
   Content:   When invoking internal cluster builds, always pass `--network-timeout 60` to accommodate macOS proxy latency.
   Condition: Retire if internal cluster proxy latency is resolved upstream.
   Review:    2026-10-01

2. [Project Rule -> ./AGENTS.md] (REPLACE)
   Operation: REPLACE
   - Do not run local cluster integration tests.
   + Integration tests may be run locally if `CLUSTER_DEV_MODE=1` is set.
   [Anchor Check]: Exact anchor match verified at ./AGENTS.md:L42.

3. [Global Preference -> ~/.codex/AGENTS.md] (NARROW)
   Operation: NARROW
   Target:    "Always format Python code with flake8."
   [Anchor Drift Warning]: Verbatim target not found (user hand-edited file).
   Found drifted candidate at ~/.codex/AGENTS.md:L18 (94% match):
     - "Always format Python code with flake8 across all projects."
   Supersede drifted line L18 with new narrowed rule? [Y/n/edit]: Y
   + Default to `ruff format` across Python projects unless the repository explicitly pins a different formatter.

[Surface Budget Guard]: Proposed updates maintain headroom (39/40 rules, 4.9KB / 5.0KB).

Apply changes to instruction files? [Y/n/edit]:

---

5. Engineering Feasibility & Possible Implementation Strategy

The exact ownership boundary should follow the maintainers' preferred architecture. The proposal does not depend on introducing a specific new crate.

A plausible implementation could:

1. Reuse Existing Memory Infrastructure Where Appropriate

• Reuse existing prompt rendering, secret-redaction, evidence hygiene, and workspace-diff helpers from codex-memories-write where those abstractions fit.

1. Add an Interactive Distillation Component

• Given the active transcript, current effective instruction files, and relevant execution diff, produce structured candidates such as {destination, operation, evidence, proposed_diff, review_after, falsifiable_condition}.
• This could live in codex-core, the existing memories write path, or a dedicated component depending on maintainership boundaries.

1. Add a First-Class CLI/TUI Binding

• Register /learn and/or /distill as explicit user-facing commands.
• Prefer a review-first flow over silent writes.

1. Generate Minimal, Transactional Instruction Diffs

• Avoid rewriting whole AGENTS.md files when a narrow change is sufficient.
• Preserve user-authored structure and unrelated sections.
• Place distilled rules into delimited sections or surgically edit existing lines.
• Reject or surface ambiguous edits instead of silently resolving them.

1. Guard Against Instruction Bloat and Authority Escalation

• Treat “no durable change” as a valid result.
• Require stronger evidence for global rules than workspace-local rules.
• Detect semantic overlap with existing instructions before proposing ADD.
• Never silently promote transient task state, speculative conclusions, secrets, or unverified fixes into authoritative instructions.

---

6. Related Work & Literature

• #34668 — Explicit, auditable promotion of operator feedback into Memories: Focuses on memory CRUD with provenance. This RFC extends this to authoritative instruction surfaces (AGENTS.md).
• #32748 — User-facing workflow retrospective / insights: Asynchronous retrospective recommendations. This RFC targets the in-session transactional /learn path.
• #21932 — Allow /init to update existing AGENTS.md: Initial repository generation vs. continuous empirical distillation.
• SkillOpt (Microsoft Research, arXiv:2605.23904): Empirically frames agent instructions as optimizable weights and proves that un-gated self-evolution leads to catastrophic degradation, validating the necessity of strict human review and held-out validation gates.
• Agent Workspace Architecture Patterns (jimy-r/agent-workspace-architecture): Documents practical patterns for heavily-instructed workspaces—specifically treating scaffolds as hypotheses and enforcing variance-floor regression checks.

The intended composition is:

passive memory consolidation
        +
explicit memory promotion / management
        +
user-visible retrospectives
        +
interactive authoritative-rule distillation (with hypothesis gating)
        ↓
safer long-term agent adaptation

---

7. Conclusion

Codex already contains much of the hard infrastructure needed for durable learning: persisted rollouts, structured extraction, consolidation agents, a Git-backed memory workspace, pruning, deduplication, skills, and authoritative AGENTS.md instruction chains.

The missing piece is a deliberate promotion boundary between “the agent learned something from experience” and “this should now govern future work as an authoritative rule.”

An interactive /learn / /distill flow could make that boundary explicit, reviewable, and auditable:

experience → evidence → candidate hypothesis → lifecycle operation → human-reviewed diff → durable instruction

That would let Codex evolve across long-running projects without turning either memory or AGENTS.md into an unbounded accumulation of historical text.