Agent should analyze reference component trees before attempting visual/layout fixes

Summary

During a UI migration/fix task, the agent was given a concrete reference implementation and screenshots showing a visual alignment issue. Instead of first reconstructing the reference component tree, layout hierarchy, clipping behavior, gravity/alignment, margins, and animation layer, the agent treated the task as a simple overlay/offset problem and repeatedly adjusted image sizes and translations.

This led to several bad outcomes:

• The agent made visual/layout changes by trial and error.
• The agent ignored the most important evidence: the reference implementation's component hierarchy and coordinate system.
• When the user pointed out alignment problems, the agent continued tuning offsets instead of stopping to re-derive the layout model.
• The agent wrote validation checks that encoded its own wrong implementation assumptions rather than the expected behavior from the reference implementation.

Expected behavior

When a user provides an existing reference implementation for a UI/layout issue, especially one involving visual alignment, the agent should:

1. Treat the reference implementation as the primary source of truth.
2. Reconstruct the reference component tree before editing code.
3. Identify parent/child layout relationships, clipping/overflow rules, gravity/alignment, margins, transforms, and animation ownership.
4. Map those structures to the target framework's component tree.
5. Only then make scoped code changes.
6. If the user says the result is still misaligned, pause and re-check the layout model instead of continuing to tune constants.
7. Ensure tests/validation assert the reference-derived behavior, not the agent's current guess.

Observed behavior

The agent skipped the component-tree analysis step and made repeated modifications to image size and position. It also created validation around the guessed approach. This was especially problematic because the correct fix depended on which container owned the overlay, how the reference layout aligned it, and which part of the component tree participated in selection animation.

Why this matters

This failure mode is not limited to one UI framework. It can affect any task where the correct change depends on structural reasoning rather than local patching:

• UI migration between frameworks
• Visual parity fixes
• Animation/layout bugs
• Component hierarchy refactors
• Accessibility or hit-test behavior tied to parent containers

For these tasks, local trial-and-error patches can actively make the implementation worse and frustrate users.

Suggested improvement

Add a behavioral guardrail for reference-driven visual/layout tasks:

• If the user provides a reference implementation, require a short component-tree/layout analysis before edits.
• If feedback says alignment/position is wrong after a patch, require the agent to stop tuning constants and re-analyze the hierarchy.
• Encourage deriving validation from the reference structure, not from the attempted implementation.
• Consider detecting repeated visual correction feedback as a signal to switch from patching mode to systematic layout analysis mode.

This is a self-reported failure from an interactive Codex session.