Self-improvement: the bounded auto-iterate loop

This is the loop that reviews a skill, evaluates it, proposes a fix, applies it, and re-evaluates, repeating until the skill passes or a round bound is hit. It is the most autonomous mode the runner offers, so it carries the most guardrails: it is opt-in, calibrated to what is at stake, fully logged, and bounded.

The benchmark is a guardrail, never the judge. The human stays the judge. A green run means the change cleared the bar the loop was given, not that the change is correct, and the loop's job is to do the mechanical iteration a human would otherwise do by hand and then hand back a fix plus the evidence for it.

When to run it, and how hard

The loop is opt-in. It never starts on its own, because applying changes to a skill in a loop is a stronger action than reporting findings, and the user decides when that is warranted.

Calibrate the aggressiveness to the stakes. A throwaway skill the user is still shaping can take a longer loop and a looser bar, because a wrong iteration costs little and is easy to throw away. A skill that other skills already depend on, or one that is shipped and in use, takes a short loop, a strict pass bar, and a close human read of every applied change, because a regression there propagates. Agree the round bound and the pass condition with the user before the first round, and write both into the run report so the run is auditable against the terms it was given.

The loop

Each round runs four beats:

1. Review. Invoke the bmad-toolsmith skill to review the skill and collect its findings. On rounds after the first, review again rather than trusting the prior findings, because the last fix may have moved something.

1. Eval. Run the modes that apply to this skill: quality against its rubric where one exists, variant to settle a finding that a section does no work, baseline to confirm the skill still beats the bare model. The review says what looks wrong; the eval says whether it measurably is. A finding the eval cannot confirm is a candidate to note for a human, not to auto-fix.

1. Propose a fix. From the confirmed findings, propose one concrete change. Address the cause the finding names rather than the single case that exposed it (see generalizing, below). Keep the change small enough that the next eval can attribute the delta to it; a round that rewrites five things at once cannot tell you which one moved the score.

1. Apply and re-eval. Apply the proposed change, then re-run the eval from beat 2 and compare. A round that improves the score and breaks nothing else is kept; a round that regresses any mode is reverted before the next round, because an applied change that made things worse is not a base to build on.

Stop when the pass condition is met or the round bound is reached, whichever comes first. The bound is a hard stop: hitting it without passing ends the loop and reports the best state reached, it does not earn extra rounds.

The trail is the run folder

Each round's proposed fix, scores and transcripts are already on disk in the run folder, so a reviewer reads the rounds back in order and sees what was tried, what each attempt did to the numbers, and why the loop stopped; note a reverted round and its reason in the round's report.

Generalize to intent, do not overfit to the case

The failure that ends most auto-iterate loops is fixing the example instead of the cause. A case fails because the skill mishandled a class of input; patching the skill to special-case that one input passes the case and leaves the class broken, and often the patch is a hardcoded branch that makes the skill worse. Read each finding as a representative of an intent category and fix the category. A case where the skill invented a fact absent from the source is not "handle this memo," it is "the skill does not ground its output in the provided source," and the fix belongs at that level.

When a proposed fix reaches for ALL-CAPS ALWAYS or NEVER or a stack of MUSTs, treat that as a warning sign. Shouting at the model is usually a sign the fix is patching a symptom; a sharper outcome statement or a small worked example generalizes where a louder rule does not. Prefer the version that explains the reasoning over the version that issues the command.

Why each guard is here