Skill Tracer
Skills › BMAD Method › bmad-toolsmith › eval-first.md

eval-first.md

19 lines · skills/bmad-toolsmith/approaches/eval-first.md at bd70c64

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Eval-first
2
3For an output that can be checked objectively and a user who wants proof. Its outcome: the cases exist before the skill does, so the skill is built to pass them and the passing run is on record.
4

Eval-first

For an output that can be checked objectively and a user who wants proof. Its outcome: the cases exist before the skill does, so the skill is built to pass them and the passing run is on record.

Raw lines 1 to 4
5## Cases before code
6
7From the discovery, write `{target}/evals/cases.json` (`{id, input, rubric[], state_prefix, files[]}`). Inputs are the user's real requests where they exist, as messy as they were typed. Each rubric entry is a PASS or FAIL condition over the output, the transcript or the files, never a quality adjective. A `state_prefix` reaches a mid-flow turn in one shot. Show the file and confirm it with the user, because cases they do not believe in prove nothing to them.
8

Cases before code

From the discovery, write {target}/evals/cases.json ({id, input, rubric[], state_prefix, files[]}). Inputs are the user's real requests where they exist, as messy as they were typed. Each rubric entry is a PASS or FAIL condition over the output, the transcript or the files, never a quality adjective. A state_prefix reaches a mid-flow turn in one shot. Show the file and confirm it with the user, because cases they do not believe in prove nothing to them.

Raw lines 5 to 8
9## Smallest skill
11Scaffold with `uv run {skill-root}/scripts/init_skill.py --name <name> --dest <parent folder> --shape <shape> --description "<approved description>"`, adding `--bmod <record> --source <update_source>` when the read-back registers it as a member, and load `shapes/<shape>/shape.md`, then the module shape the registration line names, if any. Write `SKILL.md` to `canon.md`, aimed at the cases, never to the rubric word for word, because a skill that quotes its grader passes the test and fails the user.

Smallest skill

Scaffold with uv run {skill-root}/scripts/init_skill.py --name <name> --dest <parent folder> --shape <shape> --description "<approved description>", adding --bmod <record> --source <update_source> when the read-back registers it as a member, and load shapes/<shape>/shape.md, then the module shape the registration line names, if any. Write SKILL.md to canon.md, aimed at the cases, never to the rubric word for word, because a skill that quotes its grader passes the test and fails the user.

Raw lines 9 to 12
Runs code scripts/init_skill.py :11
13## Loop
15Load `approaches/eval-loop.md` and run it to a stop condition.

Loop

Load approaches/eval-loop.md and run it to a stop condition.

Raw lines 13 to 16
17## Then
19Tell the user which run folder holds the passing run, and which cases still fail if they stopped early. Load `ship.md`.

Then

Tell the user which run folder holds the passing run, and which cases still fail if they stopped early. Load ship.md.

Raw lines 17 to 19