eval-first.md
19 lines · skills/bmad-toolsmith/approaches/eval-first.md at bd70c64
Most marks come from matching words in the file, so treat them as pointers.
Eval-first
For an output that can be checked objectively and a user who wants proof. Its outcome: the cases exist before the skill does, so the skill is built to pass them and the passing run is on record.
Raw lines 1 to 4Cases before code
From the discovery, write {target}/evals/cases.json ({id, input, rubric[], state_prefix, files[]}). Inputs are the user's real requests where they exist, as messy as they were typed. Each rubric entry is a PASS or FAIL condition over the output, the transcript or the files, never a quality adjective. A state_prefix reaches a mid-flow turn in one shot. Show the file and confirm it with the user, because cases they do not believe in prove nothing to them.
Smallest skill
Scaffold with uv run {skill-root}/scripts/init_skill.py --name <name> --dest <parent folder> --shape <shape> --description "<approved description>", adding --bmod <record> --source <update_source> when the read-back registers it as a member, and load shapes/<shape>/shape.md, then the module shape the registration line names, if any. Write SKILL.md to canon.md, aimed at the cases, never to the rubric word for word, because a skill that quotes its grader passes the test and fails the user.
Then
Tell the user which run folder holds the passing run, and which cases still fail if they stopped early. Load ship.md.