Skill Tracer
Skills › Scientific Agent Skills › arbor › executor-brief.md

executor-brief.md

72 lines · skills/arbor/references/executor-brief.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Executor brief template
2
3Each executor is a short-lived subagent that tests **one** hypothesis in an
4isolated git worktree and returns structured evidence. Use the host's supported
5subagent/worktree facility, or create a distinct
6worktree with `git worktree add`. Dispatch independent siblings in parallel
7when resources permit. Only the coordinator writes the shared tree.
8
9Fill in the bracketed parts. Keep the brief tight: the executor needs the
10hypothesis, the context that lets it implement well, and a crisp contract for
11what to return — nothing more.
13---
15```
16You are an Arbor executor. Test ONE hypothesis in an isolated git worktree and
17return structured evidence. Do not change the hypothesis — your job is to give
18the coordinator clean evidence about THIS claim, even if it turns out false.
20HYPOTHESIS (h_n):
21 [the falsifiable claim, e.g. "Aggregating K=5 independent rollouts by an
22 evidence dossier recovers correct answers that majority vote discards."]
24CURRENT BEST ARTIFACT (M_best):
25 [path or git ref of the current best, e.g. branch `arbor/best` — start from this]
27RELEVANT INSIGHTS FROM THE TREE (assume these; build on them, don't re-litigate):
28 [ancestor + sibling insights, e.g. "Verification is not the bottleneck;
29 candidate coverage is. Search-augmented judging overfits dev questions."]
31OBJECTIVE & METRIC:
32 [O and direction, e.g. "Maximize BrowseComp answer accuracy."]
34DEVELOPMENT EVALUATOR (E_dev) — run this to score your candidate:
35 [exact command, e.g. `python eval.py --split dev --n 50`]
37WHAT TO DO:
38 1. Create/confirm an isolated worktree from M_best so you don't touch other
39 experiments or the current best.
40 2. Implement the MINIMAL change that realizes the hypothesis. You may edit,
41 debug, and rerun freely to get a working implementation — but keep the
42 change bound to this hypothesis. If the metric stalls, fix YOUR code; do
43 not pivot to a different idea.
44 3. Run E_dev and record the score. Use the predeclared repetitions if it is
45 noisy. Preserve the exact command,
46 evaluator/data versions, seeds and raw output; report execution failure
47 without inventing a numeric score.
48 4. Commit the artifact on a clearly named branch.
50RETURN EXACTLY THIS (your final message IS the data the coordinator reads):
51 - dev_score: <finite number from E_dev, or null for execution failure>
52 - result: <1-3 sentences of factual outcome — what the change did>
53 - insight: <the reusable lesson: WHY this result supports / weakens /
54 bounds the hypothesis. This is the most valuable output —
55 make it a constraint future experiments can use, not a restatement
56 of the score.>
57 - branch_ref: <git branch/commit/worktree path holding the artifact>
59Do NOT run the held-out test evaluator — that is the coordinator's merge gate.
60```
62---
64After the executor returns, the coordinator records it with:
66```bash
67python "$ARBOR_TREE" set-evidence --node <id> \
68 --dev-score <n> --result "..." --insight "..." --branch-ref "<ref>"
69```
71Omit `--dev-score` when evaluation failed; preserve the failure and log path in
72`--result`. Then abstract the lesson upward with `tree.py propagate`.

Executor brief template

Each executor is a short-lived subagent that tests one hypothesis in an isolated git worktree and returns structured evidence. Use the host's supported subagent/worktree facility, or create a distinct worktree with git worktree add. Dispatch independent siblings in parallel when resources permit. Only the coordinator writes the shared tree.

Fill in the bracketed parts. Keep the brief tight: the executor needs the hypothesis, the context that lets it implement well, and a crisp contract for what to return — nothing more.


You are an Arbor executor. Test ONE hypothesis in an isolated git worktree and
return structured evidence. Do not change the hypothesis — your job is to give
the coordinator clean evidence about THIS claim, even if it turns out false.

HYPOTHESIS (h_n):
  [the falsifiable claim, e.g. "Aggregating K=5 independent rollouts by an
   evidence dossier recovers correct answers that majority vote discards."]

CURRENT BEST ARTIFACT (M_best):
  [path or git ref of the current best, e.g. branch `arbor/best` — start from this]

RELEVANT INSIGHTS FROM THE TREE (assume these; build on them, don't re-litigate):
  [ancestor + sibling insights, e.g. "Verification is not the bottleneck;
   candidate coverage is. Search-augmented judging overfits dev questions."]

OBJECTIVE & METRIC:
  [O and direction, e.g. "Maximize BrowseComp answer accuracy."]

DEVELOPMENT EVALUATOR (E_dev) — run this to score your candidate:
  [exact command, e.g. `python eval.py --split dev --n 50`]

WHAT TO DO:
  1. Create/confirm an isolated worktree from M_best so you don't touch other
     experiments or the current best.
  2. Implement the MINIMAL change that realizes the hypothesis. You may edit,
     debug, and rerun freely to get a working implementation — but keep the
     change bound to this hypothesis. If the metric stalls, fix YOUR code; do
     not pivot to a different idea.
  3. Run E_dev and record the score. Use the predeclared repetitions if it is
     noisy. Preserve the exact command,
     evaluator/data versions, seeds and raw output; report execution failure
     without inventing a numeric score.
  4. Commit the artifact on a clearly named branch.

RETURN EXACTLY THIS (your final message IS the data the coordinator reads):
  - dev_score: <finite number from E_dev, or null for execution failure>
  - result:    <1-3 sentences of factual outcome — what the change did>
  - insight:   <the reusable lesson: WHY this result supports / weakens /
               bounds the hypothesis. This is the most valuable output —
               make it a constraint future experiments can use, not a restatement
               of the score.>
  - branch_ref: <git branch/commit/worktree path holding the artifact>

Do NOT run the held-out test evaluator — that is the coordinator's merge gate.

After the executor returns, the coordinator records it with:

python "$ARBOR_TREE" set-evidence --node <id> \
  --dev-score <n> --result "..." --insight "..." --branch-ref "<ref>"

Omit --dev-score when evaluation failed; preserve the failure and log path in --result. Then abstract the lesson upward with tree.py propagate.

Raw lines 1 to 72
Starts an agent subagent :3
Starts an agent Dispatch :6
Runs code python eval.py :35