Skill Tracer
Skills › Scientific Agent Skills › arbor › report-template.md

report-template.md

42 lines · skills/arbor/references/report-template.md at 5f2bc03

Open on GitHub
1# Final report template
2
3Produce this when the run ends (budget spent, frontier exhausted, or progress
4stalled). The point is an honest, auditable account — not a victory lap. Keep it
5concise and grounded in the tree.
6
7```markdown
8# Arbor run report: [objective]
9
10## Result
11- **Best artifact**: [git branch/ref of M_best, e.g. `arbor/best`]
12- **Gate score**: [score of M_best] vs. initial [score of M_0] → delta [Δ]
13- **Final untouched evaluation**: [score/protocol, or explicitly not performed]
14- **Evidence**: [commit, evaluator/data versions, seeds, repetitions, raw logs]
15- **How to check it out**: `git checkout [ref]`
16- One-line summary of the change that won.
18## What was tried (audit trail)
19[Paste `python "$ARBOR_TREE" status` — the tree shows every direction,
20which were pruned, which merged, with dev/test scores.]
22## How understanding evolved
232-4 bullets tracing the main hypothesis shifts: which early nodes tested broad
24mechanisms, what they confirmed or ruled out, and how that reshaped later
25hypotheses. The story should explain *why* the final design looks the way it
26does — i.e. the constraints the run discovered.
28## Dev vs. test (overfitting check)
29- Nodes that improved **dev**: [count]
30- Nodes that passed the **test merge gate**: [count]
31- Comment on the gap: were there high-dev / low-test candidates? What did
32 rejecting them tell you? How many gate queries were made, and was the
33 same holdout reused for selection? An honest gap here is more trustworthy than a clean
34 "everything worked".
36## Open directions
37What you'd explore with more budget, and any direction that seemed to need a
38new high-level formulation rather than further refinement (HTR's known weak
39spot — flag it for the human).
40```
42Always leave `M_best` as a real, runnable artifact on a named git branch.

Final report template

Produce this when the run ends (budget spent, frontier exhausted, or progress stalled). The point is an honest, auditable account — not a victory lap. Keep it concise and grounded in the tree.

# Arbor run report: [objective]

## Result
- **Best artifact**: [git branch/ref of M_best, e.g. `arbor/best`]
- **Gate score**: [score of M_best] vs. initial [score of M_0] → delta [Δ]
- **Final untouched evaluation**: [score/protocol, or explicitly not performed]
- **Evidence**: [commit, evaluator/data versions, seeds, repetitions, raw logs]
- **How to check it out**: `git checkout [ref]`
- One-line summary of the change that won.

## What was tried (audit trail)
[Paste `python "$ARBOR_TREE" status` — the tree shows every direction,
which were pruned, which merged, with dev/test scores.]

## How understanding evolved
2-4 bullets tracing the main hypothesis shifts: which early nodes tested broad
mechanisms, what they confirmed or ruled out, and how that reshaped later
hypotheses. The story should explain *why* the final design looks the way it
does — i.e. the constraints the run discovered.

## Dev vs. test (overfitting check)
- Nodes that improved **dev**: [count]
- Nodes that passed the **test merge gate**: [count]
- Comment on the gap: were there high-dev / low-test candidates? What did
  rejecting them tell you? How many gate queries were made, and was the
  same holdout reused for selection? An honest gap here is more trustworthy than a clean
  "everything worked".

## Open directions
What you'd explore with more budget, and any direction that seemed to need a
new high-level formulation rather than further refinement (HTR's known weak
spot — flag it for the human).

Always leave M_best as a real, runnable artifact on a named git branch.

Raw lines 1 to 42