Skill Tracer
Skills › gstack › captain › flaky-ledger.md

flaky-ledger.md

25 lines · captain/templates/flaky-ledger.md at 3f79d81

Open on GitHub
1# Flaky test ledger
2
3The daily flaky-fix pass reads this file. Append one line per flaky event under **Open**.
4The fix pass moves an entry to **Fixed** with its PR, or to **Won't fix** with a reason.
5
6Line format:
7`- <date> | <case or test file> | <tier: paid/free/evaluator> | PR #<n> @ <head12> | failed <run id> | rerun passed <run id or "measured N/10"> | <symptom>`
8
9Sources: the gate runs' flake-records artifacts (every case the in-run rerun turned
10green) and the free suites' retried cases. Never a case that failed twice (that is a
11real failure), never a harness, infra or incomplete red.
13A fix changes the cause: a forced probe or a regression that is red on BASE and green
14with the fix. Weakened assertions, raised thresholds, added trials or samples, changed
15case kinds, skipped cases and longer timeouts are not fixes.

Flaky test ledger

The daily flaky-fix pass reads this file. Append one line per flaky event under Open. The fix pass moves an entry to Fixed with its PR, or to Won't fix with a reason.

Line format: - <date> | <case or test file> | <tier: paid/free/evaluator> | PR #<n> @ <head12> | failed <run id> | rerun passed <run id or "measured N/10"> | <symptom>

Sources: the gate runs' flake-records artifacts (every case the in-run rerun turned green) and the free suites' retried cases. Never a case that failed twice (that is a real failure), never a harness, infra or incomplete red.

A fix changes the cause: a forced probe or a regression that is red on BASE and green with the fix. Weakened assertions, raised thresholds, added trials or samples, changed case kinds, skipped cases and longer timeouts are not fixes.

Raw lines 1 to 16
17## Open
18> Last fix pass: <thread> (<date>); see reports/<thread>.md. New entries go at the top,
19> worst (most repeated) first.

Open

Last fix pass: <thread> (<date>); see reports/<thread>.md. New entries go at the top, worst (most repeated) first.
Raw lines 17 to 20
21## Fixed
22- <date> | <case> | fixed by #<n> (<one line: the cause>)

Fixed

  • <date> | <case> | fixed by #<n> (<one line: the cause>)
Raw lines 21 to 23
24## Won't fix
25- <date> | <case> | <reason, and who decided>

Won't fix

  • <date> | <case> | <reason, and who decided>
Raw lines 24 to 25