Skill Tracer
Skills › gstack › review › gate-integrity.md

gate-integrity.md

171 lines · review/gate-integrity.md at 3f79d81

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Gate Integrity catalog
2
3Adapted from the just-say-no-to-process-porn-and-ceremony skill's
4REWARD-HACKING-CATALOG; pattern names preserved because the names are the
5deterrent. Each entry names the pathology, gstack's countermeasure and where
6it lives. The ids are the `gate` values `/review` and `/ship` record on a
7finding; `gstack-gate-diff` emits the tagged ones in its read-level listing.
8Design and rationale: `docs/designs/HONEST_WORK_GATE_INTEGRITY.md`.
9
10Three levels, kept apart everywhere: **listed** (every hunk in a test, CI,
11runner/lint-config, snapshot or golden path; inventory, never a finding),
12**read** (listed hunks that carry a tag or remove lines; the agent reads each
13with one question: *does this relaxation silence the failure it was added
14for, or is it justified by a measurement or decision visible in this diff?*),
15**finding** (a read hunk that fails the question; one human decision per
16finding or coherent group). The control is the human disposition, not the
17detector: the tool inventories and tags, it never judges and never suppresses.

Gate Integrity catalog

Adapted from the just-say-no-to-process-porn-and-ceremony skill's REWARD-HACKING-CATALOG; pattern names preserved because the names are the deterrent. Each entry names the pathology, gstack's countermeasure and where it lives. The ids are the gate values /review and /ship record on a finding; gstack-gate-diff emits the tagged ones in its read-level listing. Design and rationale: docs/designs/HONEST_WORK_GATE_INTEGRITY.md.

Three levels, kept apart everywhere: listed (every hunk in a test, CI, runner/lint-config, snapshot or golden path; inventory, never a finding), read (listed hunks that carry a tag or remove lines; the agent reads each with one question: does this relaxation silence the failure it was added for, or is it justified by a measurement or decision visible in this diff?), finding (a read hunk that fails the question; one human decision per finding or coherent group). The control is the human disposition, not the detector: the tool inventories and tags, it never judges and never suppresses.

Raw lines 1 to 18
Names a skill ship :6
19## Tagged patterns (emitted by `gstack-gate-diff`)

Tagged patterns (emitted by gstack-gate-diff)

Raw lines 19 to 20
21### RH-1 — Test removed or disabled
23The agent deletes, skips or focuses a test instead of fixing the code it
24guards: a removed `test(`/`it(`/`def test_`/`func Test`/`@Test`, an added
25`.skip`, `.only`, `xit`, `xdescribe`, `@pytest.mark.skip`, `#[ignore]`,
26`t.Skip(`, `@Disabled`, `@Ignore`, `XCTSkip` or `@tag :skip`. The failure the
27test encoded is now invisible and the suite reads green. Countermeasure:
28`gstack-gate-diff` tags the hunk (tags are factual: a test deleted alongside
29its deleted source is tagged and the agent's judgment, not the tool, makes it
30a non-finding); `/review` Gate Integrity reads it and routes a finding to the
31four-option ASK (`review-gate-disposition`); `/ship` core checklist item 2.5
32runs the same scan and the PR body's `Gate edits:` line carries the kept
33reason. The testing specialist's "never accept discarded valid red tests" rule
34stays as the large-diff read.

RH-1 — Test removed or disabled

The agent deletes, skips or focuses a test instead of fixing the code it guards: a removed test(/it(/def test_/func Test/@Test, an added .skip, .only, xit, xdescribe, @pytest.mark.skip, #[ignore], t.Skip(, @Disabled, @Ignore, XCTSkip or @tag :skip. The failure the test encoded is now invisible and the suite reads green. Countermeasure: gstack-gate-diff tags the hunk (tags are factual: a test deleted alongside its deleted source is tagged and the agent's judgment, not the tool, makes it a non-finding); /review Gate Integrity reads it and routes a finding to the four-option ASK (review-gate-disposition); /ship core checklist item 2.5 runs the same scan and the PR body's Gate edits: line carries the kept reason. The testing specialist's "never accept discarded valid red tests" rule stays as the large-diff read.

Raw lines 21 to 35
Names a skill ship :31
36### RH-2 — Proof-class inflation
38A mock, fixture, replay or static read is presented as the proof a claim
39requires: "verified against production" when a stub answered, "the route is
40reachable" from a grep. No tag can see it; it is a prose check. Countermeasure:
41the preamble's Claims Need Evidence kernel (name the evidence kind you have and
42never substitute one for a kind the question requires), `/review`'s
43Verification of claims, and the Gate Integrity category's instruction to read
44RH-2 from the diff without a tag. `/ship` Step 16's "no completion claims
45without fresh verification evidence" is the publication gate.

RH-2 — Proof-class inflation

A mock, fixture, replay or static read is presented as the proof a claim requires: "verified against production" when a stub answered, "the route is reachable" from a grep. No tag can see it; it is a prose check. Countermeasure: the preamble's Claims Need Evidence kernel (name the evidence kind you have and never substitute one for a kind the question requires), /review's Verification of claims, and the Gate Integrity category's instruction to read RH-2 from the diff without a tag. /ship Step 16's "no completion claims without fresh verification evidence" is the publication gate.

Raw lines 36 to 46
Names a skill ship :44
47### RH-3 — Snapshot or golden regenerated without its owning test
49A snapshot (`__snapshots__/<name>.snap`, `*.snap`) changes while its
50conventional owner (the `<name>` test file) has no hunk in the candidate, so
51the new output was accepted rather than tested. `*.golden`, `testdata/` and
52project `Snapshot paths:` are tagged `RH-3?` (owner unresolved): listed and
53read, never inferred. Countermeasure: the tag plus the Gate Integrity read; a
54snapshot updated alongside its source is inventory, never a finding. gstack's
55own SKILL.md goldens regenerate on every template edit by design, so RH-3 is
56never auto-blocked; the PR-body line makes a product snapshot regeneration
57visible to the human.

RH-3 — Snapshot or golden regenerated without its owning test

A snapshot (__snapshots__/<name>.snap, *.snap) changes while its conventional owner (the <name> test file) has no hunk in the candidate, so the new output was accepted rather than tested. *.golden, testdata/ and project Snapshot paths: are tagged RH-3? (owner unresolved): listed and read, never inferred. Countermeasure: the tag plus the Gate Integrity read; a snapshot updated alongside its source is inventory, never a finding. gstack's own SKILL.md goldens regenerate on every template edit by design, so RH-3 is never auto-blocked; the PR-body line makes a product snapshot regeneration visible to the human.

Raw lines 47 to 58
59### RH-4 — Placeholder committed
61`todo!()`, `unimplemented!()`, `raise NotImplementedError`, `throw new
62Error("not implemented")` or `pass # TODO` lands in non-test code and the
63work is reported as done. Countermeasure: the tag admits the hunk through the
64content backstop even in product code; the plan-completion audit's
65`done/changed/partial` classification and `Linked to #N` never auto-close rule
66keep the obligation open.

RH-4 — Placeholder committed

todo!(), unimplemented!(), raise NotImplementedError, throw new Error("not implemented") or pass # TODO lands in non-test code and the work is reported as done. Countermeasure: the tag admits the hunk through the content backstop even in product code; the plan-completion audit's done/changed/partial classification and Linked to #N never auto-close rule keep the obligation open.

Raw lines 59 to 67
68### RH-5 — Tautological test
70A test that cannot fail: it asserts its own setup, mirrors the implementation
71line for line, or has no negative (nothing it rejects). Not a tag; read from
72the diff. Countermeasure: the test value card's `protects=` / `fails_when=`
73fields and the `no_claim=` sentence (what green does not prove), the
74`/ship` Step 7 authoring gate that rejects cards without them, and the Gate
75Integrity instruction to read RH-5 beside the tagged hunks.

RH-5 — Tautological test

A test that cannot fail: it asserts its own setup, mirrors the implementation line for line, or has no negative (nothing it rejects). Not a tag; read from the diff. Countermeasure: the test value card's protects= / fails_when= fields and the no_claim= sentence (what green does not prove), the /ship Step 7 authoring gate that rejects cards without them, and the Gate Integrity instruction to read RH-5 beside the tagged hunks.

Raw lines 68 to 76
Names a skill ship :74
77### RH-10 — Spec editing as progress
79Acceptance text in the plan or spec is rewritten so the delivered work matches
80it, then the item is reported complete against the new text. Countermeasure:
81`/ship`'s plan-completion audit diffs the plan's acceptance-bearing sections
82against their base-branch or first-commit baseline and reports an edited item
83as `ACCEPTANCE_EDITED`, never folded into `CHANGED`; the original obligation is
84still audited and the PR body names the scope change. Stated limit: an item
85rewritten in the commit that introduced the plan is invisible.

RH-10 — Spec editing as progress

Acceptance text in the plan or spec is rewritten so the delivered work matches it, then the item is reported complete against the new text. Countermeasure: /ship's plan-completion audit diffs the plan's acceptance-bearing sections against their base-branch or first-commit baseline and reports an edited item as ACCEPTANCE_EDITED, never folded into CHANGED; the original obligation is still audited and the PR body names the scope change. Stated limit: an item rewritten in the commit that introduced the plan is invisible.

Raw lines 77 to 86
Names a skill ship :81
87### RH-12 — Environment sniffing in product code
89Product code branches on `process.env.CI`, `NODE_ENV === 'test'`,
90`RAILS_ENV == 'test'`, `PYTEST_CURRENT_TEST` or `GITHUB_ACTIONS` so the code
91under test behaves differently from the code in production. Countermeasure:
92the tag outside test paths; the Gate Integrity read asks whether the branch
93exists to pass a check rather than to serve a user.

RH-12 — Environment sniffing in product code

Product code branches on process.env.CI, NODE_ENV === 'test', RAILS_ENV == 'test', PYTEST_CURRENT_TEST or GITHUB_ACTIONS so the code under test behaves differently from the code in production. Countermeasure: the tag outside test paths; the Gate Integrity read asks whether the branch exists to pass a check rather than to serve a user.

Raw lines 87 to 94
95### RH-13 — Suppression pragma added
97`eslint-disable`, `@ts-ignore`, `@ts-expect-error`, `# type: ignore`,
98`# noqa`, `#[allow(`, `// nolint`, `rubocop:disable` or `@SuppressWarnings`
99silences the checker instead of satisfying it. Countermeasure: the tag admits
100the hunk from any path; the read asks whether the suppressed diagnostic was
101the failure the change was supposed to fix.

RH-13 — Suppression pragma added

eslint-disable, @ts-ignore, @ts-expect-error, # type: ignore, # noqa, #[allow(, // nolint, rubocop:disable or @SuppressWarnings silences the checker instead of satisfying it. Countermeasure: the tag admits the hunk from any path; the read asks whether the suppressed diagnostic was the failure the change was supposed to fix.

Raw lines 95 to 102
103### RH-14 — Gate bypass
105A commit in range carries `[skip ci]`/`[ci skip]`, a workflow gains
106`continue-on-error: true` or `allow_failure: true`, or a script or hook adds
107`--no-verify`. Countermeasure: the tag; a removed workflow job is listed by the
108path floor and read (job-removal detection is not in v1). `/ship` never pushes
109with `--no-verify` and never force-pushes.

RH-14 — Gate bypass

A commit in range carries [skip ci]/[ci skip], a workflow gains continue-on-error: true or allow_failure: true, or a script or hook adds --no-verify. Countermeasure: the tag; a removed workflow job is listed by the path floor and read (job-removal detection is not in v1). /ship never pushes with --no-verify and never force-pushes.

Raw lines 103 to 110
Names a skill ship :108
111### RH-15 — Gate value relaxed
113In a test, eval, CI or config file a numeric gate moves in the relaxing
114direction inside one hunk: timeout, retry, retries, tolerance, epsilon, delta,
115maxDiff or jitter **up**; threshold, min_score, minimum, coverage or required
116**down**; or an exact-count assertion loosened to a bound (`toBe(3)` →
117`toBeGreaterThan(0)`). Unpaired values are tagged `RH-15?` and read. The tool
118prints any citation it finds beside the hunk (commit text naming a
119`ship-measure` report, a `gstack-shortcut(dec-*)` marker) and never
120suppresses on it, because `ship-measure` reports are gitignored and commit
121text is forgeable. Countermeasure: `/review`'s former blanket "eval threshold
122changes" suppression is narrowed to code-quality comments only; the agent
123resolves the citation read-only and pre-fills the Keep — justified option
124marked verified or unverified; `ship/sections/measure.md`'s measure-then-fix
125loop ("never raise a budget, lower a threshold, add a retry or skip a case to
126get a pass") is the native repair path.

RH-15 — Gate value relaxed

In a test, eval, CI or config file a numeric gate moves in the relaxing direction inside one hunk: timeout, retry, retries, tolerance, epsilon, delta, maxDiff or jitter up; threshold, min_score, minimum, coverage or required down; or an exact-count assertion loosened to a bound (toBe(3) → toBeGreaterThan(0)). Unpaired values are tagged RH-15? and read. The tool prints any citation it finds beside the hunk (commit text naming a ship-measure report, a gstack-shortcut(dec-*) marker) and never suppresses on it, because ship-measure reports are gitignored and commit text is forgeable. Countermeasure: /review's former blanket "eval threshold changes" suppression is narrowed to code-quality comments only; the agent resolves the citation read-only and pre-fills the Keep — justified option marked verified or unverified; ship/sections/measure.md's measure-then-fix loop ("never raise a budget, lower a threshold, add a retry or skip a case to get a pass") is the native repair path.

Raw lines 111 to 127
128### RH-16 — Exception swallowed
130`catch {}`, `catch (e) {}`, `except: pass`, `rescue nil` or `_ = err` added
131in non-test code turns a failure into silence. Countermeasure: the tag lists it
132under the RH id; slop-scan's empty-catch rule keeps the fix suggestion. The
133preamble's stderr rule (never cite a command whose stderr was silenced) covers
134the evidence-command form.

RH-16 — Exception swallowed

catch {}, catch (e) {}, except: pass, rescue nil or _ = err added in non-test code turns a failure into silence. Countermeasure: the tag lists it under the RH id; slop-scan's empty-catch rule keeps the fix suggestion. The preamble's stderr rule (never cite a command whose stderr was silenced) covers the evidence-command form.

Raw lines 128 to 135
136## Positive observables (zero-run green)

Positive observables (zero-run green)

Raw lines 136 to 137
138### PL-1 — Exit code is not evidence
140A command's exit 0 is reported as success without any observable that the
141work happened: the suite that selected nothing, the build that compiled zero
142files. Countermeasure: the preamble's Claims Need Evidence kernel (a claimed
143execution names the command and its observed result; a checked null result is
144reported as a null, not a pass) and `/ship` Step 16's receipt check.

PL-1 — Exit code is not evidence

A command's exit 0 is reported as success without any observable that the work happened: the suite that selected nothing, the build that compiled zero files. Countermeasure: the preamble's Claims Need Evidence kernel (a claimed execution names the command and its observed result; a checked null result is reported as a null, not a pass) and /ship Step 16's receipt check.

Raw lines 138 to 145
Names a skill ship :144
146### PL-2 — Zero-run green
148A test lane exits 0 having run zero tests: bun `0 pass 0 fail`, jest "No tests
149found", pytest "no tests ran", rspec "0 examples", `go test` with only `[no
150test files]`, or an empty selection after a filter. Countermeasure: `/ship`
151Step 5 passes a lane on its runner summary line, not exit 0 alone, with three
152never-merged count states (`tests_ran: N` pass; `tests_ran: 0` ZERO-RUN,
153triaged as a failed lane; `tests_ran: unknown` keeps the exit-0 pass and prints
154`count unavailable` in the PR body); `gstack-evidence check` reports ZERO-RUN
155as STALE-equivalent; the PR body's `## Test Coverage` section prints each
156lane's `tests ran` count.

PL-2 — Zero-run green

A test lane exits 0 having run zero tests: bun 0 pass 0 fail, jest "No tests found", pytest "no tests ran", rspec "0 examples", go test with only [no test files], or an empty selection after a filter. Countermeasure: /ship Step 5 passes a lane on its runner summary line, not exit 0 alone, with three never-merged count states (tests_ran: N pass; tests_ran: 0 ZERO-RUN, triaged as a failed lane; tests_ran: unknown keeps the exit-0 pass and prints count unavailable in the PR body); gstack-evidence check reports ZERO-RUN as STALE-equivalent; the PR body's ## Test Coverage section prints each lane's tests ran count.

Raw lines 146 to 157
Names a skill ship :150
158## Where the controls live
160| Surface | Control |
161|---|---|
162| preamble | Claims Need Evidence kernel: execution, evidence kind, failure disclosure, null results, stderr rule |
163| `/review` Step 3.5 Diff scans | `GATE_SCAN_BLOCK` runs `gstack-gate-diff <base>`; UNAVAILABLE and partial are missing coverage, never `none detected` |
164| `/review` checklist, Pass 1 | Gate Integrity category: three levels, the one question, ASK never AUTO-FIX, groups one decision, citations never remove a finding |
165| `/review` Step 5c | `review-gate-disposition` (one-way): A) Restore the gate, B) Keep — justified, C) Keep — other reason, D) Leave open for a later human; D recommended for spawned/headless sessions |
166| `/review` Step 5.8 | findings carry `gate`, `gate_id`, `reason`, logger-stamped `actor`; actions `kept`/`restored`/`open`; `open` counts in `issues_found`; no hunk text stored |
167| `gstack-review-log` | stamps `actor` from `gstack-session-kind` and forces `action: "open"` for non-interactive kinds |
168| `/ship` core checklist 2.5, Steps 7 and 9.4 | the same block before the checklist read, again after the test-generation child returns, again after parent fixes |
169| `/ship` Step 5 and Step 16 | runner-summary pass, three `tests_ran` states, ZERO-RUN as a failed lane |
170| `/ship` Step 16 stage 5 | each gate finding's disposition, reason and actor, or the coverage line; the auditor's posture |
171| `/ship` Step 19 | `Gate edits:` line under Pre-Landing Review, per-lane `tests ran` under Test Coverage; draft while any finding is `open` |

Where the controls live

SurfaceControl
preambleClaims Need Evidence kernel: execution, evidence kind, failure disclosure, null results, stderr rule
/review Step 3.5 Diff scansGATE_SCAN_BLOCK runs gstack-gate-diff <base>; UNAVAILABLE and partial are missing coverage, never none detected
/review checklist, Pass 1Gate Integrity category: three levels, the one question, ASK never AUTO-FIX, groups one decision, citations never remove a finding
/review Step 5creview-gate-disposition (one-way): A) Restore the gate, B) Keep — justified, C) Keep — other reason, D) Leave open for a later human; D recommended for spawned/headless sessions
/review Step 5.8findings carry gate, gate_id, reason, logger-stamped actor; actions kept/restored/open; open counts in issues_found; no hunk text stored
gstack-review-logstamps actor from gstack-session-kind and forces action: "open" for non-interactive kinds
/ship core checklist 2.5, Steps 7 and 9.4the same block before the checklist read, again after the test-generation child returns, again after parent fixes
/ship Step 5 and Step 16runner-summary pass, three tests_ran states, ZERO-RUN as a failed lane
/ship Step 16 stage 5each gate finding's disposition, reason and actor, or the coverage line; the auditor's posture
/ship Step 19Gate edits: line under Pre-Landing Review, per-lane tests ran under Test Coverage; draft while any finding is open
Raw lines 158 to 171
Starts an agent spawned :165
Names a skill ship :168