Skill Tracer
Skills › gstack › qa-only › exploratory.md

exploratory.md

107 lines · qa-only/sections/exploratory.md at 3f79d81

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1<!-- AUTO-GENERATED from exploratory.md.tmpl — do not edit directly -->
2<!-- Regenerate: bun run gen:skill-docs -->
3# Shared exploratory QA
4
5The **caller** (/qa, /qa-only, /review or /ship) owns decisions, tests, fixes and publication. Discovery writes only reports/evidence
6and owned fixture state; no workflows, framework installs or publication.
7

<!-- AUTO-GENERATED from exploratory.md.tmpl — do not edit directly --> <!-- Regenerate: bun run gen:skill-docs -->

Shared exploratory QA

The caller (/qa, /qa-only, /review or /ship) owns decisions, tests, fixes and publication. Discovery writes only reports/evidence and owned fixture state; no workflows, framework installs or publication.

Raw lines 1 to 7
Names a skill review :5
8## 0. Preparation gate
9
10Complete these Reads in order before writing charters or probing:
111. Read `sections/scope.md` relative to the installed `qa`/`gstack-qa` SKILL.md directory in full and select the surfaces.
122. Read the selected surface methods below in full.
14Use this host's installed `qa`/`gstack-qa` SKILL.md directory for these reads:
16**Functional surfaces:**
17Read `sections/system-functional.md` in full.
19**Browser surfaces only:**
20Read `sections/qa-patterns.md` in full.
22Await each successful Read result before continuing. A supplied target, isolation
23description, section index or remembered method is not a completed instruction Read.
24Do not repeat a Read already completed in this invocation; reuse only its acknowledged
25full contents. If either required Read is missing, complete it now before Charter and preflight.
26Missing or unreadable assets, prerequisites or permission block affected probes, not independent safe checks. Report QA setup blockers.

0. Preparation gate

Complete these Reads in order before writing charters or probing:

  1. Read sections/scope.md relative to the installed qa/gstack-qa SKILL.md directory in full and select the surfaces.
  2. Read the selected surface methods below in full.

Use this host's installed qa/gstack-qa SKILL.md directory for these reads:

Functional surfaces: Read sections/system-functional.md in full.

Browser surfaces only: Read sections/qa-patterns.md in full.

Await each successful Read result before continuing. A supplied target, isolation description, section index or remembered method is not a completed instruction Read. Do not repeat a Read already completed in this invocation; reuse only its acknowledged full contents. If either required Read is missing, complete it now before Charter and preflight. Missing or unreadable assets, prerequisites or permission block affected probes, not independent safe checks. Report QA setup blockers.

Raw lines 8 to 27
Names a skill qa :11
28## 1. Charter and preflight
30Reuse resolved REPORT_DIR; otherwise own a fresh `.gstack/qa-reports` subdirectory.
31Write a **charter** per behavior: contract, risk, entrypoint, isolation, exit condition, source, commands and inputs. Save charters as Markdown in the report.
34For /qa and /qa-only:
35- Browser Quick: SECONDS=180. Browser Full/Regression: SECONDS=900.
36- Functional Full, Quick and Regression have no default total timer.
37Set SECONDS to the shorter mode/caller limit; an unlimited mode uses the caller's bound.
38Without a total time limit, do not create DEADLINE_FILE; announce finite command timeouts.
39Stop when scoped contracts are tested or blocked: each field's documented input rule needs a probe that breaks only it and satisfies every other rule; rules uncovered at a probe/time cap go in limits.
40Clocks/checkpoints use REPORT_DIR; mixed standalone runs use REPORT_DIR/browser and REPORT_DIR/functional, with one final report at REPORT_DIR. Caller paths win.
41Names used below (quote every path):
42- PROBE_DIR: this surface's owned probe directory per the line above. DEADLINE_FILE: `PROBE_DIR/deadline.json`, which exists only for a bounded run.
43- SECONDS: the total probe budget set above. NNN: a fresh three-digit ID (001, 002, ...) for each capture or checkpoint; never reuse one.
44- DEADLINE_TOOL = `$HOME/.claude/skills/gstack/bin/gstack-qa-deadline`, the deadline guard: `start` creates DEADLINE_FILE, `status` prints `remainingMs` and `expired`, and `run` executes one command, stopping it at the deadline.
45- EVIDENCE_TOOL = `$HOME/.claude/skills/gstack/bin/gstack-qa-evidence`, the functional evidence recorder: `capture` runs one command, stores its exit code, stdout and stderr under `PROBE_DIR/.qa-evidence/NNN/`, and prints status `complete`, `incomplete` or `sensitive`.
47Start once before baseline: `bun DEADLINE_TOOL start DEADLINE_FILE SECONDS [EARLIER_UTC]` if bounded.
48EARLIER_UTC = caller's absolute deadline, if set.
49Functional: `bun EVIDENCE_TOOL capture PROBE_DIR NNN [--public] --deadline DEADLINE_FILE -- COMMAND ARGS`.
50Unbounded: use `--timeout-ms MS` instead of `--deadline DEADLINE_FILE`.
51--public requires approved public/synthetic output; EVIDENCE_TOOL screens credentials. For complete private captures, await a safe Read of `PROBE_DIR/.qa-evidence/NNN/observation.json` (the decoded stdout). Sensitive/incomplete captures cannot anchor checkpoints.
52Bounded browsers: `bun DEADLINE_TOOL run DEADLINE_FILE -- COMMAND ARGS`. No detached probes.
53Never reset DEADLINE_FILE/bypass DEADLINE_TOOL. Expiry or an invalid/missing DEADLINE_FILE stops probes; report unfinished coverage. QA_DEADLINE receipts are not observations.

1. Charter and preflight

Reuse resolved REPORT_DIR; otherwise own a fresh .gstack/qa-reports subdirectory. Write a charter per behavior: contract, risk, entrypoint, isolation, exit condition, source, commands and inputs. Save charters as Markdown in the report.

For /qa and /qa-only:

  • Browser Quick: SECONDS=180. Browser Full/Regression: SECONDS=900.
  • Functional Full, Quick and Regression have no default total timer.

Set SECONDS to the shorter mode/caller limit; an unlimited mode uses the caller's bound. Without a total time limit, do not create DEADLINE_FILE; announce finite command timeouts. Stop when scoped contracts are tested or blocked: each field's documented input rule needs a probe that breaks only it and satisfies every other rule; rules uncovered at a probe/time cap go in limits. Clocks/checkpoints use REPORT_DIR; mixed standalone runs use REPORT_DIR/browser and REPORT_DIR/functional, with one final report at REPORT_DIR. Caller paths win. Names used below (quote every path):

  • PROBE_DIR: this surface's owned probe directory per the line above. DEADLINE_FILE: PROBE_DIR/deadline.json, which exists only for a bounded run.
  • SECONDS: the total probe budget set above. NNN: a fresh three-digit ID (001, 002, ...) for each capture or checkpoint; never reuse one.
  • DEADLINE_TOOL = $HOME/.claude/skills/gstack/bin/gstack-qa-deadline, the deadline guard: start creates DEADLINE_FILE, status prints remainingMs and expired, and run executes one command, stopping it at the deadline.
  • EVIDENCE_TOOL = $HOME/.claude/skills/gstack/bin/gstack-qa-evidence, the functional evidence recorder: capture runs one command, stores its exit code, stdout and stderr under PROBE_DIR/.qa-evidence/NNN/, and prints status complete, incomplete or sensitive.

Start once before baseline: bun DEADLINE_TOOL start DEADLINE_FILE SECONDS [EARLIER_UTC] if bounded. EARLIER_UTC = caller's absolute deadline, if set. Functional: bun EVIDENCE_TOOL capture PROBE_DIR NNN [--public] --deadline DEADLINE_FILE -- COMMAND ARGS. Unbounded: use --timeout-ms MS instead of --deadline DEADLINE_FILE. --public requires approved public/synthetic output; EVIDENCE_TOOL screens credentials. For complete private captures, await a safe Read of PROBE_DIR/.qa-evidence/NNN/observation.json (the decoded stdout). Sensitive/incomplete captures cannot anchor checkpoints. Bounded browsers: bun DEADLINE_TOOL run DEADLINE_FILE -- COMMAND ARGS. No detached probes. Never reset DEADLINE_FILE/bypass DEADLINE_TOOL. Expiry or an invalid/missing DEADLINE_FILE stops probes; report unfinished coverage. QA_DEADLINE receipts are not observations.

Raw lines 28 to 54
Runs code bin/gstack-qa-deadline :44
Runs code bin/gstack-qa-evidence :45
55## 2. Probe loop
57Each probe is one native command/interaction plus checks, excluding bookkeeping.
58Never batch probes.
601. First demonstrate success: output AND durable effects. Guard if bounded; await completion.
612. **Decide whether another probe is needed.** If bounded, run `bun DEADLINE_TOOL status DEADLINE_FILE`.
62 If expired or no safe next probe remains, STOP exploration; write the report (§4), not a checkpoint.
63 **Classify the last result before copying it.** For public or synthetic observations,
64 retain the entire result unchanged, including owned fixture paths, IDs, hashes and
65 existing credential placeholders. An absolute state path is not itself a secret.
66 For actual secrets/private payloads, withhold those values and disclose the redaction
67 and replay limits in the report. If no safe exact observation can be retained,
68 stop the affected probe chain; never invent a substitute path, identity or state.
69 **Publish before probing.** Create (browser) or compose (functional) `exploration-NNN.json` in the probe directory, beside its deadline if bounded, with exactly four top-level fields:
70 observationCommand: last completed probe's full outer command, including guard.
71 observed: its exact decoded child JSON (no wrapper/extra keys), or its full non-JSON text.
72 For guarded text, copy the complete span between the guard's started and finished receipt lines.
73 Keep its whitespace and content fences verbatim. Do not summarize, relabel or add timing text.
74 The guard adds one newline before its finished receipt; that separator is not child text.
75 For unguarded text, copy the complete result instead.
76 If capture is incomplete, report that limit instead of reconstructing it.
77 hypothesis: why nextCommand. nextCommand: exact command/request, guarded if bounded.
78 Preserve every safe program-JSON key/value and identity hash unchanged.
79 Withhold unsafe values, disclose limits and stop that chain.
80 Check fields before publication. No drafts/placeholders or invented safe-path redactions; corrections cannot repair published notes.
81 Functional: do not write this file; the next capture publishes it: `bun EVIDENCE_TOOL capture PROBE_DIR NNN --deadline DEADLINE_FILE --after PREV --hypothesis 'why' -- CMD` (PREV: the last complete capture's ID). EVIDENCE_TOOL supplies observed; never transcribe it.
82 Browser checkpoints use Write.
83 Wait for successful checkpoint publication before dispatch.
84 Never backfill or overwrite notes.
853. Run that exact probe; DEADLINE_TOOL enforces the deadline when bounded.
86 Report refusals as not-run; retain initial state/inputs/results. Repeat from step 2.
874. Replay the exact failing command/request from the same initial fixture state via steps 2–3 (same native command, fresh capture ID)
88 to confirm it, then minimize via those gates. Expiry leaves confirmation/minimization incomplete.
89 Another input or a regression test is not that replay.
905. If the user or another process changes source, commands or fixtures, review the affected
91 contracts and return to step 2 for each affected revalidation (unproven=affected). Do not make product changes yourself.
92 Keep the original limits/notes; update outcomes only from fresh evidence.

2. Probe loop

Each probe is one native command/interaction plus checks, excluding bookkeeping. Never batch probes.

  1. First demonstrate success: output AND durable effects. Guard if bounded; await completion.
  2. Decide whether another probe is needed. If bounded, run bun DEADLINE_TOOL status DEADLINE_FILE. If expired or no safe next probe remains, STOP exploration; write the report (§4), not a checkpoint. Classify the last result before copying it. For public or synthetic observations, retain the entire result unchanged, including owned fixture paths, IDs, hashes and existing credential placeholders. An absolute state path is not itself a secret. For actual secrets/private payloads, withhold those values and disclose the redaction and replay limits in the report. If no safe exact observation can be retained, stop the affected probe chain; never invent a substitute path, identity or state. Publish before probing. Create (browser) or compose (functional) exploration-NNN.json in the probe directory, beside its deadline if bounded, with exactly four top-level fields: observationCommand: last completed probe's full outer command, including guard. observed: its exact decoded child JSON (no wrapper/extra keys), or its full non-JSON text. For guarded text, copy the complete span between the guard's started and finished receipt lines. Keep its whitespace and content fences verbatim. Do not summarize, relabel or add timing text. The guard adds one newline before its finished receipt; that separator is not child text. For unguarded text, copy the complete result instead. If capture is incomplete, report that limit instead of reconstructing it. hypothesis: why nextCommand. nextCommand: exact command/request, guarded if bounded. Preserve every safe program-JSON key/value and identity hash unchanged. Withhold unsafe values, disclose limits and stop that chain. Check fields before publication. No drafts/placeholders or invented safe-path redactions; corrections cannot repair published notes. Functional: do not write this file; the next capture publishes it: bun EVIDENCE_TOOL capture PROBE_DIR NNN --deadline DEADLINE_FILE --after PREV --hypothesis 'why' -- CMD (PREV: the last complete capture's ID). EVIDENCE_TOOL supplies observed; never transcribe it. Browser checkpoints use Write. Wait for successful checkpoint publication before dispatch. Never backfill or overwrite notes.
  3. Run that exact probe; DEADLINE_TOOL enforces the deadline when bounded. Report refusals as not-run; retain initial state/inputs/results. Repeat from step 2.
  4. Replay the exact failing command/request from the same initial fixture state via steps 2–3 (same native command, fresh capture ID) to confirm it, then minimize via those gates. Expiry leaves confirmation/minimization incomplete. Another input or a regression test is not that replay.
  5. If the user or another process changes source, commands or fixtures, review the affected contracts and return to step 2 for each affected revalidation (unproven=affected). Do not make product changes yourself. Keep the original limits/notes; update outcomes only from fresh evidence.
Raw lines 55 to 93
Starts an agent dispatch :83
94## 3. Parent handoff
96Never change product code, tests, configuration, dependencies or Git through any tool,
97including shell, rename, deletion, commit, stash or edit-then-restore. Return test_stub proposals
98with their failing contract and expected assertion; never create tests or freeze buggy output.

3. Parent handoff

Never change product code, tests, configuration, dependencies or Git through any tool, including shell, rename, deletion, commit, stash or edit-then-restore. Return test_stub proposals with their failing contract and expected assertion; never create tests or freeze buggy output.

Raw lines 94 to 99
100## 4. Final report
102Use the surface report template; link each checkpoint. Separate browser scores, functional outcomes and proposed/executed tests.
103Write PROBE_DIR/annotations.json {evidence: [{capture, command, contract, expected, classification}], limits: ["checkpoint 001"]} (limits: non-empty string array, not an object; browser-only: evidence []); before Markdown `bun EVIDENCE_TOOL materialize PROBE_DIR annotations.json` (fills observed/metadata; prints reportLinks; runs once per PROBE_DIR); you classify. Annotate every safe capture, including failures/replays: an omitted capture is withheld and keeps the verdict inconclusive. Classify a capture taken before an input change `superseded`; it closes when the same command reran on current inputs. Disclose withheld/incomplete evidence.
104Evidence is invocation-local.
105Missing prerequisites/expectations/observations, timeouts and refusal never pass.
106Pass requires all required current-input contracts to pass with no required remainder.
107Report blocked, inconclusive and not-run coverage without claiming success.

4. Final report

Use the surface report template; link each checkpoint. Separate browser scores, functional outcomes and proposed/executed tests. Write PROBE_DIR/annotations.json {evidence: [{capture, command, contract, expected, classification}], limits: ["checkpoint 001"]} (limits: non-empty string array, not an object; browser-only: evidence []); before Markdown bun EVIDENCE_TOOL materialize PROBE_DIR annotations.json (fills observed/metadata; prints reportLinks; runs once per PROBE_DIR); you classify. Annotate every safe capture, including failures/replays: an omitted capture is withheld and keeps the verdict inconclusive. Classify a capture taken before an input change superseded; it closes when the same command reran on current inputs. Disclose withheld/incomplete evidence. Evidence is invocation-local. Missing prerequisites/expectations/observations, timeouts and refusal never pass. Pass requires all required current-input contracts to pass with no required remainder. Report blocked, inconclusive and not-run coverage without claiming success.

Raw lines 100 to 107