system-functional.md
61 lines · qa/sections/system-functional.md at 3f79d81
1<!-- AUTO-GENERATED from system-functional.md.tmpl — do not edit directly -->
2<!-- Regenerate: bun run gen:skill-docs -->
3# Functional QA with repository-native tools
5Use documented repository commands, CLI/API clients and job/queue tools, not a new
6harness or browser substitution.
<!-- AUTO-GENERATED from system-functional.md.tmpl — do not edit directly --> <!-- Regenerate: bun run gen:skill-docs -->
Functional QA with repository-native tools
Use documented repository commands, CLI/API clients and job/queue tools, not a new harness or browser substitution.
Raw lines 1 to 78## Functional modes
10For /qa and /qa-only, within the selected scope:
11- **Full** (default): cover every applicable documented contract below.
12- **Quick** (`--quick`): check success and the highest-risk changed edge; mark other
13 contracts not run.
14- **Regression** (`--regression <previous-report>`): before probes, read the supplied
15 functional report and linked replay evidence. A missing, unreadable or wrong-target
16 baseline blocks regression mode. A browser-only `baseline.json` is not a functional
17 baseline. Re-establish owned setup; replay prior failed probes against the documented
18 expectation, never recorded buggy output, then check changed adjacent behavior.
19 Preserve the prior report; report fixed, still failing and new findings separately.
20 Missing safe replay inputs block affected probes, never count as passes.
22Mixed runs apply each surface's mode separately. /review and /ship retain their caller's
23bounded smoke and explicit plan checks, not Full exploration.
Functional modes
For /qa and /qa-only, within the selected scope:
- Full (default): cover every applicable documented contract below.
- Quick (
--quick): check success and the highest-risk changed edge; mark other contracts not run. - Regression (
--regression <previous-report>): before probes, read the supplied functional report and linked replay evidence. A missing, unreadable or wrong-target baseline blocks regression mode. A browser-onlybaseline.jsonis not a functional baseline. Re-establish owned setup; replay prior failed probes against the documented expectation, never recorded buggy output, then check changed adjacent behavior. Preserve the prior report; report fixed, still failing and new findings separately. Missing safe replay inputs block affected probes, never count as passes.
Mixed runs apply each surface's mode separately. /review and /ship retain their caller's bounded smoke and explicit plan checks, not Full exploration.
Raw lines 8 to 2425## Contract map
27Record each contract/source, isolated setup, exact probe, expectation and outcome:
28pass/fail/blocked/not run/inconclusive/not applicable (reason).
30| Contract | Observe |
31|---|---|
32| Successful execution | Expected return/output and final business effect, not just launch/acceptance |
33| Invalid/missing input | One probe per documented input rule of each field (allowed characters, format, sign, range, arity) breaking only it: declared rejection, correct status and no forbidden state change |
34| Authentication/authorization | Valid identity, missing/invalid identity, wrong owner/role and durable no-effect boundary |
35| CLI process contract | Exact exit code, stdout and stderr separately; resulting file/state changes |
36| State transitions | Initial, intermediate and completed/failed states and their permitted transitions |
37| Timeout/cancellation | Deadline, partial state, termination of owned work and recovery |
38| Retry | Attempts/backoff/terminal state promised by the repository; no unbounded retry |
39| Duplicates/idempotency | Repeated request/event and number of durable effects under the documented guarantee |
40| Concurrency/order | Controlled competing operations in both relevant completion orders; final invariant |
41| Partial-failure recovery | Interrupt after an effect, restart/replay, inspect completion/dead-letter state and duplicates |
43Do not impose universal exactly-once delivery. Separate acceptance, enqueue, processing,
44retry/dead-letter and final effect; 2xx is not completion. Expected rejection/injected
45failure may pass; a missing service preventing execution blocks coverage.
Contract map
Record each contract/source, isolated setup, exact probe, expectation and outcome: pass/fail/blocked/not run/inconclusive/not applicable (reason).
| Contract | Observe |
|---|---|
| Successful execution | Expected return/output and final business effect, not just launch/acceptance |
| Invalid/missing input | One probe per documented input rule of each field (allowed characters, format, sign, range, arity) breaking only it: declared rejection, correct status and no forbidden state change |
| Authentication/authorization | Valid identity, missing/invalid identity, wrong owner/role and durable no-effect boundary |
| CLI process contract | Exact exit code, stdout and stderr separately; resulting file/state changes |
| State transitions | Initial, intermediate and completed/failed states and their permitted transitions |
| Timeout/cancellation | Deadline, partial state, termination of owned work and recovery |
| Retry | Attempts/backoff/terminal state promised by the repository; no unbounded retry |
| Duplicates/idempotency | Repeated request/event and number of durable effects under the documented guarantee |
| Concurrency/order | Controlled competing operations in both relevant completion orders; final invariant |
| Partial-failure recovery | Interrupt after an effect, restart/replay, inspect completion/dead-letter state and duplicates |
Do not impose universal exactly-once delivery. Separate acceptance, enqueue, processing, retry/dead-letter and final effect; 2xx is not completion. Expected rejection/injected failure may pass; a missing service preventing execution blocks coverage.
Raw lines 25 to 4647## Execute and retain evidence
491. Apply the shared isolation/permission preflight. Verify cwd, command, environment
50 NAMES and safe reset; use synthetic data/credentials.
512. Follow the shared exploratory loop's order and written checkpoints.
52 For every probe, inspect initial/final durable state and retain exit/status and
53 stdout/stderr separately without masking failure.
543. On timeout, retain partial output/state and stop only owned work. Record setup errors
55 and untested contracts; never patch product code to hide missing prerequisites.
564. Record exact command or method/path/headers/body, setup/reset, expected contract/source,
57 observed output/state, revision/runtime, evidence paths and limits. Secrets are referenced
58 only by environment name. Disclose replay limits caused by redaction.
595. Use `templates/functional-report-template.md` relative to the installed QA SKILL.md.
60 Preserve evidence before owned cleanup and disclose leftovers. Return to the caller
61 without expanding discovery authority.
Execute and retain evidence
- Apply the shared isolation/permission preflight. Verify cwd, command, environment NAMES and safe reset; use synthetic data/credentials.
- Follow the shared exploratory loop's order and written checkpoints. For every probe, inspect initial/final durable state and retain exit/status and stdout/stderr separately without masking failure.
- On timeout, retain partial output/state and stop only owned work. Record setup errors and untested contracts; never patch product code to hide missing prerequisites.
- Record exact command or method/path/headers/body, setup/reset, expected contract/source, observed output/state, revision/runtime, evidence paths and limits. Secrets are referenced only by environment name. Disclose replay limits caused by redaction.
- Use
templates/functional-report-template.mdrelative to the installed QA SKILL.md. Preserve evidence before owned cleanup and disclose leftovers. Return to the caller without expanding discovery authority.