Skill Tracer
Skills › Compound Engineering › ce-test-xcode › test-and-report.md

test-and-report.md

64 lines · skills/ce-test-xcode/references/test-and-report.md at 3449230

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Test and report
2
3This reference owns evidence collection after the app launches.
4

Test and report

This reference owns evidence collection after the app launches.

Raw lines 1 to 4
5## Exercise the requested surfaces
6
7Derive the key screens and flows from the user's request and the changed iOS surface. For each one:
8
9- Navigate through the running app and record what was exercised.
10- Capture a descriptively named screenshot of the resulting state.
11- Check that expected content and controls render without visible error or broken layout.
12- Read the captured simulator logs for crashes, exceptions, error-level messages, and failed network requests attributable to the flow.
14A simulated action reporting success is not proof of the expected state change; verify the visible result or logs.

Exercise the requested surfaces

Derive the key screens and flows from the user's request and the changed iOS surface. For each one:

  • Navigate through the running app and record what was exercised.
  • Capture a descriptively named screenshot of the resulting state.
  • Check that expected content and controls render without visible error or broken layout.
  • Read the captured simulator logs for crashes, exceptions, error-level messages, and failed network requests attributable to the flow.

A simulated action reporting success is not proof of the expected state change; verify the visible result or logs.

Raw lines 5 to 15
16### SwiftUI inline Text links
18Simulated taps do not trigger gesture recognizers on SwiftUI `Text` views with inline `AttributedString` links because the link is not exposed as a separate accessibility element. When such a tap reports success but has no visible effect, ask the user to tap the link manually in the simulator. If the target URL is known, this is the direct fallback:
20```bash
21xcrun simctl openurl <device-uuid> <URL>
22```
24Record which fallback supplied the verification; do not report the automated tap itself as a pass.

SwiftUI inline Text links

Simulated taps do not trigger gesture recognizers on SwiftUI Text views with inline AttributedString links because the link is not exposed as a separate accessibility element. When such a tap reports success but has no visible effect, ask the user to tap the link manually in the simulator. If the target URL is known, this is the direct fallback:

xcrun simctl openurl <device-uuid> <URL>

Record which fallback supplied the verification; do not report the automated tap itself as a pass.

Raw lines 16 to 25
Involves a person ask the user :18
26## Human-only verification
28Pause only when the scoped flow requires user interaction the available simulator automation cannot complete. Examples include Sign in with Apple, push delivery, a sandbox purchase, camera/photos permission, location permission, or the inline-link case above.
30State the exact action and expected observation, then ask whether it worked. Status is derived from evidence, not the user's next-action choice. `PASS` requires a completed passing outcome. `FAIL` records a completed failing outcome. `SKIP` is only for a check with no completed outcome. An unanswered check is `SKIP` for that surface; when no `FAIL` remains, it makes the overall result `PARTIAL`. Never silently mark an unanswered or failed check as passed.

Human-only verification

Pause only when the scoped flow requires user interaction the available simulator automation cannot complete. Examples include Sign in with Apple, push delivery, a sandbox purchase, camera/photos permission, location permission, or the inline-link case above.

State the exact action and expected observation, then ask whether it worked. Status is derived from evidence, not the user's next-action choice. PASS requires a completed passing outcome. FAIL records a completed failing outcome. SKIP is only for a check with no completed outcome. An unanswered check is SKIP for that surface; when no FAIL remains, it makes the overall result PARTIAL. Never silently mark an unanswered or failed check as passed.

Raw lines 26 to 31
32## Failure route
34For a failed screen or flow, preserve its screenshot, relevant logs, and reproduction steps. Ask whether to investigate now or continue testing the remaining scope without investigation. That routing choice does not change the observed `FAIL`.
36- **Investigate now:** invoke `ce-debug` with the failure evidence and simulator reproduction context. Narrow its inherited authority to diagnosis and any fix the user approves at `ce-debug`'s informed fix gate, with no commit, push, or PR authority. Let it complete its owned quality and summary work, then return here. Only an applied fix triggers rebuild and retest. Derive any replacement status from the completed retest evidence; until that evidence exists, retain `FAIL` and continue the remaining scoped checks.
37- **Continue without investigation:** retain `FAIL`, preserve the observed failure evidence in its notes, and proceed with the rest of the scoped checks.

Failure route

For a failed screen or flow, preserve its screenshot, relevant logs, and reproduction steps. Ask whether to investigate now or continue testing the remaining scope without investigation. That routing choice does not change the observed FAIL.

  • Investigate now: invoke ce-debug with the failure evidence and simulator reproduction context. Narrow its inherited authority to diagnosis and any fix the user approves at ce-debug's informed fix gate, with no commit, push, or PR authority. Let it complete its owned quality and summary work, then return here. Only an applied fix triggers rebuild and retest. Derive any replacement status from the completed retest evidence; until that evidence exists, retain FAIL and continue the remaining scoped checks.
  • Continue without investigation: retain FAIL, preserve the observed failure evidence in its notes, and proceed with the rest of the scoped checks.
Raw lines 32 to 38
Names a skill ce-debug :36
39## Cleanup and summary
41Stop the log capture started by this run. Leave a simulator that was already booted as found; a simulator booted only for this run may be shut down after evidence is saved.
43Report these fields, omitting no field even when its value is `None` or `0`:
45Roll up the evidence statuses: any residual `FAIL` makes the result `FAIL`; otherwise any `SKIP` makes it `PARTIAL`; otherwise the result is `PASS`.
47```markdown
48## Xcode Test Results
50**Project:** <project or workspace>
51**Scheme:** <scheme>
52**Simulator:** <name>
53**Build:** Success | Failed
54**Screens tested:** <count>
56| Screen or flow | Status | Evidence / notes |
57|---|---|---|
58| <name> | PASS / FAIL / SKIP | <screenshot and observation> |
60**Console errors:** <count and relevant errors>
61**Human verifications:** <count and outcomes>
62**Failures:** <count and residual failures>
63**Result:** PASS | FAIL | PARTIAL
64```

Cleanup and summary

Stop the log capture started by this run. Leave a simulator that was already booted as found; a simulator booted only for this run may be shut down after evidence is saved.

Report these fields, omitting no field even when its value is None or 0:

Roll up the evidence statuses: any residual FAIL makes the result FAIL; otherwise any SKIP makes it PARTIAL; otherwise the result is PASS.

## Xcode Test Results

**Project:** <project or workspace>
**Scheme:** <scheme>
**Simulator:** <name>
**Build:** Success | Failed
**Screens tested:** <count>

| Screen or flow | Status | Evidence / notes |
|---|---|---|
| <name> | PASS / FAIL / SKIP | <screenshot and observation> |

**Console errors:** <count and relevant errors>
**Human verifications:** <count and outcomes>
**Failures:** <count and residual failures>
**Result:** PASS | FAIL | PARTIAL
Raw lines 39 to 64