Skill Tracer
Skills › Scientific Agent Skills › peer-review › security_validation.md

security_validation.md

78 lines · skills/peer-review/references/security_validation.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Security Validation Record
2
3Historical validation date: **2026-07-23** (local project date). Counts and scanner results below describe that run, not a new security certification.
4

Security Validation Record

Historical validation date: 2026-07-23 (local project date). Counts and scanner results below describe that run, not a new security certification.

Raw lines 1 to 4
5## Baseline
6
7The repository `SECURITY.md` entry recorded **10 findings** with maximum severity **CRITICAL**:
8
9- Cross-file environment-variable and network exfiltration
10- A multi-file collection/transmission chain
11- Environment harvesting in both schematic scripts
12- API-key transmission to an external model service
13- Full environment propagation to a subprocess
14- Repeated costly model/image operations
15- Mandatory external schematic/cross-skill behavior
17The affected files were:
19- Deleted: scripts/generate_schematic.py
20- Deleted: scripts/generate_schematic_ai.py
21- The former `SKILL.md`

Baseline

The repository SECURITY.md entry recorded 10 findings with maximum severity CRITICAL:

  • Cross-file environment-variable and network exfiltration
  • A multi-file collection/transmission chain
  • Environment harvesting in both schematic scripts
  • API-key transmission to an external model service
  • Full environment propagation to a subprocess
  • Repeated costly model/image operations
  • Mandatory external schematic/cross-skill behavior

The affected files were:

  • Deleted: scripts/generate_schematic.py
  • Deleted: scripts/generate_schematic_ai.py
  • The former SKILL.md
Raw lines 5 to 22
Runs code scripts/generate_schematic.py :19
Runs code scripts/generate_schematic_ai.py :20
23## Remediation
25- Deleted both external schematic scripts.
26- Removed credentials, environment access, `.env` loading, subprocess chaining, network requests, model calls, image generation, mandatory figures, and cross-skill calls.
27- Replaced them with bounded deterministic local JSON/CSV/Markdown validators and generators.
28- Added strict schemas, duplicate-key/header detection, size/row/cell limits, symlink rejection, private atomic output, and no implicit overwrite.
29- Added report minimization: IDs, counts, rule codes, and line numbers instead of manuscript/review prose.
30- Added AST tests that reject network libraries, executable serialization, dynamic code execution, and environment credential access.
31- Added confidentiality, no-reuse, authorization, conflict, competence, AI-policy, disclosure, and deletion/retention gates.

Remediation

  • Deleted both external schematic scripts.
  • Removed credentials, environment access, .env loading, subprocess chaining, network requests, model calls, image generation, mandatory figures, and cross-skill calls.
  • Replaced them with bounded deterministic local JSON/CSV/Markdown validators and generators.
  • Added strict schemas, duplicate-key/header detection, size/row/cell limits, symlink rejection, private atomic output, and no implicit overwrite.
  • Added report minimization: IDs, counts, rule codes, and line numbers instead of manuscript/review prose.
  • Added AST tests that reject network libraries, executable serialization, dynamic code execution, and environment credential access.
  • Added confidentiality, no-reuse, authorization, conflict, competence, AI-policy, disclosure, and deletion/retention gates.
Raw lines 23 to 32
33## Validation results
35- Agent Skills reference validator: **PASS**
36- Dependency-free CLI help checks: **PASS**
37- Synthetic standard-library tests: **26 passed**
38- Explicit AST parse with bytecode disabled: **8 scripts parsed**
39- Bytecode artifacts: **0**
40- IDE lints: **0**
41- Documented local-path link test: **PASS**
42- Markdown link check: **PASS** (access-controlled HTTP 403 treated as reachable)
43- Direct behavioral security scan: **SAFE, 0 findings**
44- Pull-request gate with `--fail-on HIGH`: **PASS**
45 - CRITICAL: 0
46 - HIGH: 0
47 - LOW: 3

Validation results

  • Agent Skills reference validator: PASS
  • Dependency-free CLI help checks: PASS
  • Synthetic standard-library tests: 26 passed
  • Explicit AST parse with bytecode disabled: 8 scripts parsed
  • Bytecode artifacts: 0
  • IDE lints: 0
  • Documented local-path link test: PASS
  • Markdown link check: PASS (access-controlled HTTP 403 treated as reachable)
  • Direct behavioral security scan: SAFE, 0 findings
  • Pull-request gate with --fail-on HIGH: PASS
  • CRITICAL: 0
  • HIGH: 0
  • LOW: 3
Raw lines 33 to 48
49## Residual LOW findings
51The final LLM-assisted pull-request scan reported:
531. **Missing `allowed-tools` declaration** — informational. This field is optional under the Agent Skills specification. The compatibility and body explicitly constrain bundled tools to local standard-library processing with no network, model, image, credential, or environment access.
542. **Broad description** — accepted as a scoped capability description. The mandatory authorization and venue-policy gate applies before confidential content is read, and the body limits all functions to peer-review assessment.
553. **Bounded CSV/JSON processing** — defensive observation with “no action required” in the scanner output. Inputs are already limited to 4 MiB, 5,000 CSV rows, 12,000 characters per cell, and finite list sizes; tests cover oversize rejection.
57None of the LOW findings permits data transmission or credential access. No CRITICAL or HIGH issue remains.

Residual LOW findings

The final LLM-assisted pull-request scan reported:

  1. **Missing allowed-tools declaration** — informational. This field is optional under the Agent Skills specification. The compatibility and body explicitly constrain bundled tools to local standard-library processing with no network, model, image, credential, or environment access.
  2. Broad description — accepted as a scoped capability description. The mandatory authorization and venue-policy gate applies before confidential content is read, and the body limits all functions to peer-review assessment.
  3. Bounded CSV/JSON processing — defensive observation with “no action required” in the scanner output. Inputs are already limited to 4 MiB, 5,000 CSV rows, 12,000 characters per cell, and finite list sizes; tests cover oversize rejection.

None of the LOW findings permits data transmission or credential access. No CRITICAL or HIGH issue remains.

Raw lines 49 to 58
59## Reproduction
61```bash
62PYTHONDONTWRITEBYTECODE=1 uv run --with pytest python -m pytest tests/peer-review -q
64uv run skills-ref validate skills/peer-review
66uv run skill-scanner scan skills/peer-review --use-behavioral
68uv run python scan_pr_skills.py \
69 --fail-on HIGH \
70 --output /tmp/peer-review-pr-scan.md \
71 skills/peer-review
72```
74The repository-level `SECURITY.md` was intentionally not edited in this scoped refresh; its generated snapshot will update through the repository’s normal scan process.

Reproduction

PYTHONDONTWRITEBYTECODE=1 uv run --with pytest python -m pytest tests/peer-review -q

uv run skills-ref validate skills/peer-review

uv run skill-scanner scan skills/peer-review --use-behavioral

uv run python scan_pr_skills.py \
  --fail-on HIGH \
  --output /tmp/peer-review-pr-scan.md \
  skills/peer-review

The repository-level SECURITY.md was intentionally not edited in this scoped refresh; its generated snapshot will update through the repository’s normal scan process.

Raw lines 59 to 75
Runs code python scan_pr_skills.py :68
76## 2026-10-01 maintenance checks
78The intake now distinguishes a disclosed conflict awaiting clearance from actual editor clearance. The reporting catalog adds SPIRIT-AI protocols and non-prediction LLM studies. Standard-library synthetic tests pass (30 tests and 7 CLI-help subtests), and `skills-ref validate` passes. A new collection-level security scan is coordinated separately; the historical scanner counts above were not rerun for this record. These checks use synthetic inputs and do not constitute a live manuscript review.

2026-10-01 maintenance checks

The intake now distinguishes a disclosed conflict awaiting clearance from actual editor clearance. The reporting catalog adds SPIRIT-AI protocols and non-prediction LLM studies. Standard-library synthetic tests pass (30 tests and 7 CLI-help subtests), and skills-ref validate passes. A new collection-level security scan is coordinated separately; the historical scanner counts above were not rerun for this record. These checks use synthetic inputs and do not constitute a live manuscript review.

Raw lines 76 to 78