Skill Tracer
Skills › Scientific Agent Skills › peer-review › tool_reference.md

tool_reference.md

261 lines · skills/peer-review/references/tool_reference.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Local Tool Contracts
2
3All bundled tools are deterministic Python 3.11+ standard-library CLIs. They make no network, model, image, subprocess, environment-variable, dynamic-code, or pickle calls.
4

Local Tool Contracts

All bundled tools are deterministic Python 3.11+ standard-library CLIs. They make no network, model, image, subprocess, environment-variable, dynamic-code, or pickle calls.

Raw lines 1 to 4
5## Shared safety behavior
6
7- Inputs: local JSON, CSV, or Markdown only
8- Maximum input size: 4 MiB
9- Maximum CSV rows: 5,000
10- UTF-8 only; NUL bytes rejected
11- Symlink inputs and outputs rejected
12- Duplicate JSON keys and CSV headers rejected
13- Unknown schema fields rejected by JSON validators
14- Existing outputs are not replaced unless `--force` is explicit
15- JSON and Markdown outputs are written atomically with owner-only permissions where supported
16- Reports contain IDs, counts, rule codes, and line numbers—not raw manuscript, review, claim, title, author, or reference prose
18Exit codes:
20- `0`: structurally valid or completed
21- `1`: a validly parsed audit is blocked or has rule errors
22- `2`: malformed input, unsafe path, unsupported field, or CLI validation error
24Run tools from the skill directory or use absolute script paths.

Shared safety behavior

  • Inputs: local JSON, CSV, or Markdown only
  • Maximum input size: 4 MiB
  • Maximum CSV rows: 5,000
  • UTF-8 only; NUL bytes rejected
  • Symlink inputs and outputs rejected
  • Duplicate JSON keys and CSV headers rejected
  • Unknown schema fields rejected by JSON validators
  • Existing outputs are not replaced unless --force is explicit
  • JSON and Markdown outputs are written atomically with owner-only permissions where supported
  • Reports contain IDs, counts, rule codes, and line numbers—not raw manuscript, review, claim, title, author, or reference prose

Exit codes:

  • 0: structurally valid or completed
  • 1: a validly parsed audit is blocked or has rule errors
  • 2: malformed input, unsafe path, unsupported field, or CLI validation error

Run tools from the skill directory or use absolute script paths.

Raw lines 5 to 25
26## Intake validator
28```bash
29python3 scripts/validate_review_intake.py \
30 assets/review_intake_template.json
31```
33Purpose:
35- Confirm documented authorization and human accountability
36- Record role, competence areas and limits, conflicts, target-venue policy, and review model
37- Enforce local-only processing, no external service use, no data reuse, and a deletion/retention record
38- Gate approved AI assistance on venue policy, permission, and disclosure
40The bundled template is intentionally blocked until the human completes the controls.
42Top-level JSON fields:
44- `schema_version`: `2.0`
45- `review_id`: safe local identifier, not a manuscript title
46- `material`: status and sensitive-data flag
47- `authorization`: basis and documented permissions
48- `reviewer`: capacity, accountability, competence, and conflicts
49- `venue_policy`: checked status, review model, confidential-note channel
50- `ai_use`: policy, plan, permission, disclosure
51- `handling`: local-only, external-service, reuse, retention controls
52- `scope`: manuscript type, requested focus, limits, specialist needs
54The report validates declarations, not their truth.
56`reviewer.conflict_status` accepts `not_assessed`, `none_identified`, `disclosed_to_editor`, `editor_cleared`, or `unresolved`. Disclosure alone does not clear a conflict: `disclosed_to_editor` blocks until an editor actually clears participation. The validator cannot verify that clearance.

Intake validator

python3 scripts/validate_review_intake.py \
  assets/review_intake_template.json

Purpose:

  • Confirm documented authorization and human accountability
  • Record role, competence areas and limits, conflicts, target-venue policy, and review model
  • Enforce local-only processing, no external service use, no data reuse, and a deletion/retention record
  • Gate approved AI assistance on venue policy, permission, and disclosure

The bundled template is intentionally blocked until the human completes the controls.

Top-level JSON fields:

  • schema_version: 2.0
  • review_id: safe local identifier, not a manuscript title
  • material: status and sensitive-data flag
  • authorization: basis and documented permissions
  • reviewer: capacity, accountability, competence, and conflicts
  • venue_policy: checked status, review model, confidential-note channel
  • ai_use: policy, plan, permission, disclosure
  • handling: local-only, external-service, reuse, retention controls
  • scope: manuscript type, requested focus, limits, specialist needs

The report validates declarations, not their truth.

reviewer.conflict_status accepts not_assessed, none_identified, disclosed_to_editor, editor_cleared, or unresolved. Disclosure alone does not clear a conflict: disclosed_to_editor blocks until an editor actually clears participation. The validator cannot verify that clearance.

Raw lines 26 to 57
Runs code python3 scripts/validate_review_intake.py :29
58## Reporting-guideline selector and coverage audit
60Selection only:
62```bash
63python3 scripts/select_reporting_guidelines.py \
64 assets/study_profile_template.json
65```
67Selection plus coverage:
69```bash
70python3 scripts/select_reporting_guidelines.py \
71 assets/study_profile_template.json \
72 --coverage assets/reporting_checklist_template.csv
73```
75Profile fields:
77- `schema_version`: `2.0`
78- `profile_id`
79- `study_types`: identifiers such as `randomized_trial`
80- `report_kind`: `results`, `protocol`, `abstract`, or `data_release`
81- `features`: for example `ai_based`, `ai_intervention`, `large_language_model`
82- `domains`: for example `health`, `genomics`, `proteomics`
84Coverage columns:
86- `guideline_id`
87- `item_id`: aggregate main item number for guidelines with a known main count
88- `status`: `reported`, `partly_reported`, `not_reported`, `not_applicable`, `not_assessed`
89- `location`: required for reported or partly reported items
90- `rationale`: required for not-applicable items
92The catalog is `assets/reporting_guidelines.json`, reviewed on the date embedded in that file; source-specific verification limits are in `assets/source_ledger.csv`. It does not fetch live updates. The output deliberately has no percentage or quality score.
94For an AI trial protocol use `study_types: ["randomized_trial"]`, `report_kind: "protocol"`, `features: ["ai_intervention"]`, and `domains: ["health"]`; this selects SPIRIT 2025 and SPIRIT-AI 2020. For an LLM development, tuning, prompt-engineering, or evaluation study use `study_types: ["llm_study"]`, `report_kind: "results"`, `features: ["large_language_model"]`, and `domains: ["health"]`. Add `prediction_model` only if that design also applies. Abstracts have different checklists; the bundled catalog has no abstract checklist and a no-match result does not mean no guideline exists.
96Main-item rows are an aggregate navigation record. `coverage_record_complete` means those row IDs are present, even if all are `not_assessed`; it does not certify coverage of subitems or a completed review.

Reporting-guideline selector and coverage audit

Selection only:

python3 scripts/select_reporting_guidelines.py \
  assets/study_profile_template.json

Selection plus coverage:

python3 scripts/select_reporting_guidelines.py \
  assets/study_profile_template.json \
  --coverage assets/reporting_checklist_template.csv

Profile fields:

  • schema_version: 2.0
  • profile_id
  • study_types: identifiers such as randomized_trial
  • report_kind: results, protocol, abstract, or data_release
  • features: for example ai_based, ai_intervention, large_language_model
  • domains: for example health, genomics, proteomics

Coverage columns:

  • guideline_id
  • item_id: aggregate main item number for guidelines with a known main count
  • status: reported, partly_reported, not_reported, not_applicable, not_assessed
  • location: required for reported or partly reported items
  • rationale: required for not-applicable items

The catalog is assets/reporting_guidelines.json, reviewed on the date embedded in that file; source-specific verification limits are in assets/source_ledger.csv. It does not fetch live updates. The output deliberately has no percentage or quality score.

For an AI trial protocol use study_types: ["randomized_trial"], report_kind: "protocol", features: ["ai_intervention"], and domains: ["health"]; this selects SPIRIT 2025 and SPIRIT-AI 2020. For an LLM development, tuning, prompt-engineering, or evaluation study use study_types: ["llm_study"], report_kind: "results", features: ["large_language_model"], and domains: ["health"]. Add prediction_model only if that design also applies. Abstracts have different checklists; the bundled catalog has no abstract checklist and a no-match result does not mean no guideline exists.

Main-item rows are an aggregate navigation record. coverage_record_complete means those row IDs are present, even if all are not_assessed; it does not certify coverage of subitems or a completed review.

Raw lines 58 to 97
Runs code python3 scripts/select_reporting_guidelines.py :63
98## Claim–evidence matrix validator
100```bash
101python3 scripts/validate_claim_evidence.py \
102 assets/claim_evidence_matrix_template.csv
103```
105Columns:
107- `claim_id`
108- `location`
109- `claim_type`
110- `claim_summary`: input-only; never echoed
111- `evidence_ids`: semicolon-delimited local IDs
112- `support_level`: `supported`, `partly_supported`, `unsupported`, `not_assessed`
113- `alignment_issue`: direction, magnitude, population, outcome, timepoint, causal language, scope, uncertainty, selective reporting, other, or none
114- `limitation`: input-only; never echoed
115- `requested_action`: input-only; never echoed
117Rules include:
119- Supported claims need evidence IDs and no declared alignment issue.
120- Partly supported claims require evidence and an issue code; a missing requested action is a warning that still needs human resolution.
121- Unsupported claims need an issue code.
122- Claim IDs must be unique.
124The tool does not determine whether evidence is true or sufficient.

Claim–evidence matrix validator

python3 scripts/validate_claim_evidence.py \
  assets/claim_evidence_matrix_template.csv

Columns:

  • claim_id
  • location
  • claim_type
  • claim_summary: input-only; never echoed
  • evidence_ids: semicolon-delimited local IDs
  • support_level: supported, partly_supported, unsupported, not_assessed
  • alignment_issue: direction, magnitude, population, outcome, timepoint, causal language, scope, uncertainty, selective reporting, other, or none
  • limitation: input-only; never echoed
  • requested_action: input-only; never echoed

Rules include:

  • Supported claims need evidence IDs and no declared alignment issue.
  • Partly supported claims require evidence and an issue code; a missing requested action is a warning that still needs human resolution.
  • Unsupported claims need an issue code.
  • Claim IDs must be unique.

The tool does not determine whether evidence is true or sufficient.

Raw lines 98 to 125
Runs code python3 scripts/validate_claim_evidence.py :101
126## Statistics and reproducibility checklist
128```bash
129python3 scripts/audit_statistics_reproducibility.py \
130 assets/statistical_reproducibility_template.json
131```
133The JSON contains:
135- Checklist and study-design IDs
136- Specialist-review declaration
137- Core item records with category, applicability, status, evidence locations, note, and requested action
139Core areas:
141- Question/estimand alignment
142- Unit, independence, sample size, allocation, and blinding
143- Inclusion/exclusion, missing data, and data handling
144- Prespecification, method alignment, assumptions, multiplicity, and dependence
145- Effect estimates, uncertainty, denominators, outcomes, and harms
146- Data/material access, code/environment, and provenance
147- Ethics/governance, claim interpretation, and selective reporting
149The tool requires the core item IDs but permits additional safe IDs. It reports gaps and specialist-review triggers without a score.

Statistics and reproducibility checklist

python3 scripts/audit_statistics_reproducibility.py \
  assets/statistical_reproducibility_template.json

The JSON contains:

  • Checklist and study-design IDs
  • Specialist-review declaration
  • Core item records with category, applicability, status, evidence locations, note, and requested action

Core areas:

  • Question/estimand alignment
  • Unit, independence, sample size, allocation, and blinding
  • Inclusion/exclusion, missing data, and data handling
  • Prespecification, method alignment, assumptions, multiplicity, and dependence
  • Effect estimates, uncertainty, denominators, outcomes, and harms
  • Data/material access, code/environment, and provenance
  • Ethics/governance, claim interpretation, and selective reporting

The tool requires the core item IDs but permits additional safe IDs. It reports gaps and specialist-review triggers without a score.

Raw lines 126 to 150
Runs code python3 scripts/audit_statistics_reproducibility.py :129
151## Citation/reference consistency audit
153The Markdown must use Pandoc-style citation keys:
155```markdown
156The synthetic method is described elsewhere [@ref-synthetic-2026].
157Several sources may be grouped [@ref-one; @ref-two].
158```
160Run:
162```bash
163python3 scripts/audit_citations.py \
164 local-manuscript.md \
165 assets/citation_references_template.csv
166```
168Reference CSV columns:
170- `reference_id`
171- `title`
172- `authors`
173- `year`
174- `doi`
175- `url`
176- `verification_status`: `verified_primary`, `verified_secondary`, or `not_verified`
178The audit finds:
180- Citation keys without reference rows
181- Reference rows not cited
182- Cited references not marked verified
183- References without DOI or URL
184- Malformed citation syntax
186It validates DOI/URL shape only. It does not resolve identifiers, search the web, verify existence, or determine whether a reference supports a claim.
188The parser is a lightweight bracket scanner, not Pandoc. Use single-line groups such as `[@ref-one; @ref-two]` with ASCII letter-leading keys up to 96 characters containing letters, digits, `.`, `_`, `:`, or `-`. Full Pandoc also supports other key characters, braced keys, author-in-text citations, and other constructions that this tool does not support. Multiline groups are not supported; code examples, links, escaped text, or other bracketed `@` text can produce false matches. Review these cases separately using the [official Pandoc citation syntax](https://pandoc.org/MANUAL.html#citation-syntax); a clean report is not proof that every citation was found.

Citation/reference consistency audit

The Markdown must use Pandoc-style citation keys:

The synthetic method is described elsewhere [@ref-synthetic-2026].
Several sources may be grouped [@ref-one; @ref-two].

Run:

python3 scripts/audit_citations.py \
  local-manuscript.md \
  assets/citation_references_template.csv

Reference CSV columns:

  • reference_id
  • title
  • authors
  • year
  • doi
  • url
  • verification_status: verified_primary, verified_secondary, or not_verified

The audit finds:

  • Citation keys without reference rows
  • Reference rows not cited
  • Cited references not marked verified
  • References without DOI or URL
  • Malformed citation syntax

It validates DOI/URL shape only. It does not resolve identifiers, search the web, verify existence, or determine whether a reference supports a claim.

The parser is a lightweight bracket scanner, not Pandoc. Use single-line groups such as [@ref-one; @ref-two] with ASCII letter-leading keys up to 96 characters containing letters, digits, ., _, :, or -. Full Pandoc also supports other key characters, braced keys, author-in-text citations, and other constructions that this tool does not support. Multiline groups are not supported; code examples, links, escaped text, or other bracketed @ text can produce false matches. Review these cases separately using the official Pandoc citation syntax; a clean report is not proof that every citation was found.

Raw lines 151 to 189
Runs code python3 scripts/audit_citations.py :163
190## Review scaffold generator
192The intake must pass first:
194```bash
195python3 scripts/generate_review_scaffold.py \
196 completed-intake.json \
197 -o private-review.md
198```
200The generator:
202- Reads `assets/review_scaffold_template.md`
203- Interpolates only safe intake identifiers
204- Never reads or embeds manuscript text
205- Separates comments to authors from confidential editor notes
206- Provides structured major/minor comment fields
207- Includes human-accountability and no-editorial-decision warnings
209It refuses unresolved intake controls and implicit overwrite.

Review scaffold generator

The intake must pass first:

python3 scripts/generate_review_scaffold.py \
  completed-intake.json \
  -o private-review.md

The generator:

  • Reads assets/review_scaffold_template.md
  • Interpolates only safe intake identifiers
  • Never reads or embeds manuscript text
  • Separates comments to authors from confidential editor notes
  • Provides structured major/minor comment fields
  • Includes human-accountability and no-editorial-decision warnings

It refuses unresolved intake controls and implicit overwrite.

Raw lines 190 to 210
Runs code python3 scripts/generate_review_scaffold.py :195
211## Tone and actionability lint
213```bash
214python3 scripts/lint_review.py private-review.md
215```
217Required headings:
219- `# Comments to authors`
220- `# Confidential comments to editor`
222Structured comment headings:
224- `### Major comment M1`
225- `### Minor comment m1`
227Each comment must contain non-placeholder values for:
229- `Location`
230- `Observation`
231- `Evidence or criterion`
232- `Why it matters`
233- `Requested action`
235The linter flags:
237- Missing or reversed author/editor channels
238- Editor-only markers in the author channel
239- Unresolved scaffold placeholders
240- A narrow lexicon of abusive or personal language
241- Role impersonation and editorial-decision phrases
242- Missing actionability fields
243- Claims of executed analysis that need provenance
245Lexical lint has false positives and false negatives. Human review remains required.

Tone and actionability lint

python3 scripts/lint_review.py private-review.md

Required headings:

  • # Comments to authors
  • # Confidential comments to editor

Structured comment headings:

  • ### Major comment M1
  • ### Minor comment m1

Each comment must contain non-placeholder values for:

  • Location
  • Observation
  • Evidence or criterion
  • Why it matters
  • Requested action

The linter flags:

  • Missing or reversed author/editor channels
  • Editor-only markers in the author channel
  • Unresolved scaffold placeholders
  • A narrow lexicon of abusive or personal language
  • Role impersonation and editorial-decision phrases
  • Missing actionability fields
  • Claims of executed analysis that need provenance

Lexical lint has false positives and false negatives. Human review remains required.

Raw lines 211 to 246
Runs code python3 scripts/lint_review.py :214
247## Private output examples
249All JSON-reporting CLIs accept:
251```bash
252-o local-report.json
253```
255To replace an existing output deliberately:
257```bash
258--force
259```
261Do not place outputs in a synced or shared directory unless the authorization and venue policy permit it. Delete or retain inputs, drafts, and reports according to the documented review policy.

Private output examples

All JSON-reporting CLIs accept:

-o local-report.json

To replace an existing output deliberately:

--force

Do not place outputs in a synced or shared directory unless the authorization and venue policy permit it. Delete or retain inputs, drafts, and reports according to the documented review policy.

Raw lines 247 to 261