tool_reference.md
261 lines · skills/peer-review/references/tool_reference.md at 5f2bc03
Most marks come from matching words in the file, so treat them as pointers.
Local Tool Contracts
All bundled tools are deterministic Python 3.11+ standard-library CLIs. They make no network, model, image, subprocess, environment-variable, dynamic-code, or pickle calls.
Raw lines 1 to 4Shared safety behavior
- Inputs: local JSON, CSV, or Markdown only
- Maximum input size: 4 MiB
- Maximum CSV rows: 5,000
- UTF-8 only; NUL bytes rejected
- Symlink inputs and outputs rejected
- Duplicate JSON keys and CSV headers rejected
- Unknown schema fields rejected by JSON validators
- Existing outputs are not replaced unless
--forceis explicit - JSON and Markdown outputs are written atomically with owner-only permissions where supported
- Reports contain IDs, counts, rule codes, and line numbers—not raw manuscript, review, claim, title, author, or reference prose
Exit codes:
0: structurally valid or completed1: a validly parsed audit is blocked or has rule errors2: malformed input, unsafe path, unsupported field, or CLI validation error
Run tools from the skill directory or use absolute script paths.
Raw lines 5 to 25Intake validator
python3 scripts/validate_review_intake.py \
assets/review_intake_template.jsonPurpose:
- Confirm documented authorization and human accountability
- Record role, competence areas and limits, conflicts, target-venue policy, and review model
- Enforce local-only processing, no external service use, no data reuse, and a deletion/retention record
- Gate approved AI assistance on venue policy, permission, and disclosure
The bundled template is intentionally blocked until the human completes the controls.
Top-level JSON fields:
schema_version:2.0review_id: safe local identifier, not a manuscript titlematerial: status and sensitive-data flagauthorization: basis and documented permissionsreviewer: capacity, accountability, competence, and conflictsvenue_policy: checked status, review model, confidential-note channelai_use: policy, plan, permission, disclosurehandling: local-only, external-service, reuse, retention controlsscope: manuscript type, requested focus, limits, specialist needs
The report validates declarations, not their truth.
reviewer.conflict_status accepts not_assessed, none_identified, disclosed_to_editor, editor_cleared, or unresolved. Disclosure alone does not clear a conflict: disclosed_to_editor blocks until an editor actually clears participation. The validator cannot verify that clearance.
Reporting-guideline selector and coverage audit
Selection only:
python3 scripts/select_reporting_guidelines.py \
assets/study_profile_template.jsonSelection plus coverage:
python3 scripts/select_reporting_guidelines.py \
assets/study_profile_template.json \
--coverage assets/reporting_checklist_template.csvProfile fields:
schema_version:2.0profile_idstudy_types: identifiers such asrandomized_trialreport_kind:results,protocol,abstract, ordata_releasefeatures: for exampleai_based,ai_intervention,large_language_modeldomains: for examplehealth,genomics,proteomics
Coverage columns:
guideline_iditem_id: aggregate main item number for guidelines with a known main countstatus:reported,partly_reported,not_reported,not_applicable,not_assessedlocation: required for reported or partly reported itemsrationale: required for not-applicable items
The catalog is assets/reporting_guidelines.json, reviewed on the date embedded in that file; source-specific verification limits are in assets/source_ledger.csv. It does not fetch live updates. The output deliberately has no percentage or quality score.
For an AI trial protocol use study_types: ["randomized_trial"], report_kind: "protocol", features: ["ai_intervention"], and domains: ["health"]; this selects SPIRIT 2025 and SPIRIT-AI 2020. For an LLM development, tuning, prompt-engineering, or evaluation study use study_types: ["llm_study"], report_kind: "results", features: ["large_language_model"], and domains: ["health"]. Add prediction_model only if that design also applies. Abstracts have different checklists; the bundled catalog has no abstract checklist and a no-match result does not mean no guideline exists.
Main-item rows are an aggregate navigation record. coverage_record_complete means those row IDs are present, even if all are not_assessed; it does not certify coverage of subitems or a completed review.
Claim–evidence matrix validator
python3 scripts/validate_claim_evidence.py \
assets/claim_evidence_matrix_template.csvColumns:
claim_idlocationclaim_typeclaim_summary: input-only; never echoedevidence_ids: semicolon-delimited local IDssupport_level:supported,partly_supported,unsupported,not_assessedalignment_issue: direction, magnitude, population, outcome, timepoint, causal language, scope, uncertainty, selective reporting, other, or nonelimitation: input-only; never echoedrequested_action: input-only; never echoed
Rules include:
- Supported claims need evidence IDs and no declared alignment issue.
- Partly supported claims require evidence and an issue code; a missing requested action is a warning that still needs human resolution.
- Unsupported claims need an issue code.
- Claim IDs must be unique.
The tool does not determine whether evidence is true or sufficient.
Raw lines 98 to 125Statistics and reproducibility checklist
python3 scripts/audit_statistics_reproducibility.py \
assets/statistical_reproducibility_template.jsonThe JSON contains:
- Checklist and study-design IDs
- Specialist-review declaration
- Core item records with category, applicability, status, evidence locations, note, and requested action
Core areas:
- Question/estimand alignment
- Unit, independence, sample size, allocation, and blinding
- Inclusion/exclusion, missing data, and data handling
- Prespecification, method alignment, assumptions, multiplicity, and dependence
- Effect estimates, uncertainty, denominators, outcomes, and harms
- Data/material access, code/environment, and provenance
- Ethics/governance, claim interpretation, and selective reporting
The tool requires the core item IDs but permits additional safe IDs. It reports gaps and specialist-review triggers without a score.
Raw lines 126 to 150Citation/reference consistency audit
The Markdown must use Pandoc-style citation keys:
The synthetic method is described elsewhere [@ref-synthetic-2026].
Several sources may be grouped [@ref-one; @ref-two].Run:
python3 scripts/audit_citations.py \
local-manuscript.md \
assets/citation_references_template.csvReference CSV columns:
reference_idtitleauthorsyeardoiurlverification_status:verified_primary,verified_secondary, ornot_verified
The audit finds:
- Citation keys without reference rows
- Reference rows not cited
- Cited references not marked verified
- References without DOI or URL
- Malformed citation syntax
It validates DOI/URL shape only. It does not resolve identifiers, search the web, verify existence, or determine whether a reference supports a claim.
The parser is a lightweight bracket scanner, not Pandoc. Use single-line groups such as [@ref-one; @ref-two] with ASCII letter-leading keys up to 96 characters containing letters, digits, ., _, :, or -. Full Pandoc also supports other key characters, braced keys, author-in-text citations, and other constructions that this tool does not support. Multiline groups are not supported; code examples, links, escaped text, or other bracketed @ text can produce false matches. Review these cases separately using the official Pandoc citation syntax; a clean report is not proof that every citation was found.
Review scaffold generator
The intake must pass first:
python3 scripts/generate_review_scaffold.py \
completed-intake.json \
-o private-review.mdThe generator:
- Reads
assets/review_scaffold_template.md - Interpolates only safe intake identifiers
- Never reads or embeds manuscript text
- Separates comments to authors from confidential editor notes
- Provides structured major/minor comment fields
- Includes human-accountability and no-editorial-decision warnings
It refuses unresolved intake controls and implicit overwrite.
Raw lines 190 to 210Tone and actionability lint
python3 scripts/lint_review.py private-review.mdRequired headings:
# Comments to authors# Confidential comments to editor
Structured comment headings:
### Major comment M1### Minor comment m1
Each comment must contain non-placeholder values for:
LocationObservationEvidence or criterionWhy it mattersRequested action
The linter flags:
- Missing or reversed author/editor channels
- Editor-only markers in the author channel
- Unresolved scaffold placeholders
- A narrow lexicon of abusive or personal language
- Role impersonation and editorial-decision phrases
- Missing actionability fields
- Claims of executed analysis that need provenance
Lexical lint has false positives and false negatives. Human review remains required.
Raw lines 211 to 246Private output examples
All JSON-reporting CLIs accept:
-o local-report.jsonTo replace an existing output deliberately:
--forceDo not place outputs in a synced or shared directory unless the authorization and venue policy permit it. Delete or retain inputs, drafts, and reports according to the documented review policy.
Raw lines 247 to 261