Skill Tracer
Skills › Scientific Agent Skills › hypothesis-generation › tool_reference.md

tool_reference.md

248 lines · skills/hypothesis-generation/references/tool_reference.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Local Tool Reference
2

Local Tool Reference

Raw lines 1 to 2
3## Runtime and safety model
4
5All bundled CLIs:
6
7- require Python 3.11+ standard library only;
8- read explicit local JSON, CSV, or Markdown paths;
9- reject URL-like paths, symlinks, wrong suffixes, oversized inputs, invalid UTF-8, and NUL bytes;
10- cap inputs at 2 MiB, CSV data at 1,000 rows, cells/Markdown lines at 8,000 characters, and JSON collections at bounded sizes;
11- reject duplicate JSON keys, nonstandard `NaN`/`Infinity` constants, and JSON nesting beyond 64 container levels; require exact ordered CSV headers;
12- require full dates in `YYYY-MM-DD` calendar form; Python's accepted compact and ISO-week forms are not accepted by this schema;
13- make no network, model, image, subprocess, credential, or environment-variable calls;
14- write only to an explicit existing local directory;
15- refuse implicit overwrite unless `--force` is given;
16- use private mode (`0600`) and atomic replacement for generated output.
18Reports contain identifiers, counts, rule codes, and line numbers rather than copying scientific prose where practical.

Runtime and safety model

All bundled CLIs:

  • require Python 3.11+ standard library only;
  • read explicit local JSON, CSV, or Markdown paths;
  • reject URL-like paths, symlinks, wrong suffixes, oversized inputs, invalid UTF-8, and NUL bytes;
  • cap inputs at 2 MiB, CSV data at 1,000 rows, cells/Markdown lines at 8,000 characters, and JSON collections at bounded sizes;
  • reject duplicate JSON keys, nonstandard NaN/Infinity constants, and JSON nesting beyond 64 container levels; require exact ordered CSV headers;
  • require full dates in YYYY-MM-DD calendar form; Python's accepted compact and ISO-week forms are not accepted by this schema;
  • make no network, model, image, subprocess, credential, or environment-variable calls;
  • write only to an explicit existing local directory;
  • refuse implicit overwrite unless --force is given;
  • use private mode (0600) and atomic replacement for generated output.

Reports contain identifiers, counts, rule codes, and line numbers rather than copying scientific prose where practical.

Raw lines 3 to 19
20## Exit codes
22- `0`: input was structurally valid; warnings or human-review gaps may remain.
23- `1`: input was parsed, but consistency or required-control errors were found.
24- `2`: malformed, unsafe, missing, oversized, or wrong-type input.
26No exit code means a hypothesis is true, novel, ethical, safe, feasible, supported, or selected.

Exit codes

  • 0: input was structurally valid; warnings or human-review gaps may remain.
  • 1: input was parsed, but consistency or required-control errors were found.
  • 2: malformed, unsafe, missing, oversized, or wrong-type input.

No exit code means a hypothesis is true, novel, ethical, safe, feasible, supported, or selected.

Raw lines 20 to 27
28## 1. Hypothesis schema validator
30Asset: `assets/hypothesis_record_template.json`
32```bash
33python3 scripts/validate_hypothesis_schema.py local-record.json
34python3 scripts/validate_hypothesis_schema.py \
35 local-record.json -o local-validation.json
36```
38The exact top-level objects are:
40- observation;
41- research question;
42- hypotheses;
43- causal estimands;
44- predictions;
45- alternative explanations;
46- null hypotheses;
47- negative controls;
48- operationalizations;
49- analysis plan;
50- evidence/search link;
51- causal-bias risk register;
52- ethics/feasibility gates;
53- AI-use record.
55All hypothesis statuses must be `candidate`. Causal questions require at least one estimand. Cross-links among candidate, prediction, rival, measurement, analysis, control, and source IDs are checked.
57The validator does not read the evidence ledger, test measurements, appraise sources, or interpret results.

1. Hypothesis schema validator

Asset: assets/hypothesis_record_template.json

python3 scripts/validate_hypothesis_schema.py local-record.json
python3 scripts/validate_hypothesis_schema.py \
  local-record.json -o local-validation.json

The exact top-level objects are:

  • observation;
  • research question;
  • hypotheses;
  • causal estimands;
  • predictions;
  • alternative explanations;
  • null hypotheses;
  • negative controls;
  • operationalizations;
  • analysis plan;
  • evidence/search link;
  • causal-bias risk register;
  • ethics/feasibility gates;
  • AI-use record.

All hypothesis statuses must be candidate. Causal questions require at least one estimand. Cross-links among candidate, prediction, rival, measurement, analysis, control, and source IDs are checked.

The validator does not read the evidence ledger, test measurements, appraise sources, or interpret results.

Raw lines 28 to 58
Runs code python3 scripts/validate_hypothesis_schema.py :33
59## 2. Operationalization and measurement checklist
61Asset: `assets/operationalization_template.json`
63```bash
64python3 scripts/check_operationalization.py local-operationalization.json
65```
67Each measurement item records:
69- construct, operational definition, population/system, unit/categories, timing, and method completion;
70- variable role;
71- validity source IDs and applicability review;
72- reliability/repeatability;
73- calibration/quality control;
74- invariance/comparability;
75- masking;
76- missingness;
77- threshold/cut-point status;
78- limitations and human review.
80`planned`, `unresolved`, and `pending` states are reported as gaps, not converted to scores. A checklist cannot establish measurement validity.

2. Operationalization and measurement checklist

Asset: assets/operationalization_template.json

python3 scripts/check_operationalization.py local-operationalization.json

Each measurement item records:

  • construct, operational definition, population/system, unit/categories, timing, and method completion;
  • variable role;
  • validity source IDs and applicability review;
  • reliability/repeatability;
  • calibration/quality control;
  • invariance/comparability;
  • masking;
  • missingness;
  • threshold/cut-point status;
  • limitations and human review.

planned, unresolved, and pending states are reported as gaps, not converted to scores. A checklist cannot establish measurement validity.

Raw lines 59 to 81
Runs code python3 scripts/check_operationalization.py :64
82## 3. Prediction/rival matrix validator
84Asset: `assets/prediction_rival_matrix_template.csv`
86Exact header:
88```text
89prediction_id,hypothesis_id,rival_hypothesis_ids,conditions,observable,expected_if_focal,expected_if_rivals,falsifier,indeterminate_result,boundary_conditions,measurement_ids,negative_control_ids,analysis_ids,uncertainty
90```
92Use semicolons inside ID-list cells:
94```bash
95python3 scripts/validate_prediction_matrix.py local-matrix.csv
96python3 scripts/validate_prediction_matrix.py \
97 local-matrix.csv --record local-record.json
98```
100Checks include:
102- unique prediction IDs;
103- focal candidate not listed as its own rival;
104- declared focal and rival expectations are not lexically identical;
105- required falsifier, indeterminate result, boundary, measurement, control, analysis, and uncertainty fields;
106- optional cross-links to the hypothesis record.
108The validator cannot determine whether two predictions are scientifically distinguishable.

3. Prediction/rival matrix validator

Asset: assets/prediction_rival_matrix_template.csv

Exact header:

prediction_id,hypothesis_id,rival_hypothesis_ids,conditions,observable,expected_if_focal,expected_if_rivals,falsifier,indeterminate_result,boundary_conditions,measurement_ids,negative_control_ids,analysis_ids,uncertainty

Use semicolons inside ID-list cells:

python3 scripts/validate_prediction_matrix.py local-matrix.csv
python3 scripts/validate_prediction_matrix.py \
  local-matrix.csv --record local-record.json

Checks include:

  • unique prediction IDs;
  • focal candidate not listed as its own rival;
  • declared focal and rival expectations are not lexically identical;
  • required falsifier, indeterminate result, boundary, measurement, control, analysis, and uncertainty fields;
  • optional cross-links to the hypothesis record.

The validator cannot determine whether two predictions are scientifically distinguishable.

Raw lines 82 to 109
Runs code python3 scripts/validate_prediction_matrix.py :95
110## 4. Causal-versus-associational Markdown lint
112```bash
113python3 scripts/lint_causal_claims.py local-draft.md
114```
116Use one claim per annotated line:
118```markdown
119[claim:associational] Exposure X was associated with outcome Y in the observed sample.
121[claim:causal][estimand:E1][identification:observational_assumption_dependent][confounding:unresolved][selection:assessed][collider:assessed][reverse-causation:assessed] Under the stated assumptions, intervention X would reduce outcome Y.
122```
124Claim types:
126- `causal`
127- `associational`
128- `descriptive`
129- `predictive`
130- `mechanistic`
132Identification values:
134- `randomized`
135- `quasi_experimental`
136- `observational_assumption_dependent`
137- `mechanistic_experiment`
138- `other_assumption_dependent`
140Risk values:
142- `assessed`
143- `unresolved`
144- `not_applicable`
146The linter flags a bounded causal lexicon and annotation consistency. It is not a semantic classifier and will have false positives and false negatives.

4. Causal-versus-associational Markdown lint

python3 scripts/lint_causal_claims.py local-draft.md

Use one claim per annotated line:

[claim:associational] Exposure X was associated with outcome Y in the observed sample.

[claim:causal][estimand:E1][identification:observational_assumption_dependent][confounding:unresolved][selection:assessed][collider:assessed][reverse-causation:assessed] Under the stated assumptions, intervention X would reduce outcome Y.

Claim types:

  • causal
  • associational
  • descriptive
  • predictive
  • mechanistic

Identification values:

  • randomized
  • quasi_experimental
  • observational_assumption_dependent
  • mechanistic_experiment
  • other_assumption_dependent

Risk values:

  • assessed
  • unresolved
  • not_applicable

The linter flags a bounded causal lexicon and annotation consistency. It is not a semantic classifier and will have false positives and false negatives.

Raw lines 110 to 147
Runs code python3 scripts/lint_causal_claims.py :113
148## 5. Falsification and negative-control checklist
150Asset: `assets/falsification_controls_template.json`
152```bash
153python3 scripts/check_falsification_controls.py local-controls.json
154python3 scripts/check_falsification_controls.py \
155 local-controls.json --record local-record.json
156```
158For each candidate it requires:
160- assumptions and boundary conditions;
161- a prediction-linked falsifier and assumption-failure checks;
162- at least one discriminating test with focal, rival, and indeterminate outcomes;
163- a linked null and interpretation limit;
164- controls including at least one negative-control type;
165- outcome paths for consistency, challenge, and neither/mixed;
166- human-review status.
168The checker does not validate that a negative control is biologically or causally appropriate.

5. Falsification and negative-control checklist

Asset: assets/falsification_controls_template.json

python3 scripts/check_falsification_controls.py local-controls.json
python3 scripts/check_falsification_controls.py \
  local-controls.json --record local-record.json

For each candidate it requires:

  • assumptions and boundary conditions;
  • a prediction-linked falsifier and assumption-failure checks;
  • at least one discriminating test with focal, rival, and indeterminate outcomes;
  • a linked null and interpretation limit;
  • controls including at least one negative-control type;
  • outcome paths for consistency, challenge, and neither/mixed;
  • human-review status.

The checker does not validate that a negative control is biologically or causally appropriate.

Raw lines 148 to 169
Runs code python3 scripts/check_falsification_controls.py :153
170## 6. Evidence ledger and search-boundary audit
172Assets:
174- `assets/evidence_ledger_template.csv`
175- `assets/search_boundary_template.json`
177Exact evidence-ledger header:
179```text
180source_id,claim_ids,title,authors_or_organization,publication_date,source_type,identifier,url,accessed_on,relation,study_design_or_document_type,limitations,notes
181```
183```bash
184python3 scripts/audit_evidence_ledger.py \
185 local-evidence.csv local-search-boundary.json
187python3 scripts/audit_evidence_ledger.py \
188 local-evidence.csv local-search-boundary.json \
189 --record local-record.json
190```
192The audit checks:
194- exact schema and date/HTTPS/identifier formats;
195- unique source IDs;
196- source-to-claim links;
197- source-type and relation declarations;
198- dated search boundary, queries, limits, stop rule, and novelty status;
199- optional source, claim, and boundary links to a hypothesis record.
201It deliberately performs no network access. It cannot verify that a URL exists, a source says what is claimed, evidence is complete, or an idea is novel.
203Allowed novelty states:
205- `not_assessed`
206- `requires_specialist_review`
207- `supported_by_documented_comprehensive_search`
209The final state still requires a qualified human and supports only a bounded statement.

6. Evidence ledger and search-boundary audit

Assets:

  • assets/evidence_ledger_template.csv
  • assets/search_boundary_template.json

Exact evidence-ledger header:

source_id,claim_ids,title,authors_or_organization,publication_date,source_type,identifier,url,accessed_on,relation,study_design_or_document_type,limitations,notes
python3 scripts/audit_evidence_ledger.py \
  local-evidence.csv local-search-boundary.json

python3 scripts/audit_evidence_ledger.py \
  local-evidence.csv local-search-boundary.json \
  --record local-record.json

The audit checks:

  • exact schema and date/HTTPS/identifier formats;
  • unique source IDs;
  • source-to-claim links;
  • source-type and relation declarations;
  • dated search boundary, queries, limits, stop rule, and novelty status;
  • optional source, claim, and boundary links to a hypothesis record.

It deliberately performs no network access. It cannot verify that a URL exists, a source says what is claimed, evidence is complete, or an idea is novel.

Allowed novelty states:

  • not_assessed
  • requires_specialist_review
  • supported_by_documented_comprehensive_search

The final state still requires a qualified human and supports only a bounded statement.

Raw lines 170 to 210
Runs code python3 scripts/audit_evidence_ledger.py :184
211## 7. Preregistration scaffold generator
213Asset: `assets/preregistration_scaffold_template.md`
215```bash
216python3 scripts/generate_preregistration_scaffold.py \
217 local-record.json -o local-preregistration.md
218```
220The generator:
222- first runs record validation;
223- refuses unresolved ethics/safety/feasibility gates;
224- renders all candidates and rivals without ranking;
225- escapes inserted text for inert Markdown;
226- marks the output as an unregistered draft;
227- labels `updated_on` as the input record's update date, without inventing a generation or registration timestamp;
228- leaves repository-, design-, oversight-, and sign-off fields for humans.
230It never uploads, registers, timestamps externally, or submits the result.

7. Preregistration scaffold generator

Asset: assets/preregistration_scaffold_template.md

python3 scripts/generate_preregistration_scaffold.py \
  local-record.json -o local-preregistration.md

The generator:

  • first runs record validation;
  • refuses unresolved ethics/safety/feasibility gates;
  • renders all candidates and rivals without ranking;
  • escapes inserted text for inert Markdown;
  • marks the output as an unregistered draft;
  • labels updated_on as the input record's update date, without inventing a generation or registration timestamp;
  • leaves repository-, design-, oversight-, and sign-off fields for humans.

It never uploads, registers, timestamps externally, or submits the result.

Raw lines 211 to 231
Runs code python3 scripts/generate_preregistration_scaffold.py :216
232## Suggested local sequence
234```bash
235python3 scripts/validate_hypothesis_schema.py local-record.json
236python3 scripts/check_operationalization.py local-operationalization.json
237python3 scripts/validate_prediction_matrix.py \
238 local-predictions.csv --record local-record.json
239python3 scripts/check_falsification_controls.py \
240 local-controls.json --record local-record.json
241python3 scripts/audit_evidence_ledger.py \
242 local-evidence.csv local-search-boundary.json --record local-record.json
243python3 scripts/lint_causal_claims.py local-draft.md
244python3 scripts/generate_preregistration_scaffold.py \
245 local-record.json -o local-preregistration.md
246```
248Qualified human review remains mandatory after every command.

Suggested local sequence

python3 scripts/validate_hypothesis_schema.py local-record.json
python3 scripts/check_operationalization.py local-operationalization.json
python3 scripts/validate_prediction_matrix.py \
  local-predictions.csv --record local-record.json
python3 scripts/check_falsification_controls.py \
  local-controls.json --record local-record.json
python3 scripts/audit_evidence_ledger.py \
  local-evidence.csv local-search-boundary.json --record local-record.json
python3 scripts/lint_causal_claims.py local-draft.md
python3 scripts/generate_preregistration_scaffold.py \
  local-record.json -o local-preregistration.md

Qualified human review remains mandatory after every command.

Raw lines 232 to 248
Runs code python3 scripts/validate_hypothesis_schema.py :235
Runs code python3 scripts/check_operationalization.py :236
Runs code python3 scripts/validate_prediction_matrix.py :237
Runs code python3 scripts/check_falsification_controls.py :239
Runs code python3 scripts/audit_evidence_ledger.py :241
Runs code python3 scripts/lint_causal_claims.py :243
Runs code python3 scripts/generate_preregistration_scaffold.py :244