Skill Tracer
Skills › Scientific Agent Skills › scientific-writing › evidence_workflow.md

evidence_workflow.md

110 lines · skills/scientific-writing/references/evidence_workflow.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Evidence and Provenance Workflow
2

Evidence and Provenance Workflow

Raw lines 1 to 2
3## Core separation
4
5Drafting and evidence verification are different activities.
6
7- **Drafting** organizes language from already recorded evidence IDs. A draft may remain
8 incomplete and uncertain.
9- **Verification** requires an accountable human to open the source, confirm the exact
10 support and locator, check bibliographic metadata, and record who verified it and
11 when.
12- A fluent sentence is not evidence. Search snippets, generated summaries, memory, and
13 another paper's reference list are discovery aids, not verified support.

Core separation

Drafting and evidence verification are different activities.

  • Drafting organizes language from already recorded evidence IDs. A draft may remain incomplete and uncertain.
  • Verification requires an accountable human to open the source, confirm the exact support and locator, check bibliographic metadata, and record who verified it and when.
  • A fluent sentence is not evidence. Search snippets, generated summaries, memory, and another paper's reference list are discovery aids, not verified support.
Raw lines 3 to 14
15## Registries
17The scaffold creates five linked records:
191. `source_manifest.json` assigns each source an `E` ID and records identifiers,
20 location, confidentiality class, and verification state.
212. `claims.csv` assigns each claim a `C` ID, stores a hash rather than raw claim text,
22 and maps the claim to one or more verified `E` IDs.
233. `consistency_manifest.json` binds numeric facts, methods, outcomes, analysis intent,
24 units, denominators, sample sizes, and result locations.
254. `authorship.json` records human authorship criteria, CRediT roles, accountability,
26 declarations, and AI use.
275. `reporting_coverage.json` records high-level coverage without scoring or certifying
28 the manuscript.
30These registries contain metadata, not full source documents. Do not paste unpublished
31manuscripts, peer-review files, PHI, sensitive datasets, proprietary content, or source
32documents into them.

Registries

The scaffold creates five linked records:

  1. source_manifest.json assigns each source an E ID and records identifiers, location, confidentiality class, and verification state.
  2. claims.csv assigns each claim a C ID, stores a hash rather than raw claim text, and maps the claim to one or more verified E IDs.
  3. consistency_manifest.json binds numeric facts, methods, outcomes, analysis intent, units, denominators, sample sizes, and result locations.
  4. authorship.json records human authorship criteria, CRediT roles, accountability, declarations, and AI use.
  5. reporting_coverage.json records high-level coverage without scoring or certifying the manuscript.

These registries contain metadata, not full source documents. Do not paste unpublished manuscripts, peer-review files, PHI, sensitive datasets, proprietary content, or source documents into them.

Raw lines 15 to 33
34## Claim markers
36Append machine-readable markers to every factual or numeric assertion while drafting:
38```text
39[claim:C001] [evidence:E001,E002]
40```
42An alternative citation marker is `[@E001]`. Keep the claim and evidence markers on the
43same physical line until the audit passes. Use exactly one claim marker per line.
44Each line can express a composite claim only when its evidence supports every part.
45A final publisher conversion may replace evidence IDs with rendered citations only
46after preserving an auditable mapping.

Claim markers

Append machine-readable markers to every factual or numeric assertion while drafting:

[claim:C001] [evidence:E001,E002]

An alternative citation marker is [@E001]. Keep the claim and evidence markers on the same physical line until the audit passes. Use exactly one claim marker per line. Each line can express a composite claim only when its evidence supports every part. A final publisher conversion may replace evidence IDs with rendered citations only after preserving an auditable mapping.

Raw lines 34 to 47
48## Verification procedure
50For each source:
521. Open the authoritative source or record.
532. Confirm title, author or organization, publication state, year, and identifiers.
543. Record the exact supporting location, such as section, page, table, figure, or
55 registry field.
564. Confirm that the source supports the claim's direction, population, intervention or
57 exposure, outcome, time point, and uncertainty.
585. Record caveats, retractions, corrections, expressions of concern, or version status.
596. Mark the source verified only after a named human completes the check.
61For each claim:
631. Classify it as factual, numeric, method, result, interpretive, or declaration.
642. Remove `[claim:C...]`, `[evidence:E...]`, and `[@E...]` markers from the
65 entire physical line, collapse whitespace with `" ".join(text.split())`, then
66 SHA-256 the UTF-8 bytes and store the lowercase hexadecimal digest in
67 `claims.csv`. Preserve case, punctuation, and other Markdown syntax. The bundled
68 `audit_claims.claim_text_hash` function implements this exact normalization.
693. Map it to verified evidence IDs.
704. Record uncertainty and whether the analysis was confirmatory, exploratory,
71 descriptive, or not applicable.
725. Keep unsupported or conflicting claims out of submission-ready prose. If useful,
73 retain them in a clearly marked unresolved-issues log.

Verification procedure

For each source:

  1. Open the authoritative source or record.
  2. Confirm title, author or organization, publication state, year, and identifiers.
  3. Record the exact supporting location, such as section, page, table, figure, or registry field.
  4. Confirm that the source supports the claim's direction, population, intervention or exposure, outcome, time point, and uncertainty.
  5. Record caveats, retractions, corrections, expressions of concern, or version status.
  6. Mark the source verified only after a named human completes the check.

For each claim:

  1. Classify it as factual, numeric, method, result, interpretive, or declaration.
  2. Remove [claim:C...], [evidence:E...], and [@E...] markers from the entire physical line, collapse whitespace with " ".join(text.split()), then SHA-256 the UTF-8 bytes and store the lowercase hexadecimal digest in claims.csv. Preserve case, punctuation, and other Markdown syntax. The bundled audit_claims.claim_text_hash function implements this exact normalization.
  3. Map it to verified evidence IDs.
  4. Record uncertainty and whether the analysis was confirmatory, exploratory, descriptive, or not applicable.
  5. Keep unsupported or conflicting claims out of submission-ready prose. If useful, retain them in a clearly marked unresolved-issues log.
Raw lines 48 to 74
75## Drafting gates
77Do not infer or complete missing:
79- citations, identifiers, quotations, or source locators;
80- data values, denominators, units, sample sizes, or statistical results;
81- methods, protocol details, analysis decisions, or deviations;
82- approvals, consent, registrations, author contributions, conflicts, funding, or
83 availability statements.
85Use explicit missing states. Preserve negative, null, adverse, unexpected, and
86inconclusive findings. Never convert absence of evidence into evidence of no effect.

Drafting gates

Do not infer or complete missing:

  • citations, identifiers, quotations, or source locators;
  • data values, denominators, units, sample sizes, or statistical results;
  • methods, protocol details, analysis decisions, or deviations;
  • approvals, consent, registrations, author contributions, conflicts, funding, or availability statements.

Use explicit missing states. Preserve negative, null, adverse, unexpected, and inconclusive findings. Never convert absence of evidence into evidence of no effect.

Raw lines 75 to 87
88## Final audit
90Run:
92```bash
93python3 scripts/validate_manifest.py source_manifest.json --kind source --require-verified
94python3 scripts/audit_claims.py manuscript.md claims.csv source_manifest.json
95python3 scripts/check_consistency.py consistency_manifest.json
96python3 scripts/check_references.py source_manifest.json
97```
99Tool output contains IDs and line numbers, not manuscript or source text. A passing
100machine audit supports review; it does not replace human scientific judgment.
102The claim audit compares that digest with the current manuscript line and rejects
103changed text or multiple claim markers on a line. Re-verify the evidence fit before
104replacing an old digest. A matching hash detects text changes; it proves neither
105scientific truth nor that a human performed the recorded verification. Hashes are
106not encryption and do not anonymize predictable sensitive statements.
108The lightweight audit skips headings, comment-start lines, and fenced code; numeric
109text outside these regions must be tagged. It cannot identify every untagged
110non-numeric assertion or inspect the contents of linked files. Review those manually.

Final audit

Run:

python3 scripts/validate_manifest.py source_manifest.json --kind source --require-verified
python3 scripts/audit_claims.py manuscript.md claims.csv source_manifest.json
python3 scripts/check_consistency.py consistency_manifest.json
python3 scripts/check_references.py source_manifest.json

Tool output contains IDs and line numbers, not manuscript or source text. A passing machine audit supports review; it does not replace human scientific judgment.

The claim audit compares that digest with the current manuscript line and rejects changed text or multiple claim markers on a line. Re-verify the evidence fit before replacing an old digest. A matching hash detects text changes; it proves neither scientific truth nor that a human performed the recorded verification. Hashes are not encryption and do not anonymize predictable sensitive statements.

The lightweight audit skips headings, comment-start lines, and fenced code; numeric text outside these regions must be tagged. It cannot identify every untagged non-numeric assertion or inspect the contents of linked files. Review those manually.

Raw lines 88 to 110
Runs code python3 scripts/validate_manifest.py :93
Runs code python3 scripts/audit_claims.py :94
Runs code python3 scripts/check_consistency.py :95
Runs code python3 scripts/check_references.py :96