Skill Tracer
Skills › Scientific Agent Skills › scientific-critical-thinking › review_sources.md

review_sources.md

71 lines · skills/scientific-critical-thinking/references/review_sources.md at 5f2bc03

Open on GitHub
1# Primary guidance and review limits
2
3Reviewed 2026-10-01. This is an evidence-appraisal workflow, not an implementation of
4formal scoring algorithms, a statistical package, or a clinical decision system. Use the
5complete current tool documents for a formal assessment and retain version/date, result
6identity, source locations, signalling answers, and judgment rationale.
7

Primary guidance and review limits

Reviewed 2026-10-01. This is an evidence-appraisal workflow, not an implementation of formal scoring algorithms, a statistical package, or a clinical decision system. Use the complete current tool documents for a formal assessment and retain version/date, result identity, source locations, signalling answers, and judgment rationale.

Raw lines 1 to 7
8## Risk of bias, reporting, and certainty
9
10| Source | Verified scope and consequence |
11| --- | --- |
12| [Cochrane Handbook, chapter 8](https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-08) | RoB 2 is result-specific; assignment versus adherence and trial variant matter. Overall judgments use domain rules, not an average score. |
13| [ROBINS-I V2 developers](https://www.riskofbias.info/welcome/robins-i-v2) | November 2025 V2 is still labelled draft, for follow-up/cohort intervention studies. It differs from ROBINS-I 2016; name the version. |
14| [QUADAS-3 developers](https://www.bristol.ac.uk/population-health-sciences/projects/quadas/quadas-3/) | Current recommended diagnostic accuracy tool, published February 2026; listed download v1.2. Assessment is per accuracy estimate, with synthesis questions and an ideal test accuracy trial. |
15| [AMSTAR 2 developers](https://amstar.ca/Amstar-2.php) | Appraises systematic reviews of healthcare interventions; critical domains determine confidence, not a summed score. |
16| [PEDro scale](https://pedro.org.au/english/resources/pedro-scale/) | Physiotherapy-trial appraisal scale; eligibility item is not part of the total. Not equivalent to RoB 2 or a GRADE certainty rating. |
17| [Newcastle-Ottawa Scale developers](https://ohri.ca/en/who-we-are/core-facilities-and-platforms/ottawa-methods-centre/newcastle-ottawa-scale) | Cohort/case-control forms address selection, comparability, and outcome/exposure ascertainment. Do not invent universal star thresholds. |
18| [CASP checklists](https://casp-uk.net/casp-tools-checklists/) | Separate appraisal checklists by design; use the matching current form. |
19| [CONSORT/SPIRIT developers](https://www.consort-spirit.org/) and [extensions](https://www.consort-spirit.org/extensions) | Core trial-results and protocol guidance is CONSORT 2025/SPIRIT 2025. Some applicable extensions still reference older cores. Reporting standards do not certify conduct. |
20| [STROBE developers](https://www.strobe-statement.org/) | Cohort, case-control, and cross-sectional reporting; explicitly not a study-quality assessment instrument. |
21| [PRISMA 2020 developers](https://www.prisma-statement.org/prisma-2020) | Systematic-review reporting checklist, expanded checklist, and flow diagrams; not a bias or certainty score. |
22| [STARD guidance](https://www.equator-network.org/reporting-guidelines/stard/) | STARD 2015 for diagnostic accuracy reports; STARD-AI is a 2025 extension for AI studies. |
23| [COREQ guidance](https://www.equator-network.org/reporting-guidelines/coreq/) | Qualitative interviews and focus groups; not all qualitative designs indiscriminately. |
24| [CARE developers](https://www.care-statement.org/) | Case-report guidance developed in 2013 with 2017 explanation; reporting does not identify a comparative treatment effect. |
25| [GRADE Book principles](https://book.gradepro.org/guideline/principles-for-assessing-the-certainty-of-interventions) | Current chapter dated August 2025: outcome-level certainty relative to explicit thresholds/ranges, absolute effects, alternative NRSI starting approaches, and no double counting. |
26| [GRADE Book overview](https://book.gradepro.org/guideline/overview-of-the-grade-approach) | Updated May 2026; intervention, diagnostic, prognosis, exposure, and values applications differ. The intervention design hierarchy is not universal. |
27| [GRADE recommendations](https://book.gradepro.org/guideline/grade-recommendations) | Updated March 2026: recommendation direction/strength requires a separate evidence-to-decision rationale and context. |
28| [Cochrane Handbook, chapter 14](https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-14) | Practical Summary of Findings and ROBINS-I/GRADE integration; preserve outcome/comparison scope and explicit domain explanations. |
30The GRADE Book is replacing the older handbook progressively. Its three chapters above
31were read through rendered-page extraction because ordinary HTTP/browser text returned a
32JavaScript shell. The newer threshold/range framing should not be replaced with the older
33shortcut that certainty only means closeness to a point estimate.

Risk of bias, reporting, and certainty

SourceVerified scope and consequence
Cochrane Handbook, chapter 8RoB 2 is result-specific; assignment versus adherence and trial variant matter. Overall judgments use domain rules, not an average score.
ROBINS-I V2 developersNovember 2025 V2 is still labelled draft, for follow-up/cohort intervention studies. It differs from ROBINS-I 2016; name the version.
QUADAS-3 developersCurrent recommended diagnostic accuracy tool, published February 2026; listed download v1.2. Assessment is per accuracy estimate, with synthesis questions and an ideal test accuracy trial.
AMSTAR 2 developersAppraises systematic reviews of healthcare interventions; critical domains determine confidence, not a summed score.
PEDro scalePhysiotherapy-trial appraisal scale; eligibility item is not part of the total. Not equivalent to RoB 2 or a GRADE certainty rating.
Newcastle-Ottawa Scale developersCohort/case-control forms address selection, comparability, and outcome/exposure ascertainment. Do not invent universal star thresholds.
CASP checklistsSeparate appraisal checklists by design; use the matching current form.
CONSORT/SPIRIT developers and extensionsCore trial-results and protocol guidance is CONSORT 2025/SPIRIT 2025. Some applicable extensions still reference older cores. Reporting standards do not certify conduct.
STROBE developersCohort, case-control, and cross-sectional reporting; explicitly not a study-quality assessment instrument.
PRISMA 2020 developersSystematic-review reporting checklist, expanded checklist, and flow diagrams; not a bias or certainty score.
STARD guidanceSTARD 2015 for diagnostic accuracy reports; STARD-AI is a 2025 extension for AI studies.
COREQ guidanceQualitative interviews and focus groups; not all qualitative designs indiscriminately.
CARE developersCase-report guidance developed in 2013 with 2017 explanation; reporting does not identify a comparative treatment effect.
GRADE Book principlesCurrent chapter dated August 2025: outcome-level certainty relative to explicit thresholds/ranges, absolute effects, alternative NRSI starting approaches, and no double counting.
GRADE Book overviewUpdated May 2026; intervention, diagnostic, prognosis, exposure, and values applications differ. The intervention design hierarchy is not universal.
GRADE recommendationsUpdated March 2026: recommendation direction/strength requires a separate evidence-to-decision rationale and context.
Cochrane Handbook, chapter 14Practical Summary of Findings and ROBINS-I/GRADE integration; preserve outcome/comparison scope and explicit domain explanations.

The GRADE Book is replacing the older handbook progressively. Its three chapters above were read through rendered-page extraction because ordinary HTTP/browser text returned a JavaScript shell. The newer threshold/range framing should not be replaced with the older shortcut that certainty only means closeness to a point estimate.

Raw lines 8 to 34
35## Statistical and causal appraisal
37- [ASA p-value statement](https://www.amstat.org/asa/files/pdfs/p-valuestatement.pdf):
38 p-values do not give hypothesis probabilities, effect importance, or a complete basis
39 for decisions. Interpretation requires the analysis context and full reporting.
40- [Greenland et al., statistical misinterpretations](https://link.springer.com/article/10.1007/s10654-016-0149-3):
41 test statistics and confidence intervals depend on the full model and sampling/selection
42 assumptions; avoid observed-power, threshold, and interval-overlap shortcuts.
43- [Hernán and Robins, Causal Inference: What If](https://miguelhernan.org/whatifbook):
44 define the causal estimand and defend consistency, exchangeability, positivity, and
45 selection assumptions. The authors' current book link is dated August 19, 2026.
46- [Cochrane Handbook, chapter 13](https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-13):
47 missing-evidence appraisal cannot be reduced to a funnel plot. Asymmetry has alternative
48 explanations; tests usually require at least ten studies and appropriate effect measures.
49- [ARRIVE 2.0 experimental-unit explanation](https://arriveguidelines.org/arrive-guidelines/study-design/1b/explanation):
50 state the independent unit and avoid treating subsamples or repeated measurements as
51 additional independent experimental units.
52- [FDA multiple-endpoints guidance](https://www.fda.gov/regulatory-information/search-fda-guidance-documents/multiple-endpoints-clinical-trials):
53 supports specifying endpoint families and multiplicity strategies in human drug/biologic
54 trials; its regulatory scope is not a universal rule for all exploratory science.
55- [WMA Declaration of Helsinki](https://www.wma.net/policies-post/wma-declaration-of-helsinki/):
56 current 2024 revision concerns medical research involving human participants. Institutional
57 review and applicable local requirements must be checked; this skill certifies no compliance.

Statistical and causal appraisal

  • ASA p-value statement: p-values do not give hypothesis probabilities, effect importance, or a complete basis for decisions. Interpretation requires the analysis context and full reporting.
  • Greenland et al., statistical misinterpretations: test statistics and confidence intervals depend on the full model and sampling/selection assumptions; avoid observed-power, threshold, and interval-overlap shortcuts.
  • Hernán and Robins, Causal Inference: What If: define the causal estimand and defend consistency, exchangeability, positivity, and selection assumptions. The authors' current book link is dated August 19, 2026.
  • Cochrane Handbook, chapter 13: missing-evidence appraisal cannot be reduced to a funnel plot. Asymmetry has alternative explanations; tests usually require at least ten studies and appropriate effect measures.
  • ARRIVE 2.0 experimental-unit explanation: state the independent unit and avoid treating subsamples or repeated measurements as additional independent experimental units.
  • FDA multiple-endpoints guidance: supports specifying endpoint families and multiplicity strategies in human drug/biologic trials; its regulatory scope is not a universal rule for all exploratory science.
  • WMA Declaration of Helsinki: current 2024 revision concerns medical research involving human participants. Institutional review and applicable local requirements must be checked; this skill certifies no compliance.
Raw lines 35 to 58
59## Executed checks and limits
61- The skill contains no scripts, SDK calls, scientific API endpoints, or statistical
62 computation engine. References to REDCap, Qualtrics, registries, and statistical software
63 identify possible workflow tools; they are not integrations or tested instructions.
64- Tiny known-answer arithmetic checks verify the illustrative predictive values and
65 independent multiple-testing probability; they do not establish diagnostic performance
66 in a real population or validate a formal risk-of-bias/GRADE assessment.
67- The optional scientific-schematics command was checked at the local CLI/help boundary.
68 No paid generation of that example or private-data transmission was performed. Its
69 provider contract and dependencies belong to that skill, not this appraisal workflow.
70- No patient data, clinical study, registry submission, or authenticated scientific service
71 was accessed. Human clinical/policy decisions require the applicable professional process.

Executed checks and limits

  • The skill contains no scripts, SDK calls, scientific API endpoints, or statistical computation engine. References to REDCap, Qualtrics, registries, and statistical software identify possible workflow tools; they are not integrations or tested instructions.
  • Tiny known-answer arithmetic checks verify the illustrative predictive values and independent multiple-testing probability; they do not establish diagnostic performance in a real population or validate a formal risk-of-bias/GRADE assessment.
  • The optional scientific-schematics command was checked at the local CLI/help boundary. No paid generation of that example or private-data transmission was performed. Its provider contract and dependencies belong to that skill, not this appraisal workflow.
  • No patient data, clinical study, registry submission, or authenticated scientific service was accessed. Human clinical/policy decisions require the applicable professional process.
Raw lines 59 to 71