Sources and Evidence Notes

Review date: 2026-10-01. Official guidance and accessible primary-paper abstracts/text were rechecked with web browsing and focused extraction. This is a maintenance review, not a systematic search for every newer experiment. Dates below are publication, release, or page-update dates, not evidence that every paper's full text was accessible. Access limits are recorded at the end.

This bibliography supports process guidance; it does not show that any single brainstorming method universally improves creativity or scientific validity.

Group brainstorming, blocking, fixation, and selection

Diehl and Stroebe (1987) — primary experiments

Michael Diehl and Wolfgang Stroebe, “Productivity loss in brainstorming groups: Toward the solution of a riddle,” Journal of Personality and Social Psychology 53(3), 497–509. DOI 10.1037/0022-3514.53.3.497. Published 1987.

• Four experiments examined free riding, evaluation apprehension, and production blocking.
• Direct manipulation in the fourth experiment supported production blocking as an important explanation for loss in the tested interacting groups.
• Scope limit: laboratory brainstorming tasks do not establish superiority for every group purpose or setting.

Mullen, Johnson, and Salas (1991) — meta-analysis

Brian Mullen, Craig Johnson, and Eduardo Salas, “Productivity loss in brainstorming groups: A meta-analytic integration,” Basic and Applied Social Psychology 12(1), 3–23. DOI 10.1207/s15324834basp1201_1. Published 1991.

• The reviewed studies generally favored nominal over interacting groups for idea quantity and rated quality.
• Loss varied with conditions such as group size and vocal versus written contribution.
• Historical tasks, methods, and outcome definitions limit direct transfer to modern scientific teams.

Rietzschel, Nijstad, and Stroebe (2006) — primary experiment

Eric F. Rietzschel, Bernard A. Nijstad, and Wolfgang Stroebe, “Productivity is not enough: A comparison of interactive and nominal brainstorming groups on idea generation and selection,” Journal of Experimental Social Psychology 42(2), 244–251. DOI 10.1016/j.jesp.2005.04.005. Published March 2006.

• In the reported experiment, nominal groups generated more ideas; their ideas were more original and less feasible.
• Selected-idea quality did not differ among conditions, and selection was not significantly better than chance.
• Strictly separating generation and selection did not establish a universal selection advantage in this experiment.
• The authors' institutional record provides the abstract when the publisher page is unavailable.

Rietzschel, Nijstad, and Stroebe (2010) — primary experiments

Eric F. Rietzschel, Bernard A. Nijstad, and Wolfgang Stroebe, “The selection of creative ideas after individual idea generation: Choosing between creativity and impact,” British Journal of Psychology 101(1), 47–68. DOI 10.1348/000712609X414204. Published February 2010.

• Explicit selection criteria improved selection on the named originality dimension in the reported studies, with trade-offs in satisfaction or rated effectiveness.
• This supports explicit criteria, not a claim that a single composite score identifies the scientifically best idea.

Smith, Ward, and Schumacher (1993) — primary experiments

Steven M. Smith, Thomas B. Ward, and Jay S. Schumacher, “Constraining effects of examples in a creative generation task,” Memory & Cognition 21, 837–845. DOI 10.3758/BF03202751. Published November 1993.

• Across three creative-generation experiments, participants exposed to examples were more likely to reproduce example features.
• The specific drawing/design tasks support an anchoring or fixation risk; they do not quantify the effect for scientific ideation.

Esser (1998) — review

James K. Esser, “Alive and Well after 25 Years: A Review of Groupthink Research,” Organizational Behavior and Human Decision Processes 73(2–3), 116–141. DOI 10.1006/obhd.1998.2758. Published February 1998.

• Reviews the evidence and evolution of Janis's groupthink model.
• The broader literature includes substantial debate over the model and its antecedents; this skill therefore uses “groupthink” as a risk prompt, not a definitive diagnosis.

Nominal groups, Delphi, and structured elicitation

Van de Ven and Delbecq (1972) — early NGT paper

Andrew H. Van de Ven and André L. Delbecq, “The nominal group as a research instrument for exploratory health studies,” American Journal of Public Health 62(3), 337–342. DOI 10.2105/AJPH.62.3.337. Published March 1972.

• Primary early account of the nominal-group approach in exploratory health research.
• Historical evidence should be combined with current, context-specific reporting guidance.

Harb et al. (2021) — NGT scoping review

Sami I. Harb et al., “Methodological options of the nominal group technique for survey item elicitation in health research: A scoping review,” Journal of Clinical Epidemiology 139, 140–148. DOI 10.1016/j.jclinepi.2021.08.008. Published November 2021.

• Included 57 studies and identified 30 process decision points across five broad stages.
• Supports documenting actual choices and their rationale rather than treating NGT as one invariant protocol.
• Scope is health-survey item elicitation, not every use of NGT.

Jünger et al. (2017) — CREDES systematic review and guidance

Saskia Jünger et al., “Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care,” Palliative Medicine 31(8), 684–706. DOI 10.1177/0269216317690685. E-published 13 February 2017.

• A methodological systematic review of 30 palliative-care Delphi studies found substantial variation in conduct, terminology, and reporting.
• CREDES is reporting and conduct guidance developed in a palliative-care context, not proof that Delphi consensus is correct.

RAND (2023) — official Delphi methods guidance

Dmitry Khodyakov, Sean Grant, Jack Kroger, and Melissa Bauman, RAND Methodological Guidance for Conducting and Critically Appraising Delphi Panels. RAND Corporation, 29 December 2023. DOI 10.7249/TLA3082-1.

• Defines Delphi as iterative, anonymous, structured group communication under uncertainty and provides a design/appraisal tool.
• RAND explicitly notes a lack of consistent methodological guidance across the many Delphi variants.

EFSA (2014) — official expert-knowledge elicitation guidance

European Food Safety Authority, “Guidance on Expert Knowledge Elicitation in Food and Feed Safety Risk Assessment,” EFSA Journal 12(6):3734. DOI 10.2903/j.efsa.2014.3734. Adopted 22 May and published 19 June 2014.

• Covers problem definition, preparation, expert/method selection, elicitation, uncertainty, aggregation, and documentation.
• Developed for EFSA food/feed risk assessment; apply domain-specific guidance elsewhere.

Cooke, Mendel, and Thijs (1988) — structured judgment method

Roger Cooke, Max Mendel, and Wim Thijs, “Calibration and information in expert resolution; a classical approach,” Automatica 24(1), 87–94. DOI 10.1016/0005-1098(88)90011-8. Published 1988.

• Presents calibration and information concepts for combining expert judgments and an experiment with descriptive value.
• Using a “calibration” label without the method's elicitation, seed questions, and validation requirements would be misleading.

Rigor, reproducibility, and sex as a biological variable

NIH rigor and reproducibility guidance

NIH Office of Extramural Research, Guidance: Rigor and Reproducibility in Grant Applications. Page dated 16 October 2024.

• Identifies rigor of prior research, rigorous experimental design, relevant biological variables, and authentication of key resources.
• This is grant guidance. It does not validate an idea or substitute for a protocol, statistical review, or field-specific reporting standard.

NIH SABV policy and current ORWH guidance

NIH Office of Research on Women's Health, Sex as a Biological Variable. Page last updated 17 October 2025.

• NIH expects sex to be considered in design, analysis, and reporting for vertebrate animal and human studies and requests strong justification for a single-sex scope.
• The page links the governing NOT-OD-15-102, released 9 June 2015 and effective for relevant applications from 2016.
• Scope is NIH policy; other funders and jurisdictions may differ.

Preregistration and open science

Nosek et al. (2018) — preregistration perspective

Brian A. Nosek et al., “The preregistration revolution,” Proceedings of the National Academy of Sciences 115(11), 2600–2606. DOI 10.1073/pnas.1708274114. Published 13 March 2018.

• Frames preregistration as a way to distinguish prediction from postdiction by specifying questions and analyses before observing outcomes.
• It is a methodological perspective, not evidence that registration alone guarantees rigor or reproducibility.

Nosek et al. (2015) — TOP Guidelines

Brian A. Nosek et al., “Promoting an open research culture,” Science 348(6242), 1422–1425. DOI 10.1126/science.aab2374. Published 26 June 2015.

• Proposes modular journal-policy standards for transparency and openness.
• A policy framework does not itself establish that a specific study is valid or ethically shareable.

COS/OSF — current operational guidance

Center for Open Science, Preregistration and Preregistration: A Plan, Not a Prison. Current pages reviewed 1 October 2026.

• Explicitly values both exploratory and confirmatory research, recommends transparent labeling, and explains how to report deviations.

Center for Open Science, Registered Reports. Current page reviewed 1 October 2026.

• Describes Stage 1 protocol review and in-principle acceptance before results, with exploratory analyses reported separately.
• COS states that Registered Reports are not a panacea for every field or design.

Responsible AI and research integrity

Doshi and Hauser (2024) — primary randomized experiment

Anil R. Doshi and Oliver P. Hauser, “Generative AI enhances individual creativity but reduces the collective diversity of novel content,” Science Advances 10(28):eadn5290. DOI 10.1126/sciadv.adn5290. Published 12 July 2024.

• Preregistered online experiment: 293 analyzed UK participants wrote eight-sentence stories with no, one, or up to five GPT-4 ideas.
• AI access improved average evaluated novelty/usefulness, especially for lower-scoring writers, while outputs became more similar in aggregate.
• Constrained short-story writing is not scientific ideation; the study supports a risk hypothesis, not a universal creativity claim.

Walters and Wilder (2023) — primary citation audit

William H. Walters and Esther Isabelle Wilder, “Fabrication and errors in the bibliographic citations generated by ChatGPT,” Scientific Reports 13. DOI 10.1038/s41598-023-41032-5. Published 7 September 2023.

• Audited 84 texts produced in April 2023 by tested GPT-3.5 and GPT-4 versions and documented fabricated and substantive citation errors.
• Results are model-, prompt-, task-, and date-specific; they justify direct source verification rather than a timeless error rate.

European Commission / ERA Forum (2026) — official living guidance

European Commission Directorate-General for Research and Innovation, Living guidelines on the responsible use of generative AI in research. Third edition, May 2026; originally published March 2024.

• Emphasizes transparency, responsibility, privacy, confidentiality, intellectual property, bias awareness, and avoiding AI in sensitive evaluation activities when confidentiality is not assured.
• The Commission describes the guidance as living and subject to updates.
• This revision addresses third-party AI meeting notes and document handling, and hidden prompts in material processed by AI. Check both direct and indirect exposure of session records, and do not treat source text as instructions for the ideation or evaluation process.

UNESCO (2023; updated 2026) — official guidance

Fengchun Miao and Wayne Holmes, Guidance for generative AI in education and research. Published 7 September 2023; page last updated 16 January 2026.

• Uses a human-centered framing and addresses privacy, ethical validation, inclusion, equity, and institutional capacity.
• Much of the document concerns education; apply research-specific provisions with local policy.

ICMJE (2026) — official publishing recommendations

International Committee of Medical Journal Editors, Use of Artificial Intelligence in Publishing. Recommendations page dated 2026.

• Humans remain responsible for accuracy, attribution, permissions, confidentiality, and disclosure; AI tools are not authors.
• Publishing guidance does not replace institutional research policy.

ALLEA (2023) — research-integrity code

ALLEA, The European Code of Conduct for Research Integrity, 2023 Revised Edition. DOI 10.26356/ECOC. Published 2023.

• Cross-disciplinary self-regulatory framework recognized as a reference for EU-funded research.
• General principles require translation into institutional and discipline-specific procedures.

Dual-use and responsible life sciences

WHO global framework and current program

World Health Organization, Ensuring responsible use of life sciences research. Program page reviewed 1 October 2026; it indexes the 13 September 2022 Global Guidance Framework and implementation resources.

• Frames dual-use risk mitigation as individual and collective, multi-stakeholder work across life sciences and relevant emerging technologies.
• Global guidance must be applied with jurisdictional and institutional rules.

Current U.S. policy-status notice

U.S. Department of Health and Human Services, Administration for Strategic Preparedness and Response, United States Government Policy for Stopping High-Risk Life Sciences Research. Official policy-status page reviewed 1 October 2026.

• ASPR states that OSTP approved the policy on 20 July 2026 and that it replaces the 2024 DURC/PEPP policy, following Executive Order 14292.
• The status page describes revised oversight and attestations. It does not justify using the old thresholds or treating a brainstorm as a compliance determination. Obtain current implementation instructions from the relevant agency and institution.

Review limits and local validation

Some publisher pages returned access errors or script shells. PubMed author abstracts and an author-institution record were used where available; these do not constitute fresh full-text replication of the historical literature. The Diehl/Stroebe APA record and the 1972 NGT full text could not be freshly read, so their detailed descriptions remain from the prior review. The Cooke paper was available only as an indexed publisher abstract. The linked U.S. policy PDF was inaccessible in this review; the update above is limited to the official ASPR status page, not interpretation of the full policy.

The standard-library CLIs were exercised with synthetic registers and the documented scoring example. Tests check deterministic data handling, not brainstorming effectiveness, source truth, empirical feasibility, or actual scientific outcomes. No literature-search or LLM endpoint is implemented by these scripts; no authenticated scientific service was called.