Skill Tracer
Skills › Scientific Agent Skills › pathway-enrichment › interpretation.md

interpretation.md

134 lines · skills/pathway-enrichment/references/interpretation.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Interpreting Enrichment Results
2

Interpreting Enrichment Results

Raw lines 1 to 2
3## Contents
4- [ORA vs GSEA: the statistics](#ora-vs-gsea-the-statistics)
5- [The background universe (ORA)](#the-background-universe-ora)
6- [Multiple-testing correction](#multiple-testing-correction)
7- [Reading GSEA output](#reading-gsea-output)
8- [Reducing redundant terms](#reducing-redundant-terms)
9- [Significance vs relevance](#significance-vs-relevance)
10- [Reproducibility checklist](#reproducibility-checklist)
11- [Publication table template](#publication-table-template)
12- [Common misinterpretations](#common-misinterpretations)

Contents

  • ORA vs GSEA: the statistics
  • The background universe (ORA)
  • Multiple-testing correction
  • Reading GSEA output
  • Reducing redundant terms
  • Significance vs relevance
  • Reproducibility checklist
  • Publication table template
  • Common misinterpretations
Raw lines 3 to 13
14## ORA vs GSEA: the statistics
16**ORA** asks: among my *k* hits (out of a background of *N* genes), are more in
17gene set *S* (size *K*) than expected by chance? This is a hypergeometric /
18Fisher's exact test. It depends entirely on the threshold used to define hits and
19on the background *N*. Good when there is a clear, strong hit list.
21**GSEA** asks: walking down the *fully ranked* list of all tested genes, is gene
22set *S* concentrated near the top (or bottom)? It uses a weighted Kolmogorov–
23Smirnov-like running sum; significance comes from permutations. No arbitrary
24threshold; sensitive to coordinated, modest shifts across many genes. Better when
25effects are broad/subtle or when a hit list would be very short or very long.
27Rule of thumb: a discrete hit list → ORA; a ranked table with per-gene scores →
28GSEA. They answer different questions and can legitimately disagree.

ORA vs GSEA: the statistics

ORA asks: among my k hits (out of a background of N genes), are more in gene set S (size K) than expected by chance? This is a hypergeometric / Fisher's exact test. It depends entirely on the threshold used to define hits and on the background N. Good when there is a clear, strong hit list.

GSEA asks: walking down the fully ranked list of all tested genes, is gene set S concentrated near the top (or bottom)? It uses a weighted Kolmogorov– Smirnov-like running sum; significance comes from permutations. No arbitrary threshold; sensitive to coordinated, modest shifts across many genes. Better when effects are broad/subtle or when a hit list would be very short or very long.

Rule of thumb: a discrete hit list → ORA; a ranked table with per-gene scores → GSEA. They answer different questions and can legitimately disagree.

Raw lines 14 to 29
30## The background universe (ORA)
32The background (the "domain" / universe) is the set of genes that *could* have
33appeared as a hit. For RNA-seq that is the set of **expressed/tested genes**, not
34all ~20,000 protein-coding genes. Using too large a background makes ordinary
35housekeeping categories look significant — the most common way ORA results
36mislead.
38- GSEApy 1.3.1 routes explicit online background lists through Speedrichr;
39 standard Enrichr uses its default background. **g:Profiler** accepts
40 `domain_scope="custom"` and `background`; local **`gp.enrich()`** with a pinned
41 GMT and explicit gene list makes the universe directly inspectable.
42- The background must contain the query, use the same ID mapping as the query
43 and library, and represent the actual selection opportunity (tested genes, not
44 a separately chosen set that makes results smaller). Record N, k, K and overlap
45 after mapping/intersection. Length, abundance and detection-dependent selection
46 can bias ORA even with the right universe; consider a bias-aware null.

The background universe (ORA)

The background (the "domain" / universe) is the set of genes that could have appeared as a hit. For RNA-seq that is the set of expressed/tested genes, not all ~20,000 protein-coding genes. Using too large a background makes ordinary housekeeping categories look significant — the most common way ORA results mislead.

  • GSEApy 1.3.1 routes explicit online background lists through Speedrichr; standard Enrichr uses its default background. g:Profiler accepts domain_scope="custom" and background; local **gp.enrich()** with a pinned GMT and explicit gene list makes the universe directly inspectable.
  • The background must contain the query, use the same ID mapping as the query and library, and represent the actual selection opportunity (tested genes, not a separately chosen set that makes results smaller). Record N, k, K and overlap after mapping/intersection. Length, abundance and detection-dependent selection can bias ORA even with the right universe; consider a bias-aware null.
Raw lines 30 to 47
48## Multiple-testing correction
50- **Benjamini–Hochberg (FDR)** — used for Enrichr/gseapy ORA `Adjusted P-value`.
51 Classic GSEA `FDR q-val` instead uses normalized enrichment-score permutation
52 distributions. GSEApy 1.3.1 multilevel GSEA uses BH; state the backend.
53- **g:SCS** — g:Profiler's default; accounts for the correlated structure of GO
54 and overlapping terms. It is a different error-control procedure, not an FDR
55 estimate; avoid a universal claim that it is superior to BH.
56- **Bonferroni** controls family-wise error and does not require independent
57 tests; it can be conservative with dependent/overlapping terms.
59Define the testing family before looking at results. Classic GSEApy 1.3.1 scopes
60FDR by library prefix (`library__term`), while multilevel BH covers all submitted
61terms. Local GSEApy ORA applies BH to positive-overlap terms only. g:Profiler
62returns adjusted `p_value` and source-specific domain metadata. None of these
63automatically corrects across every cluster, contrast, library and threshold you
64tried. Report those families and avoid selecting the most favorable run.

Multiple-testing correction

  • Benjamini–Hochberg (FDR) — used for Enrichr/gseapy ORA Adjusted P-value. Classic GSEA FDR q-val instead uses normalized enrichment-score permutation distributions. GSEApy 1.3.1 multilevel GSEA uses BH; state the backend.
  • g:SCS — g:Profiler's default; accounts for the correlated structure of GO and overlapping terms. It is a different error-control procedure, not an FDR estimate; avoid a universal claim that it is superior to BH.
  • Bonferroni controls family-wise error and does not require independent tests; it can be conservative with dependent/overlapping terms.

Define the testing family before looking at results. Classic GSEApy 1.3.1 scopes FDR by library prefix (library__term), while multilevel BH covers all submitted terms. Local GSEApy ORA applies BH to positive-overlap terms only. g:Profiler returns adjusted p_value and source-specific domain metadata. None of these automatically corrects across every cluster, contrast, library and threshold you tried. Report those families and avoid selecting the most favorable run.

Raw lines 48 to 65
66## Reading GSEA output
68- **NES (normalized enrichment score)** — the headline metric; normalized for set
69 size and the chosen null. Compare cautiously within the same method/run; it
70 is not a fold-change or an interchangeable activity scale across cohorts. Sign = direction (positive = enriched at
71 the top of your ranking, e.g., up in the test condition).
72- **FDR q-val** — significance; filter on this. GSEA recommends `< 0.05` for
73 gene-set permutations (including preranked analyses); the exploratory `< 0.25`
74 convention applies to phenotype permutations. State the chosen threshold.
75- **Leading-edge genes** (`Lead_genes`) — the subset of genes that drive the
76 signal: hits at/before the positive peak or at/after the negative trough. Report these; they are the concrete
77 biology and are useful for overlap/redundancy analysis.

Reading GSEA output

  • NES (normalized enrichment score) — the headline metric; normalized for set size and the chosen null. Compare cautiously within the same method/run; it is not a fold-change or an interchangeable activity scale across cohorts. Sign = direction (positive = enriched at the top of your ranking, e.g., up in the test condition).
  • FDR q-val — significance; filter on this. GSEA recommends < 0.05 for gene-set permutations (including preranked analyses); the exploratory < 0.25 convention applies to phenotype permutations. State the chosen threshold.
  • Leading-edge genes (Lead_genes) — the subset of genes that drive the signal: hits at/before the positive peak or at/after the negative trough. Report these; they are the concrete biology and are useful for overlap/redundancy analysis.
Raw lines 66 to 78
79## Reducing redundant terms
81GO and large pathway sets return many overlapping terms describing the same
82biology. Don't list 40 near-duplicates. Options:
83- **Enrichment map** — graph with terms as nodes and edges weighted by gene
84 overlap (Jaccard/overlap coefficient); cluster it and label clusters. gseapy:
85 `gp.enrichment_map(...)`; render with `networkx` (see the networkx skill).
86- **Leading-edge / gene overlap clustering** — group terms sharing most genes;
87 keep one representative per group.
88- **Parent terms / semantic similarity** — collapse child GO terms to a parent;
89 REVIGO-style reduction by semantic similarity.
90- Report a representative term per cluster plus the count of related terms.

Reducing redundant terms

GO and large pathway sets return many overlapping terms describing the same biology. Don't list 40 near-duplicates. Options:

  • Enrichment map — graph with terms as nodes and edges weighted by gene overlap (Jaccard/overlap coefficient); cluster it and label clusters. gseapy: gp.enrichment_map(...); render with networkx (see the networkx skill).
  • Leading-edge / gene overlap clustering — group terms sharing most genes; keep one representative per group.
  • Parent terms / semantic similarity — collapse child GO terms to a parent; REVIGO-style reduction by semantic similarity.
  • Report a representative term per cluster plus the count of related terms.
Raw lines 79 to 91
Names a skill networkx :85
92## Significance vs relevance
94- Check the **overlap count**, not just the p-value. "Term enriched, padj=0.01"
95 with 2 genes out of a 1500-gene set is rarely meaningful.
96- Watch **gene-set size**: tiny sets reach significance with few genes; huge,
97 generic sets ("metabolic process") are uninformative — the `min_size`/`max_size`
98 filters (15–500) exist for this reason.
99- Assess power relative to the universe and term size; there is no universal
100 minimum/maximum hit-list length. GSEA requires an appropriate complete ranking,
101 which cannot be reconstructed from a hit list alone.

Significance vs relevance

  • Check the overlap count, not just the p-value. "Term enriched, padj=0.01" with 2 genes out of a 1500-gene set is rarely meaningful.
  • Watch gene-set size: tiny sets reach significance with few genes; huge, generic sets ("metabolic process") are uninformative — the min_size/max_size filters (15–500) exist for this reason.
  • Assess power relative to the universe and term size; there is no universal minimum/maximum hit-list length. GSEA requires an appropriate complete ranking, which cannot be reconstructed from a hit list alone.
Raw lines 92 to 102
103## Reproducibility checklist
105- Record exact **library names and versions/date** (Enrichr/GO libraries drift).
106- Record the **background** used (or state the default).
107- For GSEA, record `permutation_num`, `seed`, `min_size`, `max_size`, weight, and
108 the **ranking metric** (e.g., DESeq2 `stat`).
109- State the **organism** and **gene-ID namespace**.
110- Save the full results table, not just the filtered top hits.

Reproducibility checklist

  • Record exact library names and versions/date (Enrichr/GO libraries drift).
  • Record the background used (or state the default).
  • For GSEA, record permutation_num, seed, min_size, max_size, weight, and the ranking metric (e.g., DESeq2 stat).
  • State the organism and gene-ID namespace.
  • Save the full results table, not just the filtered top hits.
Raw lines 103 to 111
112## Publication table template
114Report a compact, reviewer-friendly table:
116| Term | Source | NES or odds ratio | Overlap / Set size | FDR | Key genes |
117|------|--------|------------------------------|--------------------|-----|-----------|
118| Interferon alpha response | Hallmark | NES +2.1 | 38/97 | 1e-4 | STAT1, IRF7, ISG15 |
120For ORA use Odds Ratio + Overlap (k/K); for GSEA use NES + leading-edge size.
121Note method, library version, background, and correction in the legend.

Publication table template

Report a compact, reviewer-friendly table:

TermSourceNES or odds ratioOverlap / Set sizeFDRKey genes
Interferon alpha responseHallmarkNES +2.138/971e-4STAT1, IRF7, ISG15

For ORA use Odds Ratio + Overlap (k/K); for GSEA use NES + leading-edge size. Note method, library version, background, and correction in the legend.

Raw lines 112 to 122
123## Common misinterpretations
125- "Enriched pathway X" does **not** mean pathway X is activated — ORA is
126 direction-agnostic unless you split up/down lists; GSEA NES sign gives rank direction, not necessarily activation (sets can contain inhibitors).
127- Overlapping significant GO terms are **not** independent findings.
128- Absence of enrichment ≠ absence of biology (power, annotation gaps, wrong
129 background, or ID mismatch can all hide real signal).
130- Don't compare raw ES across gene sets as effect sizes; NES still depends on the
131 null, gene-set collection and submitted ranking. Neither proves causality.
132- Report Monte Carlo resolution and method; nominal p-values at the simulation
133 floor do not establish arbitrarily tiny probabilities. More permutations do
134 not fix confounding, selection bias, or an invalid null.

Common misinterpretations

  • "Enriched pathway X" does not mean pathway X is activated — ORA is direction-agnostic unless you split up/down lists; GSEA NES sign gives rank direction, not necessarily activation (sets can contain inhibitors).
  • Overlapping significant GO terms are not independent findings.
  • Absence of enrichment ≠ absence of biology (power, annotation gaps, wrong background, or ID mismatch can all hide real signal).
  • Don't compare raw ES across gene sets as effect sizes; NES still depends on the null, gene-set collection and submitted ranking. Neither proves causality.
  • Report Monte Carlo resolution and method; nominal p-values at the simulation floor do not establish arbitrarily tiny probabilities. More permutations do not fix confounding, selection bias, or an invalid null.
Raw lines 123 to 134