Skills › Scientific Agent Skills › database-lookup › database_selection_guide.md
database_selection_guide.md
169 lines · skills/database-lookup/references/database_selection_guide.md at 5f2bc03
Most marks come from matching words in the file, so treat them as pointers.
1# Database Selection Guide
3Which database answers which question, grouped by domain: physics and astronomy, earth and
4environmental sciences, chemistry and drugs, materials science and crystallography,
5biology and genomics, disease and clinical, patents and regulatory, economics and finance,
6social sciences and demographics, and cross-domain queries.
Database Selection Guide
Which database answers which question, grouped by domain: physics and astronomy, earth and environmental sciences, chemistry and drugs, materials science and crystallography, biology and genomics, disease and clinical, patents and regulatory, economics and finance, social sciences and demographics, and cross-domain queries.
Raw lines 1 to 78## Database Selection Guide
10Match the user's intent to the right database(s). Many queries benefit from hitting multiple databases.
Database Selection Guide
Match the user's intent to the right database(s). Many queries benefit from hitting multiple databases.
Raw lines 8 to 1112### Physics & Astronomy
13| User is asking about... | Primary database(s) | Also consider |
14|---|---|---|
15| Near-Earth objects, asteroids | NASA (NeoWs) | — |
16| Mars rover images | NASA archival imagery (legacy Mars Rover Photos API retired) | — |
17| Exoplanets, orbital parameters | NASA Exoplanet Archive | — |
18| Astronomical objects by name/coordinates | SIMBAD | SDSS |
19| Galaxy/star spectra, photometry | SDSS | SIMBAD |
20| Physical constants | NIST | — |
21| Atomic spectra, spectral lines | NIST (ASD) | — |
Physics & Astronomy
| User is asking about... | Primary database(s) | Also consider |
|---|---|---|
| Near-Earth objects, asteroids | NASA (NeoWs) | — |
| Mars rover images | NASA archival imagery (legacy Mars Rover Photos API retired) | — |
| Exoplanets, orbital parameters | NASA Exoplanet Archive | — |
| Astronomical objects by name/coordinates | SIMBAD | SDSS |
| Galaxy/star spectra, photometry | SDSS | SIMBAD |
| Physical constants | NIST | — |
| Atomic spectra, spectral lines | NIST (ASD) | — |
23### Earth & Environmental Sciences
24| User is asking about... | Primary database(s) | Also consider |
25|---|---|---|
26| Earthquakes, seismic events | USGS Earthquakes | — |
27| Water data, streamflow, groundwater | USGS Water Data APIs (legacy NWIS migration) | — |
28| Weather (current, forecast, historical) | OpenWeatherMap | NOAA |
29| Climate data, historical weather stations | NOAA (CDO) | — |
30| Air quality, toxic releases | EPA (Envirofacts) | — |
Earth & Environmental Sciences
| User is asking about... | Primary database(s) | Also consider |
|---|---|---|
| Earthquakes, seismic events | USGS Earthquakes | — |
| Water data, streamflow, groundwater | USGS Water Data APIs (legacy NWIS migration) | — |
| Weather (current, forecast, historical) | OpenWeatherMap | NOAA |
| Climate data, historical weather stations | NOAA (CDO) | — |
| Air quality, toxic releases | EPA (Envirofacts) | — |
32### Chemistry & Drugs
33| User is asking about... | Primary database(s) | Also consider |
34|---|---|---|
35| Chemical compounds, molecules | PubChem | ChEMBL |
36| Molecular properties (weight, formula, SMILES) | PubChem | — |
37| Drug synonyms, CAS numbers | PubChem (synonyms) | DrugBank |
38| Bioactivity data, IC50, binding assays | ChEMBL | BindingDB, PubChem |
39| Drug binding affinities (Ki, IC50, Kd) | ChEMBL, BindingDB | PubChem |
40| Drug-target interactions | ChEMBL, DrugBank | BindingDB, Open Targets |
41| Ligands for a protein target (by UniProt) | BindingDB | ChEMBL |
42| Target identification from compound structure | BindingDB (SMILES similarity) | ChEMBL |
43| Drug labels, adverse events, recalls | FDA (OpenFDA) | DailyMed |
44| Drug labels (structured product labels) | DailyMed | FDA (OpenFDA) |
45| Drug pharmacology, indications | DrugBank | FDA |
46| Chemical cross-referencing | PubChem (xrefs) | ChEMBL |
47| Commercially available compounds for screening | ZINC | PubChem |
48| Similarity/substructure search (purchasable) | ZINC | PubChem, ChEMBL |
49| Drug-like compound libraries, building blocks | ZINC | — |
50| FDA-approved drug structures | ZINC (fda subset) | PubChem, FDA |
51| Compound purchasability, vendor catalogs | ZINC | — |
Chemistry & Drugs
| User is asking about... | Primary database(s) | Also consider |
|---|---|---|
| Chemical compounds, molecules | PubChem | ChEMBL |
| Molecular properties (weight, formula, SMILES) | PubChem | — |
| Drug synonyms, CAS numbers | PubChem (synonyms) | DrugBank |
| Bioactivity data, IC50, binding assays | ChEMBL | BindingDB, PubChem |
| Drug binding affinities (Ki, IC50, Kd) | ChEMBL, BindingDB | PubChem |
| Drug-target interactions | ChEMBL, DrugBank | BindingDB, Open Targets |
| Ligands for a protein target (by UniProt) | BindingDB | ChEMBL |
| Target identification from compound structure | BindingDB (SMILES similarity) | ChEMBL |
| Drug labels, adverse events, recalls | FDA (OpenFDA) | DailyMed |
| Drug labels (structured product labels) | DailyMed | FDA (OpenFDA) |
| Drug pharmacology, indications | DrugBank | FDA |
| Chemical cross-referencing | PubChem (xrefs) | ChEMBL |
| Commercially available compounds for screening | ZINC | PubChem |
| Similarity/substructure search (purchasable) | ZINC | PubChem, ChEMBL |
| Drug-like compound libraries, building blocks | ZINC | — |
| FDA-approved drug structures | ZINC (fda subset) | PubChem, FDA |
| Compound purchasability, vendor catalogs | ZINC | — |
53### Materials Science & Crystallography
54| User is asking about... | Primary database(s) | Also consider |
55|---|---|---|
56| Materials by formula or elements | Materials Project | COD |
57| Band gap, electronic structure | Materials Project | — |
58| Crystal structures, CIF files | COD | Materials Project |
59| Elastic/mechanical properties | Materials Project | — |
60| Formation energy, thermodynamics | Materials Project | — |
61| Cell parameters, space groups | COD | Materials Project |
Materials Science & Crystallography
| User is asking about... | Primary database(s) | Also consider |
|---|---|---|
| Materials by formula or elements | Materials Project | COD |
| Band gap, electronic structure | Materials Project | — |
| Crystal structures, CIF files | COD | Materials Project |
| Elastic/mechanical properties | Materials Project | — |
| Formation energy, thermodynamics | Materials Project | — |
| Cell parameters, space groups | COD | Materials Project |
63### Biology & Genomics
64| User is asking about... | Primary database(s) | Also consider |
65|---|---|---|
66| Biological pathways | Reactome, KEGG | — |
67| What pathways a gene/protein is in | Reactome (mapping), KEGG | — |
68| Enzyme kinetics, catalytic activity | BRENDA | KEGG |
69| Metabolomics studies, metabolite profiles | Metabolomics Workbench | PubChem |
70| m/z or exact mass lookup | Metabolomics Workbench (moverz/exactmass) | PubChem |
71| Protein sequence, function, annotation | UniProt | Ensembl |
72| Protein-protein interactions | STRING | BioGRID |
73| Gene information, genomic location | NCBI Gene | Ensembl |
74| Genome sequences, variants, transcripts | Ensembl | NCBI Gene |
75| Gene expression datasets | GEO (NCBI E-utilities) | — |
76| Gene expression across tissues | GTEx | Human Protein Atlas |
77| Gene expression signatures (CMap/L1000) | LINCS L1000 | GEO |
78| Gene set enrichment vs GEO | RummaGEO | GEO |
79| Protein sequences (NCBI) | NCBI Protein | UniProt |
80| Taxonomic classification | NCBI Taxonomy | — |
81| SNP/variant data (dbSNP) | dbSNP | ClinVar, gnomAD |
82| Variant consequence / CADD PHRED | Ensembl VEP (`CADD=1`) | dbSNP |
83| Noncoding regulatory evidence | RegulomeDB | ENCODE, JASPAR |
84| Cached variant annotation bundle | MyVariant.info | Ensembl VEP (check model/build/release) |
85| Population variant frequencies | gnomAD | dbSNP |
86| Sequencing run metadata | SRA | ENA, GEO |
87| Nucleotide sequences (European archive) | ENA | SRA, NCBI Gene |
88| Genome assemblies, raw reads (European) | ENA | SRA, Ensembl |
89| Cross-references from sequence accessions | ENA (xref) | NCBI Gene, UniProt |
90| Viral sequence datasets with NCBI Virus-style filters | `gget virus` deterministic layer | SRA, ENA, NCBI Protein |
91| Genome annotations, tracks | UCSC Genome Browser | Ensembl |
92| 3D protein structures (experimental) | PDB (RCSB) | EMDB |
93| 3D protein structures (predicted) | AlphaFold DB | PDB |
94| EM maps, cryo-EM structures | EMDB | PDB |
95| Protein families, domains | InterPro | UniProt |
96| Chemical entities (biological) | ChEBI | PubChem |
97| Protein/genetic interactions | BioGRID | STRING |
98| Gene function annotations (GO terms) | QuickGO | Gene Ontology |
99| Regulatory elements, ChIP-seq, ATAC-seq | ENCODE | — |
100| TF binding profiles/motifs | JASPAR | ENCODE |
101| Protein expression across tissues | Human Protein Atlas | UniProt |
102| Single-cell atlas projects | Human Cell Atlas | — |
103| Proteomics datasets | PRIDE | — |
104| Mouse gene data | MouseMine | NCBI Gene |
105| Plasmid repository | Addgene | — |
107**Organism/species matters.** Most biology databases cover multiple organisms. If the user's query is about a specific organism, pass it explicitly — don't assume human. Common patterns: Ensembl uses `{species}` in the URL path (e.g. `homo_sapiens`), STRING/BioGRID/QuickGO use NCBI taxon IDs with different parameter names (`species`, `taxId`, `taxonId` respectively; human `9606`, mouse `10090`), UniProt uses `organism_id:9606` in search queries, KEGG uses organism codes (`hsa`, `mmu`). GTEx and Human Protein Atlas are human-only. Check the reference file for each database's specific parameter.
109**Viral sequence retrieval is high risk.** For NCBI Virus-style requests with filters such as host, geography, collection dates, sequence length, completeness, ambiguous bases, segment, lab passage, source database, or protein annotation, prefer the `gget` skill's `gget virus` deterministic retrieval layer over hand-assembling browser or API workflows. If you must use SRA/ENA/NCBI APIs directly, document which filters were enforced server-side and which were validated locally, then reconcile final accession counts.
Biology & Genomics
| User is asking about... | Primary database(s) | Also consider |
|---|---|---|
| Biological pathways | Reactome, KEGG | — |
| What pathways a gene/protein is in | Reactome (mapping), KEGG | — |
| Enzyme kinetics, catalytic activity | BRENDA | KEGG |
| Metabolomics studies, metabolite profiles | Metabolomics Workbench | PubChem |
| m/z or exact mass lookup | Metabolomics Workbench (moverz/exactmass) | PubChem |
| Protein sequence, function, annotation | UniProt | Ensembl |
| Protein-protein interactions | STRING | BioGRID |
| Gene information, genomic location | NCBI Gene | Ensembl |
| Genome sequences, variants, transcripts | Ensembl | NCBI Gene |
| Gene expression datasets | GEO (NCBI E-utilities) | — |
| Gene expression across tissues | GTEx | Human Protein Atlas |
| Gene expression signatures (CMap/L1000) | LINCS L1000 | GEO |
| Gene set enrichment vs GEO | RummaGEO | GEO |
| Protein sequences (NCBI) | NCBI Protein | UniProt |
| Taxonomic classification | NCBI Taxonomy | — |
| SNP/variant data (dbSNP) | dbSNP | ClinVar, gnomAD |
| Variant consequence / CADD PHRED | Ensembl VEP (CADD=1) | dbSNP |
| Noncoding regulatory evidence | RegulomeDB | ENCODE, JASPAR |
| Cached variant annotation bundle | MyVariant.info | Ensembl VEP (check model/build/release) |
| Population variant frequencies | gnomAD | dbSNP |
| Sequencing run metadata | SRA | ENA, GEO |
| Nucleotide sequences (European archive) | ENA | SRA, NCBI Gene |
| Genome assemblies, raw reads (European) | ENA | SRA, Ensembl |
| Cross-references from sequence accessions | ENA (xref) | NCBI Gene, UniProt |
| Viral sequence datasets with NCBI Virus-style filters | gget virus deterministic layer | SRA, ENA, NCBI Protein |
| Genome annotations, tracks | UCSC Genome Browser | Ensembl |
| 3D protein structures (experimental) | PDB (RCSB) | EMDB |
| 3D protein structures (predicted) | AlphaFold DB | PDB |
| EM maps, cryo-EM structures | EMDB | PDB |
| Protein families, domains | InterPro | UniProt |
| Chemical entities (biological) | ChEBI | PubChem |
| Protein/genetic interactions | BioGRID | STRING |
| Gene function annotations (GO terms) | QuickGO | Gene Ontology |
| Regulatory elements, ChIP-seq, ATAC-seq | ENCODE | — |
| TF binding profiles/motifs | JASPAR | ENCODE |
| Protein expression across tissues | Human Protein Atlas | UniProt |
| Single-cell atlas projects | Human Cell Atlas | — |
| Proteomics datasets | PRIDE | — |
| Mouse gene data | MouseMine | NCBI Gene |
| Plasmid repository | Addgene | — |
Organism/species matters. Most biology databases cover multiple organisms. If the user's query is about a specific organism, pass it explicitly — don't assume human. Common patterns: Ensembl uses {species} in the URL path (e.g. homo_sapiens), STRING/BioGRID/QuickGO use NCBI taxon IDs with different parameter names (species, taxId, taxonId respectively; human 9606, mouse 10090), UniProt uses organism_id:9606 in search queries, KEGG uses organism codes (hsa, mmu). GTEx and Human Protein Atlas are human-only. Check the reference file for each database's specific parameter.
Viral sequence retrieval is high risk. For NCBI Virus-style requests with filters such as host, geography, collection dates, sequence length, completeness, ambiguous bases, segment, lab passage, source database, or protein annotation, prefer the gget skill's gget virus deterministic retrieval layer over hand-assembling browser or API workflows. If you must use SRA/ENA/NCBI APIs directly, document which filters were enforced server-side and which were validated locally, then reconcile final accession counts.
111### Disease & Clinical
112| User is asking about... | Primary database(s) | Also consider |
113|---|---|---|
114| Somatic mutations in cancer | COSMIC | Open Targets, cBioPortal |
115| Cancer genomics (TCGA) | GDC (TCGA) | COSMIC, cBioPortal |
116| Cancer study mutations, CNA, expression | cBioPortal | GDC (TCGA), COSMIC |
117| Tumor clinical data (survival, staging) | cBioPortal | GDC (TCGA) |
118| Drug-target-disease associations | Open Targets | ChEMBL |
119| Gene-disease associations | DisGeNET | Open Targets, Monarch |
120| Mendelian disease-gene relationships | OMIM | NCBI Gene |
121| Variant clinical significance | ClinVar (NCBI) | OMIM |
122| GWAS SNP-trait associations | GWAS Catalog | — |
123| Disease-phenotype-gene links | Monarch Initiative | HPO |
124| Phenotype ontology, HPO terms | HPO | Monarch |
125| Pharmacogenomics, drug-gene interactions | ClinPGx (PharmGKB) | DrugBank |
126| Clinical trials for a drug/disease | ClinicalTrials.gov | FDA |
127| Disease-related expression data | GEO | Open Targets |
Disease & Clinical
| User is asking about... | Primary database(s) | Also consider |
|---|---|---|
| Somatic mutations in cancer | COSMIC | Open Targets, cBioPortal |
| Cancer genomics (TCGA) | GDC (TCGA) | COSMIC, cBioPortal |
| Cancer study mutations, CNA, expression | cBioPortal | GDC (TCGA), COSMIC |
| Tumor clinical data (survival, staging) | cBioPortal | GDC (TCGA) |
| Drug-target-disease associations | Open Targets | ChEMBL |
| Gene-disease associations | DisGeNET | Open Targets, Monarch |
| Mendelian disease-gene relationships | OMIM | NCBI Gene |
| Variant clinical significance | ClinVar (NCBI) | OMIM |
| GWAS SNP-trait associations | GWAS Catalog | — |
| Disease-phenotype-gene links | Monarch Initiative | HPO |
| Phenotype ontology, HPO terms | HPO | Monarch |
| Pharmacogenomics, drug-gene interactions | ClinPGx (PharmGKB) | DrugBank |
| Clinical trials for a drug/disease | ClinicalTrials.gov | FDA |
| Disease-related expression data | GEO | Open Targets |
129### Patents & Regulatory
130| User is asking about... | Primary database(s) | Also consider |
131|---|---|---|
132| Patents by keyword or technology | USPTO ODP PatentsView bulk (PatentSearch API paused) | — |
133| Patents by inventor or assignee | USPTO ODP PatentsView bulk (PatentSearch API paused) | — |
134| Patent prosecution status | USPTO ODP Patent File Wrapper (PEDS replacement) | — |
135| Trademark lookup | USPTO (TSDR) | — |
136| SEC company filings, 10-K, 10-Q | SEC EDGAR | — |
Patents & Regulatory
| User is asking about... | Primary database(s) | Also consider |
|---|---|---|
| Patents by keyword or technology | USPTO ODP PatentsView bulk (PatentSearch API paused) | — |
| Patents by inventor or assignee | USPTO ODP PatentsView bulk (PatentSearch API paused) | — |
| Patent prosecution status | USPTO ODP Patent File Wrapper (PEDS replacement) | — |
| Trademark lookup | USPTO (TSDR) | — |
| SEC company filings, 10-K, 10-Q | SEC EDGAR | — |
138### Economics & Finance
139| User is asking about... | Primary database(s) | Also consider |
140|---|---|---|
141| US economic time series (GDP, CPI, rates) | FRED | BEA |
142| Employment, wages, labor statistics | BLS | FRED |
143| GDP, national accounts | BEA | FRED, World Bank |
144| International development indicators | World Bank | FRED |
145| Interest rates, money supply | Federal Reserve | FRED |
146| Euro exchange rates, ECB monetary stats | ECB | — |
147| US debt, yield curves, fiscal data | US Treasury | FRED |
148| Stock prices, forex, crypto | Alpha Vantage | — |
149| Statistical data across many topics | Data Commons | — |
Economics & Finance
| User is asking about... | Primary database(s) | Also consider |
|---|---|---|
| US economic time series (GDP, CPI, rates) | FRED | BEA |
| Employment, wages, labor statistics | BLS | FRED |
| GDP, national accounts | BEA | FRED, World Bank |
| International development indicators | World Bank | FRED |
| Interest rates, money supply | Federal Reserve | FRED |
| Euro exchange rates, ECB monetary stats | ECB | — |
| US debt, yield curves, fiscal data | US Treasury | FRED |
| Stock prices, forex, crypto | Alpha Vantage | — |
| Statistical data across many topics | Data Commons | — |
151### Social Sciences & Demographics
152| User is asking about... | Primary database(s) | Also consider |
153|---|---|---|
154| US population, housing, income data | US Census | Data Commons |
155| EU statistics (economy, trade, health) | Eurostat | World Bank |
156| Global health indicators (mortality, disease) | WHO GHO | World Bank |
Social Sciences & Demographics
| User is asking about... | Primary database(s) | Also consider |
|---|---|---|
| US population, housing, income data | US Census | Data Commons |
| EU statistics (economy, trade, health) | Eurostat | World Bank |
| Global health indicators (mortality, disease) | WHO GHO | World Bank |
158### Cross-domain queries
159| User is asking about... | Primary database(s) | Also consider |
160|---|---|---|
161| Everything about a compound | PubChem + ChEMBL + DrugBank | BindingDB, ZINC, Reactome, FDA |
162| Everything about a gene | NCBI Gene + UniProt + Ensembl | Reactome, STRING, COSMIC, cBioPortal, ENA |
163| Everything about a variant | dbSNP + ClinVar + gnomAD | GWAS Catalog, COSMIC, cBioPortal |
164| Drug target pathways | ChEMBL + Reactome | Open Targets, GEO |
165| Prior art for a chemical invention | USPTO + PubChem | ChEMBL |
166| Everything about a material | Materials Project + COD | — |
167| US economic overview | FRED + BLS + BEA | Federal Reserve |
169When the user's query spans multiple domains (e.g. "what do we know about aspirin" or "find everything about BRCA1"), rank sources by authority and start with the 2-3 databases most likely to answer the question. Add more databases only when the first pass leaves a specific gap. Keep at most 5 independent API requests in flight at once.
Cross-domain queries
| User is asking about... | Primary database(s) | Also consider |
|---|---|---|
| Everything about a compound | PubChem + ChEMBL + DrugBank | BindingDB, ZINC, Reactome, FDA |
| Everything about a gene | NCBI Gene + UniProt + Ensembl | Reactome, STRING, COSMIC, cBioPortal, ENA |
| Everything about a variant | dbSNP + ClinVar + gnomAD | GWAS Catalog, COSMIC, cBioPortal |
| Drug target pathways | ChEMBL + Reactome | Open Targets, GEO |
| Prior art for a chemical invention | USPTO + PubChem | ChEMBL |
| Everything about a material | Materials Project + COD | — |
| US economic overview | FRED + BLS + BEA | Federal Reserve |
When the user's query spans multiple domains (e.g. "what do we know about aspirin" or "find everything about BRCA1"), rank sources by authority and start with the 2-3 databases most likely to answer the question. Add more databases only when the first pass leaves a specific gap. Keep at most 5 independent API requests in flight at once.
Raw lines 158 to 169