Skill Tracer
Skills › Scientific Agent Skills › ontology-term-resolution › curation-rules.md

curation-rules.md

115 lines · skills/ontology-term-resolution/references/curation-rules.md at 5f2bc03

Open on GitHub
1# Curation rules
2
3How to choose among candidates, and what to do when the honest answer is "no term".
4

Curation rules

How to choose among candidates, and what to do when the honest answer is "no term".

Raw lines 1 to 4
5## The decision procedure
6
7Run it per string. Stop at the first step that gives a defensible answer.
8
91. **Exact label match in the expected ontology, from its defining ontology.** Check its
10 definition, current term detail, organism/context and target-schema constraints before accepting.
11 Multiple exact candidates require disambiguation; a bounded top-hit list is not exhaustive.
122. **Exact synonym match with an exact synonym annotation.** Apply the same context checks and
13 record the primary label. A related/broad/narrow or unscoped synonym is a curation candidate,
14 even when its spelling exactly matches the input.
153. **Exact match, wrong ontology.** Usually a category error in the source column, not a naming
16 problem — `hepatocyte` in a tissue field means the column mixes tissue and cell type. Fix the
17 column, do not force a match.
184. **Partial match only.** Do not accept silently. Either:
19 - normalise the input and retry (see below), or
20 - present the top candidates with their labels and let a human choose, or
21 - mark it unresolved.
225. **Nothing.** Mark unresolved and say so. An unresolved row is a correct output.
24`resolve_terms.py` implements steps 1–4's search side and labels every hit `exact_label`,
25`exact_synonym`, scoped/unscoped synonym matches, or `partial`. The judgement about a match is yours;
26the tool will not make it for you.

The decision procedure

Run it per string. Stop at the first step that gives a defensible answer.

  1. Exact label match in the expected ontology, from its defining ontology. Check its definition, current term detail, organism/context and target-schema constraints before accepting. Multiple exact candidates require disambiguation; a bounded top-hit list is not exhaustive.
  2. Exact synonym match with an exact synonym annotation. Apply the same context checks and record the primary label. A related/broad/narrow or unscoped synonym is a curation candidate, even when its spelling exactly matches the input.
  3. Exact match, wrong ontology. Usually a category error in the source column, not a naming problem — hepatocyte in a tissue field means the column mixes tissue and cell type. Fix the column, do not force a match.
  4. Partial match only. Do not accept silently. Either:
  5. normalise the input and retry (see below), or
  6. present the top candidates with their labels and let a human choose, or
  7. mark it unresolved.
  8. Nothing. Mark unresolved and say so. An unresolved row is a correct output.

resolve_terms.py implements steps 1–4's search side and labels every hit exact_label, exact_synonym, scoped/unscoped synonym matches, or partial. The judgement about a match is yours; the tool will not make it for you.

Raw lines 5 to 27
28## Normalisations worth retrying
30Cheap rewrites that convert a `partial` into an `exact_label`, in rough order of yield:
32- Separate preparation and donor notes from anatomy, preserving them in other fields. Do not
33 reduce `Liver - left lobe [FFPE]` to `liver`: the left-lobe qualifier carries anatomy.
34- Expand lab shorthand: `PBMC` → `peripheral blood mononuclear cell`, `WT` → the actual genotype,
35 `M`/`F` → `male`/`female` only with an explicit source codebook.
36- Reverse an inverted phrase: `ventricle, left` → `left ventricle`, `cortex, kidney` → `kidney
37 cortex`.
38- Singularise: `hepatocytes` → `hepatocyte`. Check the actual label; labels are not universally singular.
39- Anglicise or Americanise: ontology labels vary; try both `oesophagus` and `esophagus`.
40- Strip species prefixes: `human liver` → `liver` (species belongs in a separate NCBITaxon field).
42Do **not** normalise away hyphens, Greek letters, digits, or capitalised gene symbols — `CD4-positive`
43and `alpha-beta T cell` mean what they say, and `normalize_label()` deliberately folds only case and
44whitespace.
46When plain search keeps failing on lab shorthand, run `map_terms.py` with `--ontology` set
47(see `companion-apis.md`). ZOOMA can combine curated, lexical and embedding evidence. Inspect underlying provenance;
48its v2 wrapper can call non-curated matches `ZOOMA_INFERRED_FROM_CURATED`. Every HIGH/GOOD
49CURIE still goes through `validate_terms.py` before it is written down.

Normalisations worth retrying

Cheap rewrites that convert a partial into an exact_label, in rough order of yield:

  • Separate preparation and donor notes from anatomy, preserving them in other fields. Do not reduce Liver - left lobe [FFPE] to liver: the left-lobe qualifier carries anatomy.
  • Expand lab shorthand: PBMC → peripheral blood mononuclear cell, WT → the actual genotype, M/F → male/female only with an explicit source codebook.
  • Reverse an inverted phrase: ventricle, left → left ventricle, cortex, kidney → kidney cortex.
  • Singularise: hepatocytes → hepatocyte. Check the actual label; labels are not universally singular.
  • Anglicise or Americanise: ontology labels vary; try both oesophagus and esophagus.
  • Strip species prefixes: human liver → liver (species belongs in a separate NCBITaxon field).

Do not normalise away hyphens, Greek letters, digits, or capitalised gene symbols — CD4-positive and alpha-beta T cell mean what they say, and normalize_label() deliberately folds only case and whitespace.

When plain search keeps failing on lab shorthand, run map_terms.py with --ontology set (see companion-apis.md). ZOOMA can combine curated, lexical and embedding evidence. Inspect underlying provenance; its v2 wrapper can call non-curated matches ZOOMA_INFERRED_FROM_CURATED. Every HIGH/GOOD CURIE still goes through validate_terms.py before it is written down.

Raw lines 28 to 50
51## What "unresolved" should look like
53Never invent an ID to fill a cell. An unresolved row carries the original string, an empty ID, and
54the reason. Downstream that is a visible gap; an invented `UBERON:0002108` is a silent error that
55survives review because it looks exactly like a real ID.
57If a concept genuinely has no term and the project depends on it, the route is a new-term request
58to the ontology (GitHub issue on the ontology's tracker, with a definition and a reference), not a
59locally minted identifier.

What "unresolved" should look like

Never invent an ID to fill a cell. An unresolved row carries the original string, an empty ID, and the reason. Downstream that is a visible gap; an invented UBERON:0002108 is a silent error that survives review because it looks exactly like a real ID.

If a concept genuinely has no term and the project depends on it, the route is a new-term request to the ontology (GitHub issue on the ontology's tracker, with a definition and a reference), not a locally minted identifier.

Raw lines 51 to 60
61## Auditing an existing metadata table
63The high-yield checks, in order:
651. **Every ID exists.** `validate_terms.py --input table.tsv`.
662. **No obsolete IDs.** Obsolete terms carry `term_replaced_by` often enough that the fix is
67 straightforward — but apply replacements deliberately, since a replacement can be broader or
68 narrower than the original.
693. **Labels match IDs.** Supply the label column. Mismatches are where copy-paste drift and
70 hallucinated IDs surface: the ID is real, the label is real, and they describe different things.
714. **Right ontology per column.** `--expect-ontology`.
725. **Right branch per column.** `--branch`, remembering it does not exclude cell types from
73 anatomy (`ontology-registry.md`).
75`--strict` turns warnings into failures, which is the right setting for a CI gate. Warnings are
76`matched_synonym` (label is a synonym rather than the primary label), `imported_only` (no defining copy verified by this lookup), and `not_a_class`.

Auditing an existing metadata table

The high-yield checks, in order:

  1. Every ID exists. validate_terms.py --input table.tsv.
  2. No obsolete IDs. Obsolete terms carry term_replaced_by often enough that the fix is straightforward — but apply replacements deliberately, since a replacement can be broader or narrower than the original.
  3. Labels match IDs. Supply the label column. Mismatches are where copy-paste drift and hallucinated IDs surface: the ID is real, the label is real, and they describe different things.
  4. Right ontology per column. --expect-ontology.
  5. Right branch per column. --branch, remembering it does not exclude cell types from anatomy (ontology-registry.md).

--strict turns warnings into failures, which is the right setting for a CI gate. Warnings are matched_synonym (label is a synonym rather than the primary label), imported_only (no defining copy verified by this lookup), and not_a_class.

Raw lines 61 to 77
78## Obsolete terms
80Obsoletion is not deletion — the ID keeps resolving, and its label is usually prefixed
81`obsolete_`. That prefix is a useful smell in any metadata file:
83```
84EFO:0001067 obsolete_parasitic infection -> replaced by MONDO:0005135
85```
87Some obsolete terms have no replacement, only a `consider` annotation or nothing at all. Then the
88term must be re-curated by hand; there is no automatic answer.

Obsolete terms

Obsoletion is not deletion — the ID keeps resolving, and its label is usually prefixed obsolete_. That prefix is a useful smell in any metadata file:

EFO:0001067  obsolete_parasitic infection  ->  replaced by MONDO:0005135

Some obsolete terms have no replacement, only a consider annotation or nothing at all. Then the term must be re-curated by hand; there is no automatic answer.

Raw lines 78 to 89
90## Cross-ontology mapping
92[OxO2](https://github.com/EBISPOT/oxo2) is live with compatibility APIs. The bundled scripts
93do not perform cross-ontology mapping; two source-grounded routes are:
95- **Term cross-references.** `term_detail(curie)["annotation"]["database_cross_reference"]` lists
96 cross-reference candidates — `UBERON:0002107` carries `MESH:D008099`, `NCIT:C12392`, `FMA:7197`, `UMLS:C0023884`,
97 and more.
98- **SSSOM mapping sets** published by Monarch and the OBO community, when provenance and mapping
99 predicates (`skos:exactMatch` vs `closeMatch`) matter.
101Cross-references are asserted by curators at varying confidence and are not all `exactMatch`.
102Treat a single xref as a lead, not a proof, when the mapping drives analysis rather than display.
104```python
105from ols_client import term_detail
106xrefs = (term_detail("UBERON:0002107") or {}).get("annotation", {}).get(
107 "database_cross_reference", []
109```

Cross-ontology mapping

OxO2 is live with compatibility APIs. The bundled scripts do not perform cross-ontology mapping; two source-grounded routes are:

  • Term cross-references. term_detail(curie)["annotation"]["database_cross_reference"] lists cross-reference candidates — UBERON:0002107 carries MESH:D008099, NCIT:C12392, FMA:7197, UMLS:C0023884, and more.
  • SSSOM mapping sets published by Monarch and the OBO community, when provenance and mapping predicates (skos:exactMatch vs closeMatch) matter.

Cross-references are asserted by curators at varying confidence and are not all exactMatch. Treat a single xref as a lead, not a proof, when the mapping drives analysis rather than display.

from ols_client import term_detail
xrefs = (term_detail("UBERON:0002107") or {}).get("annotation", {}).get(
    "database_cross_reference", []
)
Raw lines 90 to 110
111## Reporting
113When you hand back resolved terms, give the ID *and* the label, and say how each was matched. A
114table of bare IDs cannot be reviewed — no reader can tell `UBERON:0002107` from `UBERON:0002108`
115by eye, which is precisely why invented IDs survive review.

Reporting

When you hand back resolved terms, give the ID and the label, and say how each was matched. A table of bare IDs cannot be reviewed — no reader can tell UBERON:0002107 from UBERON:0002108 by eye, which is precisely why invented IDs survive review.

Raw lines 111 to 115