Skill Tracer
Skills › Scientific Agent Skills › torchdrug › datasets.md

datasets.md

248 lines · skills/torchdrug/references/datasets.md at 5f2bc03

Open on GitHub
1# Datasets
2
3Use the
4[TorchDrug 0.2.1 dataset reference](https://torchdrug.ai/docs/api/datasets.html)
5as the class inventory and signature source. Dataset constructors download and
6cache data under the path supplied by the caller.
7
8These are release-era loaders, not live database clients. There is no API token,
9query pagination, or current-release discovery. Constructors can fetch large
10archives as a side effect. Check the selected loader's source URL, expected
11checksum, size, and dataset license before starting a download.

Datasets

Use the TorchDrug 0.2.1 dataset reference as the class inventory and signature source. Dataset constructors download and cache data under the path supplied by the caller.

These are release-era loaders, not live database clients. There is no API token, query pagination, or current-release discovery. Constructors can fetch large archives as a side effect. Check the selected loader's source URL, expected checksum, size, and dataset license before starting a download.

Raw lines 1 to 12
13## ClinTox download repair
15The 0.2.1 class embeds a legacy DeepChem HTTP URL that returned HTTP 400 during
16this review. DeepChem's current official loader uses the HTTPS URL below. Its
1719,870-byte gzip was fetched and matches TorchDrug's MD5 exactly; it has 1,484
18raw rows before molecular sanitization/filtering. Populate the cache once before
19running any ClinTox example in this skill:
21```python
22from pathlib import Path
23from torchdrug import datasets, utils
25cache = Path("~/molecule-datasets/").expanduser()
26cache.mkdir(parents=True, exist_ok=True)
27archive = utils.download(
28 "https://deepchemdata.s3-us-west-1.amazonaws.com/datasets/clintox.csv.gz",
29 str(cache),
30 md5=datasets.ClinTox.md5,
32if utils.compute_md5(archive) != datasets.ClinTox.md5:
33 raise RuntimeError("ClinTox checksum mismatch; do not train on this cache")
34```
36`utils.download(..., md5=...)` checks an existing cache before deciding whether
37to fetch. In this release it does **not** verify freshly downloaded bytes, hence
38the explicit post-download check. MD5 here identifies the historical benchmark
39asset; record a SHA-256 for project provenance as well.
41The old URLs for BACE, BBBP, HIV, MUV, SIDER, Tox21, ToxCast, QM8, QM9, and
42Lipophilicity also failed during endpoint probes. Do not blindly replace a host
43and assume equivalent contents: verify each current official loader and checksum
44or load a separately versioned local dataset with `data.MoleculeDataset.load_csv`.
45FreeSolv's old S3 route returned an unresolved HTTP 301. These are observed
46download failures, not evidence that the datasets themselves have been retired.

ClinTox download repair

The 0.2.1 class embeds a legacy DeepChem HTTP URL that returned HTTP 400 during this review. DeepChem's current official loader uses the HTTPS URL below. Its 19,870-byte gzip was fetched and matches TorchDrug's MD5 exactly; it has 1,484 raw rows before molecular sanitization/filtering. Populate the cache once before running any ClinTox example in this skill:

from pathlib import Path
from torchdrug import datasets, utils

cache = Path("~/molecule-datasets/").expanduser()
cache.mkdir(parents=True, exist_ok=True)
archive = utils.download(
    "https://deepchemdata.s3-us-west-1.amazonaws.com/datasets/clintox.csv.gz",
    str(cache),
    md5=datasets.ClinTox.md5,
)
if utils.compute_md5(archive) != datasets.ClinTox.md5:
    raise RuntimeError("ClinTox checksum mismatch; do not train on this cache")

utils.download(..., md5=...) checks an existing cache before deciding whether to fetch. In this release it does not verify freshly downloaded bytes, hence the explicit post-download check. MD5 here identifies the historical benchmark asset; record a SHA-256 for project provenance as well.

The old URLs for BACE, BBBP, HIV, MUV, SIDER, Tox21, ToxCast, QM8, QM9, and Lipophilicity also failed during endpoint probes. Do not blindly replace a host and assume equivalent contents: verify each current official loader and checksum or load a separately versioned local dataset with data.MoleculeDataset.load_csv. FreeSolv's old S3 route returned an unresolved HTTP 301. These are observed download failures, not evidence that the datasets themselves have been retired.

Raw lines 13 to 47
48## Dataset families

Dataset families

Raw lines 48 to 49
50### Molecule property prediction
52Documented classes include:
54- Classification: `BACE`, `BBBP`, `ClinTox`, `HIV`, `MUV`, `SIDER`, `Tox21`,
55 `ToxCast`
56- Regression / quantum properties: `FreeSolv`, `Lipophilicity`, `QM8`, `QM9`,
57 `PCQM4M`
58- Pretraining / generation: `ChEMBLFiltered`, `ZINC250k`, `ZINC2m`, `MOSES`
60The official property tutorial uses `ClinTox`; the pretraining tutorial uses
61`ClinTox` for a small demonstration and recommends larger data such as `ZINC2m`
62for real pretraining; the generation tutorial uses `ZINC250k`.
64```python
65from torchdrug import datasets
67dataset = datasets.ClinTox(
68 "~/molecule-datasets/",
69 atom_feature="default",
70 bond_feature="default",
72print(dataset.tasks)
73print(dataset.node_feature_dim)
74print(dataset.edge_feature_dim)
75```
77Common molecule options include `atom_feature`, `bond_feature`, `mol_feature`,
78`with_hydrogen`, and `kekulize`. Availability varies by class; inspect the class
79signature before adding options.

Molecule property prediction

Documented classes include:

  • Classification: BACE, BBBP, ClinTox, HIV, MUV, SIDER, Tox21, ToxCast
  • Regression / quantum properties: FreeSolv, Lipophilicity, QM8, QM9, PCQM4M
  • Pretraining / generation: ChEMBLFiltered, ZINC250k, ZINC2m, MOSES

The official property tutorial uses ClinTox; the pretraining tutorial uses ClinTox for a small demonstration and recommends larger data such as ZINC2m for real pretraining; the generation tutorial uses ZINC250k.

from torchdrug import datasets

dataset = datasets.ClinTox(
    "~/molecule-datasets/",
    atom_feature="default",
    bond_feature="default",
)
print(dataset.tasks)
print(dataset.node_feature_dim)
print(dataset.edge_feature_dim)

Common molecule options include atom_feature, bond_feature, mol_feature, with_hydrogen, and kekulize. Availability varies by class; inspect the class signature before adding options.

Raw lines 50 to 80
81### Protein properties and structure
83Documented families include:
85- Sequence / property: `BetaLactamase`, `BinaryLocalization`,
86 `SubcellularLocalization`
87- Structure / function: `EnzymeCommission`, `GeneOntology`, `AlphaFoldDB`
88- Structure labels: `Fold`, `SecondaryStructure`
89- Protein-protein: `HumanPPI`, `YeastPPI`, `PPIAffinity`
90- Protein-ligand: `BindingDB`, `PDBBind`
92```python
93dataset = datasets.EnzymeCommission(
94 "~/protein-datasets/",
95 atom_feature=None,
96 bond_feature=None,
97 residue_feature="default",
99train_set, valid_set, test_set = dataset.split()
100```
102Protein datasets can be expensive to parse. Where supported, `lazy=True` trades
103lower startup memory for slower item loading. For sequence-only models, omitting
104atom and bond features avoids unnecessary atom-level construction.

Protein properties and structure

Documented families include:

  • Sequence / property: BetaLactamase, BinaryLocalization, SubcellularLocalization
  • Structure / function: EnzymeCommission, GeneOntology, AlphaFoldDB
  • Structure labels: Fold, SecondaryStructure
  • Protein-protein: HumanPPI, YeastPPI, PPIAffinity
  • Protein-ligand: BindingDB, PDBBind
dataset = datasets.EnzymeCommission(
    "~/protein-datasets/",
    atom_feature=None,
    bond_feature=None,
    residue_feature="default",
)
train_set, valid_set, test_set = dataset.split()

Protein datasets can be expensive to parse. Where supported, lazy=True trades lower startup memory for slower item loading. For sequence-only models, omitting atom and bond features avoids unnecessary atom-level construction.

Raw lines 81 to 105
106### Knowledge graphs
108Documented classes:
110- `FB15k`
111- `FB15k237`
112- `WN18`
113- `WN18RR`
114- `Hetionet`
116```python
117dataset = datasets.FB15k237("~/kg-datasets/")
118train_set, valid_set, test_set = dataset.split()
120print(dataset.num_entity)
121print(dataset.num_relation)
122```
124These datasets provide predefined benchmark splits. Preserve those splits for
125comparable evaluation.

Knowledge graphs

Documented classes:

  • FB15k
  • FB15k237
  • WN18
  • WN18RR
  • Hetionet
dataset = datasets.FB15k237("~/kg-datasets/")
train_set, valid_set, test_set = dataset.split()

print(dataset.num_entity)
print(dataset.num_relation)

These datasets provide predefined benchmark splits. Preserve those splits for comparable evaluation.

Raw lines 106 to 126
127### Retrosynthesis
129`USPTO50k` contains 50,017 reactions across 10 reaction classes. The official
130G2Gs workflow loads two views:
132```python
133reaction_dataset = datasets.USPTO50k(
134 "~/molecule-datasets/",
135 atom_feature="center_identification",
136 kekulize=True,
138synthon_dataset = datasets.USPTO50k(
139 "~/molecule-datasets/",
140 as_synthon=True,
141 atom_feature="synthon_completion",
142 kekulize=True,
144```
146Reaction mode yields reactant/product pairs for center identification. Synthon
147mode yields reactant/synthon pairs for synthon completion.

Retrosynthesis

USPTO50k contains 50,017 reactions across 10 reaction classes. The official G2Gs workflow loads two views:

reaction_dataset = datasets.USPTO50k(
    "~/molecule-datasets/",
    atom_feature="center_identification",
    kekulize=True,
)
synthon_dataset = datasets.USPTO50k(
    "~/molecule-datasets/",
    as_synthon=True,
    atom_feature="synthon_completion",
    kekulize=True,
)

Reaction mode yields reactant/product pairs for center identification. Synthon mode yields reactant/synthon pairs for synthon completion.

Raw lines 127 to 148
149## Splitting correctly
151Some benchmark datasets expose predefined splits:
153```python
154train_set, valid_set, test_set = dataset.split()
155```
157For the property-prediction tutorial's random 80/10/10 split, use PyTorch:
159```python
160import torch
162lengths = [int(0.8 * len(dataset)), int(0.1 * len(dataset))]
163lengths.append(len(dataset) - sum(lengths))
164train_set, valid_set, test_set = torch.utils.data.random_split(dataset, lengths)
165```
167Do not assume `dataset.split([0.8, 0.1, 0.1])` is a documented universal API.
169For paired retrosynthesis views, reset the same seed before each `split()`:
171```python
172torch.manual_seed(1)
173reaction_train, reaction_valid, reaction_test = reaction_dataset.split()
174torch.manual_seed(1)
175synthon_train, synthon_valid, synthon_test = synthon_dataset.split()
176```
178The two views can have different row counts because one reaction yields several
179synthons. This is source-reaction partition alignment, not row-by-row alignment.
180Verify it explicitly after preprocessing/filtering:
182```python
183def source_ids(dataset, subset):
184 return {dataset.targets["sample id"][i] for i in subset.indices}
186reaction_parts = (reaction_train, reaction_valid, reaction_test)
187synthon_parts = (synthon_train, synthon_valid, synthon_test)
188ids = [source_ids(reaction_dataset, part) for part in reaction_parts]
189for expected, part in zip(ids, synthon_parts):
190 if expected != source_ids(synthon_dataset, part):
191 raise ValueError("Reaction/synthon source IDs differ; split by common IDs")
192if any(ids[i] & ids[j] for i in range(3) for j in range(i + 1, 3)):
193 raise ValueError("Reaction source IDs leak across partitions")
194```

Splitting correctly

Some benchmark datasets expose predefined splits:

train_set, valid_set, test_set = dataset.split()

For the property-prediction tutorial's random 80/10/10 split, use PyTorch:

import torch

lengths = [int(0.8 * len(dataset)), int(0.1 * len(dataset))]
lengths.append(len(dataset) - sum(lengths))
train_set, valid_set, test_set = torch.utils.data.random_split(dataset, lengths)

Do not assume dataset.split([0.8, 0.1, 0.1]) is a documented universal API.

For paired retrosynthesis views, reset the same seed before each split():

torch.manual_seed(1)
reaction_train, reaction_valid, reaction_test = reaction_dataset.split()
torch.manual_seed(1)
synthon_train, synthon_valid, synthon_test = synthon_dataset.split()

The two views can have different row counts because one reaction yields several synthons. This is source-reaction partition alignment, not row-by-row alignment. Verify it explicitly after preprocessing/filtering:

def source_ids(dataset, subset):
    return {dataset.targets["sample id"][i] for i in subset.indices}

reaction_parts = (reaction_train, reaction_valid, reaction_test)
synthon_parts = (synthon_train, synthon_valid, synthon_test)
ids = [source_ids(reaction_dataset, part) for part in reaction_parts]
for expected, part in zip(ids, synthon_parts):
    if expected != source_ids(synthon_dataset, part):
        raise ValueError("Reaction/synthon source IDs differ; split by common IDs")
if any(ids[i] & ids[j] for i in range(3) for j in range(i + 1, 3)):
    raise ValueError("Reaction source IDs leak across partitions")
Raw lines 149 to 195
196## Feature configuration
198Dataset dimensions depend on feature choices. Construct models from the loaded
199dataset rather than hard-coding dimensions:
201```python
202from torchdrug import models
204model = models.GIN(
205 input_dim=dataset.node_feature_dim,
206 hidden_dims=[256, 256, 256],
207 edge_input_dim=dataset.edge_feature_dim,
209```
211Generation and retrosynthesis often require specialized feature sets:
213- Pretraining: `atom_feature="pretrain"`, `bond_feature="pretrain"`
214- GCPN / GraphAF: `atom_feature="symbol"`, `kekulize=True`
215- Center identification: `atom_feature="center_identification"`
216- Synthon completion: `atom_feature="synthon_completion"`
218Do not mix checkpoint weights across incompatible feature configurations.

Feature configuration

Dataset dimensions depend on feature choices. Construct models from the loaded dataset rather than hard-coding dimensions:

from torchdrug import models

model = models.GIN(
    input_dim=dataset.node_feature_dim,
    hidden_dims=[256, 256, 256],
    edge_input_dim=dataset.edge_feature_dim,
)

Generation and retrosynthesis often require specialized feature sets:

  • Pretraining: atom_feature="pretrain", bond_feature="pretrain"
  • GCPN / GraphAF: atom_feature="symbol", kekulize=True
  • Center identification: atom_feature="center_identification"
  • Synthon completion: atom_feature="synthon_completion"

Do not mix checkpoint weights across incompatible feature configurations.

Raw lines 196 to 219
220## Data integrity and evaluation
222- Cache datasets in a controlled project or user data directory.
223- Record TorchDrug version, feature arguments, split method, and random seed.
224- Preserve predefined KG splits.
225- For molecular benchmarks, use the split protocol required by the benchmark;
226 do not claim a random split is a scaffold split.
227- Inspect downloaded data licenses and provenance before redistribution.
228- Validate labels, missing-value masks, and task names before training.
229- `EnzymeCommission` and `GeneOntology` return `sample["targets"]` vectors;
230 their `dataset.tasks` names are not separate sample dictionary keys. Use
231 `MultipleBinaryClassification(task=list(range(len(dataset.tasks))))`.
232- `AlphaFoldDB` in this release is pinned to historical **v2** proteome archives,
233 not the current AlphaFold DB. The embedded zebrafish URL returned 404; do not
234 silently relabel a newer archive as v2.
235- HEAD probes reached ZINC250k, USPTO50k, all FB15k237 split files, and the
236 EnzymeCommission archive (about 1 GB), but do not validate bytes or parsing.
237 None of those large datasets or pretrained weights were downloaded in this audit.

Data integrity and evaluation

  • Cache datasets in a controlled project or user data directory.
  • Record TorchDrug version, feature arguments, split method, and random seed.
  • Preserve predefined KG splits.
  • For molecular benchmarks, use the split protocol required by the benchmark; do not claim a random split is a scaffold split.
  • Inspect downloaded data licenses and provenance before redistribution.
  • Validate labels, missing-value masks, and task names before training.
  • EnzymeCommission and GeneOntology return sample["targets"] vectors; their dataset.tasks names are not separate sample dictionary keys. Use MultipleBinaryClassification(task=list(range(len(dataset.tasks)))).
  • AlphaFoldDB in this release is pinned to historical v2 proteome archives, not the current AlphaFold DB. The embedded zebrafish URL returned 404; do not silently relabel a newer archive as v2.
  • HEAD probes reached ZINC250k, USPTO50k, all FB15k237 split files, and the EnzymeCommission archive (about 1 GB), but do not validate bytes or parsing. None of those large datasets or pretrained weights were downloaded in this audit.
Raw lines 220 to 238
239## Source links
241- [Dataset API](https://torchdrug.ai/docs/api/datasets.html)
242- [Property prediction tutorial](https://torchdrug.ai/docs/tutorials/property_prediction.html)
243- [Pretraining tutorial](https://torchdrug.ai/docs/tutorials/pretrain.html)
244- [Generation tutorial](https://torchdrug.ai/docs/tutorials/generation.html)
245- [Retrosynthesis tutorial](https://torchdrug.ai/docs/tutorials/retrosynthesis.html)
246- [Knowledge graph tutorial](https://torchdrug.ai/docs/tutorials/reasoning.html)
247- [Current DeepChem ClinTox loader](https://github.com/deepchem/deepchem/blob/master/deepchem/molnet/load_function/clintox_datasets.py)
248- [Released dataset source](https://github.com/DeepGraphLearning/torchdrug/tree/v0.2.1/torchdrug/datasets)