Skill Tracer
Skills › Scientific Agent Skills › datamol › core_workflows.md

core_workflows.md

196 lines · skills/datamol/references/core_workflows.md at 5f2bc03

Open on GitHub
1# Datamol core workflows (0.13.0)
2
3These local snippets were executed with Python 3.13 / RDKit 2026.03.6. Run the setup
4first. Each step preserves the relationship between input structures and outputs.
5Use [the pipeline patterns](workflow_patterns.md) for rejected-row and selection provenance.
6

Datamol core workflows (0.13.0)

These local snippets were executed with Python 3.13 / RDKit 2026.03.6. Run the setup first. Each step preserves the relationship between input structures and outputs. Use the pipeline patterns for rejected-row and selection provenance.

Raw lines 1 to 6
7## 1. Molecules and explicit structure policy
8
9```python
10import datamol as dm
11import numpy as np
12import pandas as pd
13from pathlib import Path
14from tempfile import TemporaryDirectory
16smiles_list = ["CCO", "CCCO", "c1ccccc1O", "c1ccccc1CCN", "c1ccncc1", "CC(=O)O"]
17mols = [dm.to_mol(s) for s in smiles_list]
18assert all(m is not None for m in mols)
19with dm.without_rdkit_log():
20 invalid = dm.to_mol("invalid_smiles")
21assert invalid is None
22mol = mols[0] # keep a valid molecule for subsequent examples
23smiles = dm.to_smiles(mol, canonical=True, isomeric=True)
24inchi, inchikey, selfies = dm.to_inchi(mol), dm.to_inchikey(mol), dm.to_selfies(mol)
25assert dm.from_selfies(selfies, as_mol=True) is not None
26# Illustrative, deliberately specified policy for these ordinary organic molecules.
27standardized = dm.standardize_mol(mol, disconnect_metals=False, normalize=True,
28 reionize=True, uncharge=False, stereo=True)
29assert standardized is not None
30```
32Choose the policy from the assayed entity before standardizing. Preserve source structures;
33metal disconnection, salt/fragment selection and neutralization are not universally appropriate.
34Sanitization/fixing is not evidence that an invalid source structure has been recovered.

1. Molecules and explicit structure policy

import datamol as dm
import numpy as np
import pandas as pd
from pathlib import Path
from tempfile import TemporaryDirectory

smiles_list = ["CCO", "CCCO", "c1ccccc1O", "c1ccccc1CCN", "c1ccncc1", "CC(=O)O"]
mols = [dm.to_mol(s) for s in smiles_list]
assert all(m is not None for m in mols)
with dm.without_rdkit_log():
    invalid = dm.to_mol("invalid_smiles")
assert invalid is None
mol = mols[0]  # keep a valid molecule for subsequent examples
smiles = dm.to_smiles(mol, canonical=True, isomeric=True)
inchi, inchikey, selfies = dm.to_inchi(mol), dm.to_inchikey(mol), dm.to_selfies(mol)
assert dm.from_selfies(selfies, as_mol=True) is not None
# Illustrative, deliberately specified policy for these ordinary organic molecules.
standardized = dm.standardize_mol(mol, disconnect_metals=False, normalize=True,
                                  reionize=True, uncharge=False, stereo=True)
assert standardized is not None

Choose the policy from the assayed entity before standardizing. Preserve source structures; metal disconnection, salt/fragment selection and neutralization are not universally appropriate. Sanitization/fixing is not evidence that an invalid source structure has been recovered.

Raw lines 7 to 35
36## 2. Local I/O and portable tables
38```python
39with TemporaryDirectory() as directory:
40 base = Path(directory)
41 dm.to_sdf(mols, base / "compounds.sdf")
42 df = dm.read_sdf(base / "compounds.sdf", as_df=True, mol_column="mol",
43 discard_invalid=False)
44 assert len(df) == len(mols)
45 dm.to_smi(mols, base / "molecules.smi", error_if_empty=True)
46 assert len(dm.read_smi(base / "molecules.smi")) == len(mols)
47 dm.to_xlsx(df, base / "compounds.xlsx", mol_column="mol")
48 portable = df.drop(columns="mol")
49 dm.save_df(portable, str(base / "compounds.parquet"))
50 assert len(dm.open_df(str(base / "compounds.parquet"))) == len(mols)
51```
53`read_sdf` returns a list unless `as_df=True`; `open_df` selects DataFrame mode for SDF.
54Do not serialize Python Mol objects into Parquet/CSV. Use SMILES plus scalar properties
55and reconstruct on read. See [I/O contracts](io_module.md) for rejection handling,
56file-format parameters, and illustrative cloud paths. Remote credentials/writes were not tested.

2. Local I/O and portable tables

with TemporaryDirectory() as directory:
    base = Path(directory)
    dm.to_sdf(mols, base / "compounds.sdf")
    df = dm.read_sdf(base / "compounds.sdf", as_df=True, mol_column="mol",
                     discard_invalid=False)
    assert len(df) == len(mols)
    dm.to_smi(mols, base / "molecules.smi", error_if_empty=True)
    assert len(dm.read_smi(base / "molecules.smi")) == len(mols)
    dm.to_xlsx(df, base / "compounds.xlsx", mol_column="mol")
    portable = df.drop(columns="mol")
    dm.save_df(portable, str(base / "compounds.parquet"))
    assert len(dm.open_df(str(base / "compounds.parquet"))) == len(mols)

read_sdf returns a list unless as_df=True; open_df selects DataFrame mode for SDF. Do not serialize Python Mol objects into Parquet/CSV. Use SMILES plus scalar properties and reconstruct on read. See I/O contracts for rejection handling, file-format parameters, and illustrative cloud paths. Remote credentials/writes were not tested.

Raw lines 36 to 57
58## 3. Descriptors with an explicit feature schema
60```python
61properties = {"mw": "MolWt", "logp": "MolLogP", "hbd": "NumHDonors",
62 "hba": "NumHAcceptors", "tpsa": "TPSA"}
63desc_df = dm.descriptors.batch_compute_many_descriptors(
64 mols, properties_fn=properties, add_properties=False, n_jobs=1,
66passes_screen = ((desc_df["mw"] <= 500) & (desc_df["logp"] <= 5)
67 & (desc_df["hbd"] <= 5) & (desc_df["hba"] <= 10))
68selected = [m for m, keep in zip(mols, passes_screen) if keep]
69assert len(desc_df) == len(mols)
70n_aromatic = dm.descriptors.n_aromatic_atoms(mol)
71n_stereo = dm.descriptors.n_stereo_centers(mol)
72n_unspecified = dm.descriptors.n_stereo_centers_unspecified(mol)
73n_rigid = dm.descriptors.n_rigid_bonds(mol)
74```
76This defines average MolWt and RDKit donor/acceptor counts. Datamol defaults instead use
77exact mass (`mw`), `clogp`, `n_lipinski_hbd`, and `n_lipinski_hba`. See
78[descriptor semantics](descriptors_viz.md). Screen passage is not validated drug-likeness.

3. Descriptors with an explicit feature schema

properties = {"mw": "MolWt", "logp": "MolLogP", "hbd": "NumHDonors",
              "hba": "NumHAcceptors", "tpsa": "TPSA"}
desc_df = dm.descriptors.batch_compute_many_descriptors(
    mols, properties_fn=properties, add_properties=False, n_jobs=1,
)
passes_screen = ((desc_df["mw"] <= 500) & (desc_df["logp"] <= 5)
                 & (desc_df["hbd"] <= 5) & (desc_df["hba"] <= 10))
selected = [m for m, keep in zip(mols, passes_screen) if keep]
assert len(desc_df) == len(mols)
n_aromatic = dm.descriptors.n_aromatic_atoms(mol)
n_stereo = dm.descriptors.n_stereo_centers(mol)
n_unspecified = dm.descriptors.n_stereo_centers_unspecified(mol)
n_rigid = dm.descriptors.n_rigid_bonds(mol)

This defines average MolWt and RDKit donor/acceptor counts. Datamol defaults instead use exact mass (mw), clogp, n_lipinski_hbd, and n_lipinski_hba. See descriptor semantics. Screen passage is not validated drug-likeness.

Raw lines 58 to 79
80## 4. Fingerprints and similarities
82```python
83fp_options = dict(fp_type="ecfp", radius=2, fpSize=2048, includeChirality=True)
84fps = np.stack([dm.to_fp(m, **fp_options) for m in mols])
85distance_matrix = dm.pdist(mols, n_jobs=1, **fp_options) # square by default
86query_distances = dm.cdist(mols[:1], mols, n_jobs=1, **fp_options)
87nearest = np.argsort(query_distances[0])
88assert nearest[0] == 0 and np.isclose(query_distances[0, 0], 0)
89for fp_type in ["maccs", "topological", "atompair", "rdkit"]:
90 assert dm.to_fp(mol, fp_type=fp_type).ndim == 1
91```
93ECFP radius 2 is ECFP4; Datamol defaults to radius 3. Use `fpSize`, not `n_bits`.
94`pdist(..., squareform=False)` returns condensed distances; calling SciPy `squareform`
95on the default square output would convert it into a vector. Binary Tanimoto similarity
96is 1 - distance. Compare libraries only with the same fingerprint configuration.

4. Fingerprints and similarities

fp_options = dict(fp_type="ecfp", radius=2, fpSize=2048, includeChirality=True)
fps = np.stack([dm.to_fp(m, **fp_options) for m in mols])
distance_matrix = dm.pdist(mols, n_jobs=1, **fp_options)  # square by default
query_distances = dm.cdist(mols[:1], mols, n_jobs=1, **fp_options)
nearest = np.argsort(query_distances[0])
assert nearest[0] == 0 and np.isclose(query_distances[0, 0], 0)
for fp_type in ["maccs", "topological", "atompair", "rdkit"]:
    assert dm.to_fp(mol, fp_type=fp_type).ndim == 1

ECFP radius 2 is ECFP4; Datamol defaults to radius 3. Use fpSize, not n_bits. pdist(..., squareform=False) returns condensed distances; calling SciPy squareform on the default square output would convert it into a vector. Binary Tanimoto similarity is 1 - distance. Compare libraries only with the same fingerprint configuration.

Raw lines 80 to 97
98## 5. Clustering and diversity
100```python
101from functools import partial
102feature_fn = partial(dm.to_fp, as_array=False, **fp_options)
103cluster_indices, cluster_molecules = dm.cluster_mols(mols, cutoff=0.2, feature_fn=feature_fn)
104for indices, molecules in zip(cluster_indices, cluster_molecules):
105 assert len(indices) == len(molecules)
106picked_indices, diverse_mols = dm.pick_diverse(mols, npick=min(3, len(mols)),
107 feature_fn=feature_fn, seed=42)
108centroid_indices, centroid_mols = dm.pick_centroids(mols, npick=3, feature_fn=feature_fn)
109```
111Both selectors return `(indices, molecules)`; preserve indices for labels/IDs. Butina needs
112quadratic distance storage. Select a resource budget, not a universal molecule-count cutoff.
113The sphere centroid selector may return fewer than requested due to its distance threshold.

5. Clustering and diversity

from functools import partial
feature_fn = partial(dm.to_fp, as_array=False, **fp_options)
cluster_indices, cluster_molecules = dm.cluster_mols(mols, cutoff=0.2, feature_fn=feature_fn)
for indices, molecules in zip(cluster_indices, cluster_molecules):
    assert len(indices) == len(molecules)
picked_indices, diverse_mols = dm.pick_diverse(mols, npick=min(3, len(mols)),
                                              feature_fn=feature_fn, seed=42)
centroid_indices, centroid_mols = dm.pick_centroids(mols, npick=3, feature_fn=feature_fn)

Both selectors return (indices, molecules); preserve indices for labels/IDs. Butina needs quadratic distance storage. Select a resource budget, not a universal molecule-count cutoff. The sphere centroid selector may return fewer than requested due to its distance threshold.

Raw lines 98 to 114
115## 6. Scaffolds and a toy scaffold-disjoint split
117```python
118from collections import defaultdict
119import random
120scaffold_groups = defaultdict(list)
121for index, molecule in enumerate(mols):
122 key = dm.to_smiles(dm.to_scaffold_murcko(molecule))
123 scaffold_groups[key].append(index)
124keys = sorted(scaffold_groups)
125random.Random(42).shuffle(keys)
126split_index = int(0.8 * len(keys))
127train_keys, test_keys = keys[:split_index], keys[split_index:]
128train_indices = [i for key in train_keys for i in scaffold_groups[key]]
129test_indices = [i for key in test_keys for i in scaffold_groups[key]]
130assert set(train_keys).isdisjoint(test_keys)
131assert set(train_indices).isdisjoint(test_indices)
132```
134This divides scaffold groups, not exactly 80% of molecules. All acyclic molecules share an
135empty Murcko scaffold here; report that policy and inspect resulting class/size balance.
136Validate chemical duplicate/analog and task-specific leakage separately.

6. Scaffolds and a toy scaffold-disjoint split

from collections import defaultdict
import random
scaffold_groups = defaultdict(list)
for index, molecule in enumerate(mols):
    key = dm.to_smiles(dm.to_scaffold_murcko(molecule))
    scaffold_groups[key].append(index)
keys = sorted(scaffold_groups)
random.Random(42).shuffle(keys)
split_index = int(0.8 * len(keys))
train_keys, test_keys = keys[:split_index], keys[split_index:]
train_indices = [i for key in train_keys for i in scaffold_groups[key]]
test_indices = [i for key in test_keys for i in scaffold_groups[key]]
assert set(train_keys).isdisjoint(test_keys)
assert set(train_indices).isdisjoint(test_indices)

This divides scaffold groups, not exactly 80% of molecules. All acyclic molecules share an empty Murcko scaffold here; report that policy and inspect resulting class/size balance. Validate chemical duplicate/analog and task-specific leakage separately.

Raw lines 115 to 137
138## 7. Fragmentation
140```python
141from collections import Counter
142fragment_counts = Counter()
143for molecule in mols:
144 fragments = dm.fragment.brics(molecule, remove_parent=True, fix=False)
145 fragment_counts.update({dm.to_smiles(f) for f in fragments})
146recap_fragments = dm.fragment.recap(mol, remove_parent=True, fix=False)
147```
149These functions return lists of Mol. Canonicalize to strings before set overlap/counting;
150otherwise Python object identity can silently corrupt counts. This counts compound prevalence,
151not repeated fragment sites. See [fragment contracts](fragments_scaffolds.md).

7. Fragmentation

from collections import Counter
fragment_counts = Counter()
for molecule in mols:
    fragments = dm.fragment.brics(molecule, remove_parent=True, fix=False)
    fragment_counts.update({dm.to_smiles(f) for f in fragments})
recap_fragments = dm.fragment.recap(mol, remove_parent=True, fix=False)

These functions return lists of Mol. Canonicalize to strings before set overlap/counting; otherwise Python object identity can silently corrupt counts. This counts compound prevalence, not repeated fragment sites. See fragment contracts.

Raw lines 138 to 152
153## 8. Conformers
155```python
156mol_3d = dm.conformers.generate(dm.to_mol("CC(C)CCO"), n_confs=10, rms_cutoff=0.5,
157 minimize_energy=True, forcefield="UFF", random_seed=19)
158assert mol_3d.GetNumConformers() > 0
159coords = dm.conformers.get_coords(mol_3d)
160cluster_molecules_3d = dm.conformers.cluster(mol_3d, rms_cutoff=1.0, centroids=False)
161centroid_molecule_3d = dm.conformers.cluster(mol_3d, rms_cutoff=1.0, centroids=True)
162# SASA is supported on this tested macOS stack; Datamol disables it on Windows.
163sasa_values = dm.conformers.sasa(mol_3d, n_jobs=1)
164```
166Generation defaults to no minimization; this example enables it explicitly. Conformer
167clustering returns Mol objects, not index tuples. See [conformer contracts](conformers_module.md)
168for symmetry-aware RMS, finite-energy versus convergence checks, units and property storage.

8. Conformers

mol_3d = dm.conformers.generate(dm.to_mol("CC(C)CCO"), n_confs=10, rms_cutoff=0.5,
                                minimize_energy=True, forcefield="UFF", random_seed=19)
assert mol_3d.GetNumConformers() > 0
coords = dm.conformers.get_coords(mol_3d)
cluster_molecules_3d = dm.conformers.cluster(mol_3d, rms_cutoff=1.0, centroids=False)
centroid_molecule_3d = dm.conformers.cluster(mol_3d, rms_cutoff=1.0, centroids=True)
# SASA is supported on this tested macOS stack; Datamol disables it on Windows.
sasa_values = dm.conformers.sasa(mol_3d, n_jobs=1)

Generation defaults to no minimization; this example enables it explicitly. Conformer clustering returns Mol objects, not index tuples. See conformer contracts for symmetry-aware RMS, finite-energy versus convergence checks, units and property storage.

Raw lines 153 to 169
170## 9. Visualize
172```python
173with TemporaryDirectory() as directory:
174 dm.viz.to_image(diverse_mols, legends=[str(i) for i in picked_indices],
175 n_cols=3, use_svg=False, outfile=str(Path(directory) / "diverse.png"))
176 dm.viz.to_image(mols, use_svg=True, outfile=str(Path(directory) / "molecules.svg"))
177 dm.viz.to_image(mols[0], highlight_atom=[0, 1, 2], highlight_bond=[0, 1])
178```
180The default is SVG, so set `use_svg=False` for PNG. Use `align=True` for related series
181or an explicit common template. Interactive `dm.viz.conformers(mol_3d, n_confs=None)`
182needs a notebook frontend; only widget construction was checked locally.

9. Visualize

with TemporaryDirectory() as directory:
    dm.viz.to_image(diverse_mols, legends=[str(i) for i in picked_indices],
                     n_cols=3, use_svg=False, outfile=str(Path(directory) / "diverse.png"))
    dm.viz.to_image(mols, use_svg=True, outfile=str(Path(directory) / "molecules.svg"))
    dm.viz.to_image(mols[0], highlight_atom=[0, 1, 2], highlight_bond=[0, 1])

The default is SVG, so set use_svg=False for PNG. Use align=True for related series or an explicit common template. Interactive dm.viz.conformers(mol_3d, n_confs=None) needs a notebook frontend; only widget construction was checked locally.

Raw lines 170 to 183
184## 10. Reaction enumeration
186```python
187from rdkit.Chem import rdChemReactions
188rxn = rdChemReactions.ReactionFromSmarts("[C:1](=[O:2])[OH:3]>>[C:1](=[O:2])[Cl:3]")
189groups = dm.reactions.apply_reaction(rxn, (dm.to_mol("CC(=O)O"),))
190products = [p for group in groups for p in group if p is not None]
191assert {dm.to_smiles(p) for p in products} == {"CC(=O)Cl"}
192```
194Default output is nested product groups. Empty matches and invalid products are separate
195outcomes. SMARTS application/sanitization does not demonstrate a feasible synthesis;
196see [reaction and dataset patterns](reactions_data.md).

10. Reaction enumeration

from rdkit.Chem import rdChemReactions
rxn = rdChemReactions.ReactionFromSmarts("[C:1](=[O:2])[OH:3]>>[C:1](=[O:2])[Cl:3]")
groups = dm.reactions.apply_reaction(rxn, (dm.to_mol("CC(=O)O"),))
products = [p for group in groups for p in group if p is not None]
assert {dm.to_smiles(p) for p in products} == {"CC(=O)Cl"}

Default output is nested product groups. Empty matches and invalid products are separate outcomes. SMARTS application/sanitization does not demonstrate a feasible synthesis; see reaction and dataset patterns.

Raw lines 184 to 196