Datasets and event access

Contracts below were checked in the released PyHealth 2.0.2 wheel and dataset documentation. Constructors are not interchangeable across modalities.

Two dataset layers

BaseDataset exposes a lazy, table-backed event registry. get_patient(id) and iter_patients() return Patient objects; set_task(task) builds a processed, indexable sample dataset. Use those samples with models, splitters and data loaders. create_sample_dataset(records, input_schema=..., output_schema=...) is the public helper for in-memory dictionaries; SampleDataset(...) itself takes a cache path, not samples=.

from pyhealth.datasets import MIMIC3Dataset
base = MIMIC3Dataset(
    root="https://storage.googleapis.com/pyhealth/Synthetic_MIMIC-III/",
    tables=["diagnoses_icd", "procedures_icd", "prescriptions"],
    cache_dir="./cache/mimic3", num_workers=1, dev=True,
)

The MIMIC-III constructor automatically adds patients, admissions, icustays. All selectors above are lowercase; the bundled config translates them to uppercase CSV filenames. labevents additionally needs D_LABITEMS; note events have their own recording-time and availability caveats.

MIMIC-IV: two supported constructors

from pyhealth.datasets import MIMIC4Dataset, MIMIC4EHRDataset

# Combined EHR/note/CXR wrapper: modality-specific argument names
combined = MIMIC4Dataset(
    ehr_root="./data/mimic-iv",
    ehr_tables=["diagnoses_icd", "procedures_icd", "prescriptions"],
    cache_dir="./cache/mimic4",
)
# EHR-only reader: ordinary root/tables arguments
base = MIMIC4EHRDataset(
    root="./data/mimic-iv",
    tables=["diagnoses_icd", "procedures_icd", "prescriptions"],
    cache_dir="./cache/mimic4-ehr",
)

Both roots contain hosp/ and icu/. tables= is not an argument to the combined wrapper. Its other modalities use note_root/note_tables, cxr_root/cxr_tables. These restricted-data examples are illustrative; constructor/config contracts were verified, but no private MIMIC-IV records were loaded.

Available dataset families

These names are exported by 2.0.2; this inventory does not certify every task/model combination or dataset release. Read the selected class and bundled config first.

In 2.0.2, SleepEDFDataset(root=..., subset="cassette") accepts root, dataset_name, config_path, subset; it does not accept cache_dir or dev. The root contains SC-subjects.xls and the sleep-cassette/ subdirectory (or the corresponding telemetry files), unless metadata is already prepared. Reading raw .xls metadata can require the optional pandas engine xlrd. Genomics and clinical text are not imaging datasets even when they share the pipeline abstraction.

Inspect actual events and samples

patient = next(base.iter_patients())
admissions = patient.get_events(event_type="admissions")
if admissions:
    diagnoses = patient.get_events(
        event_type="diagnoses_icd",
        filters=[("hadm_id", "==", admissions[0].hadm_id)],
    )
    print([event.icd9_code for event in diagnoses])  # MIMIC-III only

MIMIC-IV diagnosis events use icd_code and icd_version. Keep the version when harmonizing vocabularies; identically shaped code strings need not mean the same thing. get_events(start=..., end=...) includes both time boundaries; return_df=True returns prefixed Polars columns such as diagnoses_icd/icd9_code. Missing event types can yield an empty frame without the expected columns: check before selecting. Do not use the legacy patient.visits, next_visit, or visit.get_code_list APIs.

Splitting and preprocessing

from pyhealth.datasets import split_by_patient, get_dataloader
train, val, test = split_by_patient(samples, [0.6, 0.2, 0.2], seed=42)
train_loader = get_dataloader(train, batch_size=32, shuffle=True)
val_loader = get_dataloader(val, batch_size=32, shuffle=False)
test_loader = get_dataloader(test, batch_size=32, shuffle=False)

Check nonempty partitions and pairwise-disjoint patient IDs. Group all recordings from the same person together for sleep/EEG as well. split_by_visit relies on record identifiers and permits patient overlap; it is unsuitable for a new-patient claim. split_by_patient does not implement time cutoffs or stratification. For a temporal study build the date-based partition explicitly and state whether returning patients are allowed. Fit feature transforms on training data only; the convenience set_task -> split route fits processors before splitting. See train-only processing.

For custom datasets, inspect BaseDataset and its config-driven load_data/ load_table interfaces. Do not start from obsolete patient/visit parsers.