Skill Tracer
Skills › Scientific Agent Skills › pathml › image_loading.md

image_loading.md

333 lines · skills/pathml/references/image_loading.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Image loading, formats, levels, and coordinates
2
3This reference targets the **PyPI-stable PathML 3.0.8 API**. All sources were
4checked on 2026-10-01 against the v3.0.8 wheel/tag. PathML examples are
5illustrative: native readers were not executed in this review.
6

Image loading, formats, levels, and coordinates

This reference targets the PyPI-stable PathML 3.0.8 API. All sources were checked on 2026-10-01 against the v3.0.8 wheel/tag. PathML examples are illustrative: native readers were not executed in this review.

Raw lines 1 to 6
7## Start with a local, de-identified file
8
9Never infer authorization from the fact that a file is readable. Whole-slide
10images and DICOM objects can carry identifiers in pixels, labels, filenames, and
11metadata. Keep the original on approved storage, use a pseudonymous working name,
12and do not print arbitrary metadata. The bundled inspector emits only an
13allowlist of technical fields:
15```bash
16python scripts/slide_manifest.py inspect \
17 --slide data/pseudonymous_slide.svs \
18 --root .
19```
21It rejects URLs and symlinks. PathML itself accepts paths more broadly, so validate
22before constructing a slide object.

Start with a local, de-identified file

Never infer authorization from the fact that a file is readable. Whole-slide images and DICOM objects can carry identifiers in pixels, labels, filenames, and metadata. Keep the original on approved storage, use a pseudonymous working name, and do not print arbitrary metadata. The bundled inspector emits only an allowlist of technical fields:

python scripts/slide_manifest.py inspect \
  --slide data/pseudonymous_slide.svs \
  --root .

It rejects URLs and symlinks. PathML itself accepts paths more broadly, so validate before constructing a slide object.

Raw lines 7 to 23
Runs code python scripts/slide_manifest.py :16
24## Stable slide classes
26```python
27from pathml.core import (
28 CODEXSlide,
29 HESlide,
30 IHCSlide,
31 MultiparametricSlide,
32 SlideData,
33 SlideDataset,
34 VectraSlide,
35 types,
37```
39Convenience classes pass a stable `SlideType`:
41- `HESlide(...)` → `types.HE`
42- `IHCSlide(...)` → `types.IHC`
43- `MultiparametricSlide(...)` → `types.IF`, Bio-Formats by default
44- `VectraSlide(...)` → `types.Vectra`, Bio-Formats by default
45- `CODEXSlide(...)` → `types.CODEX`, Bio-Formats by default
47The generic constructor is:
49```python
50slide = SlideData(
51 "data/pseudonymous_slide.svs",
52 name="slide-001",
53 backend="openslide",
54 slide_type=types.HE,
56```
58`SlideData.from_slide()`, `read_region()`, `level_dimensions`, and
59`level_downsamples` are not stable `SlideData` APIs. Use the constructor,
60`extract_region()`, `shape`, and backend-specific objects where necessary.
62For a local cohort, instantiate slides first:
64```python
65from pathlib import Path
66from pathml.core import HESlide, SlideDataset
68root = Path("data/slides")
69paths = sorted(root.glob("*.svs"))
70slides = [HESlide(path, backend="openslide", name=path.stem) for path in paths]
71dataset = SlideDataset(slides)
72```
74Do not recursively accept arbitrary user-controlled paths. Validate a manifest,
75freeze the patient split, and then build this list.

Stable slide classes

from pathml.core import (
    CODEXSlide,
    HESlide,
    IHCSlide,
    MultiparametricSlide,
    SlideData,
    SlideDataset,
    VectraSlide,
    types,
)

Convenience classes pass a stable SlideType:

  • HESlide(...) → types.HE
  • IHCSlide(...) → types.IHC
  • MultiparametricSlide(...) → types.IF, Bio-Formats by default
  • VectraSlide(...) → types.Vectra, Bio-Formats by default
  • CODEXSlide(...) → types.CODEX, Bio-Formats by default

The generic constructor is:

slide = SlideData(
    "data/pseudonymous_slide.svs",
    name="slide-001",
    backend="openslide",
    slide_type=types.HE,
)

SlideData.from_slide(), read_region(), level_dimensions, and level_downsamples are not stable SlideData APIs. Use the constructor, extract_region(), shape, and backend-specific objects where necessary.

For a local cohort, instantiate slides first:

from pathlib import Path
from pathml.core import HESlide, SlideDataset

root = Path("data/slides")
paths = sorted(root.glob("*.svs"))
slides = [HESlide(path, backend="openslide", name=path.stem) for path in paths]
dataset = SlideDataset(slides)

Do not recursively accept arbitrary user-controlled paths. Validate a manifest, freeze the patient split, and then build this list.

Raw lines 24 to 76
77## Backends and file types

Backends and file types

Raw lines 77 to 78
79### OpenSlide
81Use `backend="openslide"` for common brightfield pyramid formats. Stable PathML
82lists:
84`.svs`, `.tif`, `.tiff`, `.bif`, `.ndpi`, `.vms`, `.vmu`, `.scn`, `.mrxs`,
85and `.svslide`.
87The complete capability depends on the installed OpenSlide build and the vendor
88subtype, not only the suffix. Some generic TIFFs are not valid WSIs, and some
89files with a supported suffix use unsupported compression.
91Native OpenSlide is required. Official PathML guidance uses
92`openslide-tools` on Debian/Ubuntu, Homebrew `openslide` on macOS, and vcpkg or
93official prebuilt binaries on Windows.

OpenSlide

Use backend="openslide" for common brightfield pyramid formats. Stable PathML lists:

.svs, .tif, .tiff, .bif, .ndpi, .vms, .vmu, .scn, .mrxs, and .svslide.

The complete capability depends on the installed OpenSlide build and the vendor subtype, not only the suffix. Some generic TIFFs are not valid WSIs, and some files with a supported suffix use unsupported compression.

Native OpenSlide is required. Official PathML guidance uses openslide-tools on Debian/Ubuntu, Homebrew openslide on macOS, and vcpkg or official prebuilt binaries on Windows.

Raw lines 79 to 94
95### Bio-Formats
97Use `backend="bioformats"` for multidimensional microscopy, OME-TIFF, QPTIFF, and
98formats OpenSlide cannot read. Bio-Formats supports a large catalogue (the
99upstream examples describe 160+ formats), including `.ome.tif`, `.ome.tiff`,
100`.qptiff`, `.czi`, `.vsi`, `.zvi`, and many laboratory formats.
102This backend requires Java, `python-bioformats`, and `python-javabridge`. It
103starts a JVM and stable source configures a large maximum heap, so isolate and
104resource-limit untrusted images. Java has an approximately 2 GB array limit in
105the backend. A listed extension is not proof that every variant loads.
107Bio-Formats returns five-dimensional arrays in PathML order:
109`(i, j, z, channel, time)` = `(row, column, z, c, t)`.
111Even singleton `z` and `time` dimensions are retained until a transform such as
112`CollapseRunsCODEX` or `CollapseRunsVectra` changes the layout.
114`extract_region(..., normalize=True)` is the default. It divides by
115`2**(8 * source_dtype.itemsize)`, multiplies by 256, then casts to `uint8`.
116This loses high-bit-depth intensity precision and is not a valid generic
117normalization for signed/floating-point acquisitions. For quantification use
118`normalize=False` in `extract_region`, `generate_tiles`, or `SlideData.run`
119(forwarded through `**kwargs`), and verify dtype/range with known pixels.
120This preserves the read values in memory; h5path storage has a separate float16
121conversion described in `data_management.md`.

Bio-Formats

Use backend="bioformats" for multidimensional microscopy, OME-TIFF, QPTIFF, and formats OpenSlide cannot read. Bio-Formats supports a large catalogue (the upstream examples describe 160+ formats), including .ome.tif, .ome.tiff, .qptiff, .czi, .vsi, .zvi, and many laboratory formats.

This backend requires Java, python-bioformats, and python-javabridge. It starts a JVM and stable source configures a large maximum heap, so isolate and resource-limit untrusted images. Java has an approximately 2 GB array limit in the backend. A listed extension is not proof that every variant loads.

Bio-Formats returns five-dimensional arrays in PathML order:

(i, j, z, channel, time) = (row, column, z, c, t).

Even singleton z and time dimensions are retained until a transform such as CollapseRunsCODEX or CollapseRunsVectra changes the layout.

extract_region(..., normalize=True) is the default. It divides by 2**(8 * source_dtype.itemsize), multiplies by 256, then casts to uint8. This loses high-bit-depth intensity precision and is not a valid generic normalization for signed/floating-point acquisitions. For quantification use normalize=False in extract_region, generate_tiles, or SlideData.run (forwarded through **kwargs), and verify dtype/range with known pixels. This preserves the read values in memory; h5path storage has a separate float16 conversion described in data_management.md.

Raw lines 95 to 122
123### DICOM
125Use `backend="dicom"` for `.dcm` or `.dicom`. Stable PathML treats DICOM frames as
126tiles. DICOM metadata is especially likely to contain PHI; de-identify with an
127approved DICOM process before PathML, preserve required UIDs consistently, and
128never dump the full dataset to logs.

DICOM

Use backend="dicom" for .dcm or .dicom. Stable PathML treats DICOM frames as tiles. DICOM metadata is especially likely to contain PHI; de-identify with an approved DICOM process before PathML, preserve required UIDs consistently, and never dump the full dataset to logs.

Raw lines 123 to 129
130### h5path
132`.h5` and `.h5path` inputs are inferred as PathML's processed HDF5 format:
134```python
135from pathml.core import SlideData
137processed = SlideData("derived/slide-001.h5path")
138```
140There is no stable `from_hdf5()` constructor. See `data_management.md`.

h5path

.h5 and .h5path inputs are inferred as PathML's processed HDF5 format:

from pathml.core import SlideData

processed = SlideData("derived/slide-001.h5path")

There is no stable from_hdf5() constructor. See data_management.md.

Raw lines 130 to 141
142## Backend inference versus explicit selection
144If `backend=None`, PathML infers a backend from the suffix. Prefer an explicit
145backend in reproducible work:
147```python
148from pathml.core import HESlide
150slide = HESlide("data/slide-001.svs", backend="openslide")
151```
153Reasons to be explicit:
155- `.tif` can mean a brightfield pyramid, OME-TIFF, or a plain raster.
156- Bio-Formats is broader but slower and starts Java.
157- Backend metadata and pyramid interpretation differ.
158- A file renamed to a recognized suffix is not thereby valid.

Backend inference versus explicit selection

If backend=None, PathML infers a backend from the suffix. Prefer an explicit backend in reproducible work:

from pathml.core import HESlide

slide = HESlide("data/slide-001.svs", backend="openslide")

Reasons to be explicit:

  • .tif can mean a brightfield pyramid, OME-TIFF, or a plain raster.
  • Bio-Formats is broader but slower and starts Java.
  • Backend metadata and pyramid interpretation differ.
  • A file renamed to a recognized suffix is not thereby valid.
Raw lines 142 to 159
160## Shape, regions, and tile generation
162`slide.shape` returns `(height, width)` for the backend's default level.
164```python
165height, width = slide.shape
167region = slide.extract_region(
168 location=(2_000, 3_000), # (i, j) = (row, column)
169 size=(512, 768), # (height, width)
170 level=1,
173tiles = slide.generate_tiles(
174 shape=(512, 512),
175 stride=(256, 256),
176 pad=False,
177 level=1,
179```
181`generate_tiles()` is lazy. Do not materialize all tiles just to count them.
182Use the bounded planner first:
184```bash
185python scripts/plan_pipeline.py \
186 --width 100000 --height 80000 \
187 --tile-size 512 --stride 256 \
188 --level-downsample 4 \
189 --level-width 25000 --level-height 20000
190```
192The level dimensions above are illustrative; supply values read from your file.
194`SlideData.run()` uses different parameter names: `tile_size`, `tile_stride`,
195`tile_pad`, and `level`.

Shape, regions, and tile generation

slide.shape returns (height, width) for the backend's default level.

height, width = slide.shape

region = slide.extract_region(
    location=(2_000, 3_000),  # (i, j) = (row, column)
    size=(512, 768),          # (height, width)
    level=1,
)

tiles = slide.generate_tiles(
    shape=(512, 512),
    stride=(256, 256),
    pad=False,
    level=1,
)

generate_tiles() is lazy. Do not materialize all tiles just to count them. Use the bounded planner first:

python scripts/plan_pipeline.py \
  --width 100000 --height 80000 \
  --tile-size 512 --stride 256 \
  --level-downsample 4 \
  --level-width 25000 --level-height 20000

The level dimensions above are illustrative; supply values read from your file.

SlideData.run() uses different parameter names: tile_size, tile_stride, tile_pad, and level.

Raw lines 160 to 196
Runs code python scripts/plan_pipeline.py :185
197## Coordinate convention
199PathML's `Tile.coords` is the top-left `(i, j)`:
201- `i`: row / vertical / image `y`
202- `j`: column / horizontal / image `x`
203- origin: top-left pixel `(0, 0)`
204- units: pixels at the **selected pyramid level**
206For OpenSlide, 3.0.8 uses `int(level_downsamples[level])`, then swaps `(i, j)`
207before calling OpenSlide's level-0 `(x, y)` API. A true downsample of 3.99 becomes
2083 for the region origin, although the requested image still uses that pyramid
209level. This shifts tiles and breaks a naive coordinate join. Prefer level 0;
210otherwise reject nonintegral downsamples before extraction:
212```python
213downsample_level = float(slide.slide.slide.level_downsamples[level])
214if not downsample_level.is_integer():
215 raise ValueError("PathML 3.0.8 truncates fractional OpenSlide downsamples; use level 0")
216```
218For the supported integral-downsample case:
220```text
221row_level0 = i_level * downsample_level
222col_level0 = j_level * downsample_level
223y_um = row_level0 * mpp_y
224x_um = col_level0 * mpp_x
225```
227Use scanner-provided level-0 MPP when reliable. Do not silently derive MPP from
228objective power. Record:
230- coordinate convention (`ij` or `xy`)
231- pyramid level and exact downsample
232- whether MPP is measured, metadata-derived, or unavailable
233- tile height/width, stride, and padding
235`QuantifyMIF` later writes `obsm["spatial"]` in `(x, y)` order, so a conversion is
236required when joining it to `Tile.coords`. For a previously processed fractional
237level, recover actual sampled origins plus within-tile coordinates; multiplying
238the combined coordinate by the true downsample cannot repair the origin error.

Coordinate convention

PathML's Tile.coords is the top-left (i, j):

  • i: row / vertical / image y
  • j: column / horizontal / image x
  • origin: top-left pixel (0, 0)
  • units: pixels at the selected pyramid level

For OpenSlide, 3.0.8 uses int(level_downsamples[level]), then swaps (i, j) before calling OpenSlide's level-0 (x, y) API. A true downsample of 3.99 becomes 3 for the region origin, although the requested image still uses that pyramid level. This shifts tiles and breaks a naive coordinate join. Prefer level 0; otherwise reject nonintegral downsamples before extraction:

downsample_level = float(slide.slide.slide.level_downsamples[level])
if not downsample_level.is_integer():
    raise ValueError("PathML 3.0.8 truncates fractional OpenSlide downsamples; use level 0")

For the supported integral-downsample case:

row_level0 = i_level * downsample_level
col_level0 = j_level * downsample_level
y_um = row_level0 * mpp_y
x_um = col_level0 * mpp_x

Use scanner-provided level-0 MPP when reliable. Do not silently derive MPP from objective power. Record:

  • coordinate convention (ij or xy)
  • pyramid level and exact downsample
  • whether MPP is measured, metadata-derived, or unavailable
  • tile height/width, stride, and padding

QuantifyMIF later writes obsm["spatial"] in (x, y) order, so a conversion is required when joining it to Tile.coords. For a previously processed fractional level, recover actual sampled origins plus within-tile coordinates; multiplying the combined coordinate by the true downsample cannot repair the origin error.

Raw lines 197 to 239
240## Pyramid levels
242For OpenSlide, level 0 is highest resolution. Later levels are downsampled, but
243the factors are slide-specific; do not assume `4x`, `16x`, or a particular
244magnification sequence.
246Backend-level inspection:
248```python
249level_count = slide.slide.level_count
250level0_shape = slide.slide.get_image_shape(level=0) # (height, width)
252# OpenSlide-specific internals, not a backend-neutral PathML contract:
253downsamples = tuple(slide.slide.slide.level_downsamples)
254dimensions_xy = tuple(slide.slide.slide.level_dimensions)
255```
257Guard backend-specific access and record it as such. Bio-Formats maps image series
258to levels; those series are not necessarily an optical pyramid.

Pyramid levels

For OpenSlide, level 0 is highest resolution. Later levels are downsampled, but the factors are slide-specific; do not assume 4x, 16x, or a particular magnification sequence.

Backend-level inspection:

level_count = slide.slide.level_count
level0_shape = slide.slide.get_image_shape(level=0)  # (height, width)

# OpenSlide-specific internals, not a backend-neutral PathML contract:
downsamples = tuple(slide.slide.slide.level_downsamples)
dimensions_xy = tuple(slide.slide.slide.level_dimensions)

Guard backend-specific access and record it as such. Bio-Formats maps image series to levels; those series are not necessarily an optical pyramid.

Raw lines 240 to 259
260## Tile count and edge behavior
262For one dimension `D`, tile extent `T`, and stride `S`, `pad=False` yields:
264```text
2650 if D < T
266floor((D - T) / S) + 1 otherwise
267```
269With `pad=True`, stable PathML follows its backend implementation, which is not
270identical to a generic `ceil(D / S)` rule for every overlapping configuration.
271Use the bundled planner and verify a small synthetic case. Padded pixels are zero,
272which can bias tissue/stain/QC transforms.
274At nonzero levels supply the exact backend width/height to the planner using
275`--level-width` and `--level-height`. The default `ceil(level0/downsample)` is an
276estimate; vendor pyramid dimensions can differ and change the tile count.
278Important stable limitation: `SlideData.generate_tiles()` does not slice
279slide-level masks into padded tiles. Do not combine a slide-level mask with
280`pad=True` without an explicit, tested padding policy.

Tile count and edge behavior

For one dimension D, tile extent T, and stride S, pad=False yields:

0                         if D < T
floor((D - T) / S) + 1    otherwise

With pad=True, stable PathML follows its backend implementation, which is not identical to a generic ceil(D / S) rule for every overlapping configuration. Use the bundled planner and verify a small synthetic case. Padded pixels are zero, which can bias tissue/stain/QC transforms.

At nonzero levels supply the exact backend width/height to the planner using --level-width and --level-height. The default ceil(level0/downsample) is an estimate; vendor pyramid dimensions can differ and change the tile count.

Important stable limitation: SlideData.generate_tiles() does not slice slide-level masks into padded tiles. Do not combine a slide-level mask with pad=True without an explicit, tested padding policy.

Raw lines 260 to 281
282## Technical metadata without PHI leakage
284PathML 3.0.8 has no backend-neutral `slide.metadata` mapping. Technical metadata
285is backend-specific:
287- OpenSlide properties are under the wrapped OpenSlide object.
288- Bio-Formats stores OME-XML in its backend `metadata`.
289- DICOM contains a full clinical metadata model.
291Default to a strict allowlist such as:
293- dimensions and level count
294- level downsamples
295- MPP X/Y
296- objective power
297- scanner vendor/model
298- pixel dtype, channels, Z, and time dimensions
300Do not emit patient name/ID, accession, dates, institution, free text, UIDs, or
301file paths. Even technical fields can be identifying in a small cohort; minimize
302what is retained.

Technical metadata without PHI leakage

PathML 3.0.8 has no backend-neutral slide.metadata mapping. Technical metadata is backend-specific:

  • OpenSlide properties are under the wrapped OpenSlide object.
  • Bio-Formats stores OME-XML in its backend metadata.
  • DICOM contains a full clinical metadata model.

Default to a strict allowlist such as:

  • dimensions and level count
  • level downsamples
  • MPP X/Y
  • objective power
  • scanner vendor/model
  • pixel dtype, channels, Z, and time dimensions

Do not emit patient name/ID, accession, dates, institution, free text, UIDs, or file paths. Even technical fields can be identifying in a small cohort; minimize what is retained.

Raw lines 282 to 303
304## Loading/QC checklist
306Before large-scale processing:
3081. Validate suffix, regular-file status, symlinks, size, and manifest uniqueness.
3092. Confirm backend and native dependencies with a non-sensitive test slide.
3103. Read a thumbnail or a few bounded regions, not the full level-0 image.
3114. Confirm color/channel order, dtype, level count, dimensions, and MPP.
3125. Check orientation, blank areas, focus, folds, pen, bubbles, coverslip edges,
313 clipping, and scanner artifacts.
3146. Confirm tile coordinates by overlaying a few sampled tiles on a thumbnail.
3157. Record failures instead of silently dropping slides.

Loading/QC checklist

Before large-scale processing:

  1. Validate suffix, regular-file status, symlinks, size, and manifest uniqueness.
  2. Confirm backend and native dependencies with a non-sensitive test slide.
  3. Read a thumbnail or a few bounded regions, not the full level-0 image.
  4. Confirm color/channel order, dtype, level count, dimensions, and MPP.
  5. Check orientation, blank areas, focus, folds, pen, bubbles, coverslip edges, clipping, and scanner artifacts.
  6. Confirm tile coordinates by overlaying a few sampled tiles on a thumbnail.
  7. Record failures instead of silently dropping slides.
Raw lines 304 to 316
317## Sources and further reading
319API baseline reviewed 2026-10-01 using the released wheel/tag; hosted docs may lag.
321- Stable loading guide:
322 https://pathml.readthedocs.io/en/stable/loading_slides.html
323- Stable core API:
324 https://pathml.readthedocs.io/en/stable/api_core_reference.html
325- Stable source (`slide_data.py`):
326 https://github.com/Dana-Farber-AIOS/pathml/blob/v3.0.8/pathml/core/slide_data.py
327- Stable source (`slide_backends.py`):
328 https://github.com/Dana-Farber-AIOS/pathml/blob/v3.0.8/pathml/core/slide_backends.py
329- Stable source (`tile.py`):
330 https://github.com/Dana-Farber-AIOS/pathml/blob/v3.0.8/pathml/core/tile.py
331- OpenSlide formats: https://openslide.org/formats/
332- Bio-Formats supported formats:
333 https://docs.openmicroscopy.org/bio-formats/latest/supported-formats.html

Sources and further reading

API baseline reviewed 2026-10-01 using the released wheel/tag; hosted docs may lag.

  • Stable loading guide: https://pathml.readthedocs.io/en/stable/loading_slides.html
  • Stable core API: https://pathml.readthedocs.io/en/stable/api_core_reference.html
  • Stable source (slide_data.py): https://github.com/Dana-Farber-AIOS/pathml/blob/v3.0.8/pathml/core/slide_data.py
  • Stable source (slide_backends.py): https://github.com/Dana-Farber-AIOS/pathml/blob/v3.0.8/pathml/core/slide_backends.py
  • Stable source (tile.py): https://github.com/Dana-Farber-AIOS/pathml/blob/v3.0.8/pathml/core/tile.py
  • OpenSlide formats: https://openslide.org/formats/
  • Bio-Formats supported formats: https://docs.openmicroscopy.org/bio-formats/latest/supported-formats.html
Raw lines 317 to 333