Skill Tracer
Skills › Scientific Agent Skills › omero-integration › image_processing.md

image_processing.md

287 lines · skills/omero-integration/references/image_processing.md at 5f2bc03

Open on GitHub
1# Pixels, Rendering, and Derived Images
2
3Pixel planes, thumbnails, rendered images, channel labels, and physical sizes
4are data exports. Set explicit image IDs, coordinates, byte/memory caps, and
5output paths before retrieval.
6

Pixels, Rendering, and Derived Images

Pixel planes, thumbnails, rendered images, channel labels, and physical sizes are data exports. Set explicit image IDs, coordinates, byte/memory caps, and output paths before retrieval.

Raw lines 1 to 6
7## Dimensions Before Data
8
9Inspect dimensions without loading a plane:
11```python
12image = conn.getObject("Image", image_id)
13if image is None:
14 raise LookupError("Image unavailable")
16dimensions = {
17 "size_x": image.getSizeX(),
18 "size_y": image.getSizeY(),
19 "size_z": image.getSizeZ(),
20 "size_c": image.getSizeC(),
21 "size_t": image.getSizeT(),
22 "pixels_type": image.getPixelsType(),
24```
26Estimate element count and memory before a read. A single `uint16` plane uses
27roughly `size_x * size_y * 2` bytes before NumPy/container overhead. Do not
28retrieve a full 5D image by default.

Dimensions Before Data

Inspect dimensions without loading a plane:

image = conn.getObject("Image", image_id)
if image is None:
    raise LookupError("Image unavailable")

dimensions = {
    "size_x": image.getSizeX(),
    "size_y": image.getSizeY(),
    "size_z": image.getSizeZ(),
    "size_c": image.getSizeC(),
    "size_t": image.getSizeT(),
    "pixels_type": image.getPixelsType(),
}

Estimate element count and memory before a read. A single uint16 plane uses roughly size_x * size_y * 2 bytes before NumPy/container overhead. Do not retrieve a full 5D image by default.

Raw lines 7 to 29
30## One Raw Plane
32Raw pixel access is zero-based in Z, C, and T:
34```python
35z = 0
36c = 0
37t = 0
39if not 0 <= z < image.getSizeZ():
40 raise ValueError("Z out of range")
41if not 0 <= c < image.getSizeC():
42 raise ValueError("C out of range")
43if not 0 <= t < image.getSizeT():
44 raise ValueError("T out of range")
46max_pixels = 16_000_000
47if image.getSizeX() * image.getSizeY() > max_pixels:
48 raise ValueError("Plane exceeds approved pixel count; use tiles")
50pixels = image.getPrimaryPixels()
51plane = pixels.getPlane(z, c, t)
52print(plane.shape, plane.dtype)
53```
55Do not print arrays. Summaries such as min/max may still reveal signal
56distribution and should be included only when requested.

One Raw Plane

Raw pixel access is zero-based in Z, C, and T:

z = 0
c = 0
t = 0

if not 0 <= z < image.getSizeZ():
    raise ValueError("Z out of range")
if not 0 <= c < image.getSizeC():
    raise ValueError("C out of range")
if not 0 <= t < image.getSizeT():
    raise ValueError("T out of range")

max_pixels = 16_000_000
if image.getSizeX() * image.getSizeY() > max_pixels:
    raise ValueError("Plane exceeds approved pixel count; use tiles")

pixels = image.getPrimaryPixels()
plane = pixels.getPlane(z, c, t)
print(plane.shape, plane.dtype)

Do not print arrays. Summaries such as min/max may still reveal signal distribution and should be included only when requested.

Raw lines 30 to 57
58## Several Explicit Planes
60`getPlanes()` accepts a list of `(z, c, t)` tuples and returns an iterator.
61Bound the coordinate list and process incrementally:
63```python
64coordinates = [(0, 0, 0), (1, 0, 0), (2, 0, 0)]
65if len(coordinates) > 20:
66 raise ValueError("Too many planes")
68for (z, c, t), plane in zip(coordinates, pixels.getPlanes(coordinates)):
69 print({"z": z, "c": c, "t": t, "shape": plane.shape})
70```
72Do not create a coordinate list from all dimensions until the resulting count
73has been checked.

Several Explicit Planes

getPlanes() accepts a list of (z, c, t) tuples and returns an iterator. Bound the coordinate list and process incrementally:

coordinates = [(0, 0, 0), (1, 0, 0), (2, 0, 0)]
if len(coordinates) > 20:
    raise ValueError("Too many planes")

for (z, c, t), plane in zip(coordinates, pixels.getPlanes(coordinates)):
    print({"z": z, "c": c, "t": t, "shape": plane.shape})

Do not create a coordinate list from all dimensions until the resulting count has been checked.

Raw lines 58 to 74
75## Tiles for Large Images
77`getTiles()` accepts `(z, c, t, (x, y, width, height))` tuples:
79```python
80x = 0
81y = 0
82width = 512
83height = 512
84z = 0
85c = 0
86t = 0
88if width <= 0 or height <= 0:
89 raise ValueError("Tile dimensions must be positive")
90if x < 0 or y < 0:
91 raise ValueError("Tile origin must be non-negative")
92if x + width > image.getSizeX() or y + height > image.getSizeY():
93 raise ValueError("Tile exceeds image bounds")
94if width * height > 1_048_576:
95 raise ValueError("Tile exceeds approved pixel count")
97request = [(z, c, t, (x, y, width, height))]
98tile = next(pixels.getTiles(request))
99```
101For a tiled scan, cap:
103- number of tiles;
104- pixels per tile;
105- total pixels;
106- channels/Z/T;
107- memory retained at once.
109Do not infer that a rectangular tile is equivalent to a nonrectangular ROI.

Tiles for Large Images

getTiles() accepts (z, c, t, (x, y, width, height)) tuples:

x = 0
y = 0
width = 512
height = 512
z = 0
c = 0
t = 0

if width <= 0 or height <= 0:
    raise ValueError("Tile dimensions must be positive")
if x < 0 or y < 0:
    raise ValueError("Tile origin must be non-negative")
if x + width > image.getSizeX() or y + height > image.getSizeY():
    raise ValueError("Tile exceeds image bounds")
if width * height > 1_048_576:
    raise ValueError("Tile exceeds approved pixel count")

request = [(z, c, t, (x, y, width, height))]
tile = next(pixels.getTiles(request))

For a tiled scan, cap:

  • number of tiles;
  • pixels per tile;
  • total pixels;
  • channels/Z/T;
  • memory retained at once.

Do not infer that a rectangular tile is equivalent to a nonrectangular ROI.

Raw lines 75 to 110
111## Channel Metadata
113Channel metadata may include sensitive labels:
115```python
116max_channels = min(image.getSizeC(), 16)
117for index, channel in enumerate(image.getChannels()):
118 if index >= max_channels:
119 break
120 print(
121 {
122 "index": index,
123 "label_redacted": True,
124 "color": channel.getColor().getRGB(),
125 "lut": channel.getLut(),
126 "reverse_intensity": channel.isInverted(),
127 }
128 )
129```
131Raw pixel channel indices are zero-based. BlitzGateway rendering channel
132selectors are one-based. Keep this conversion explicit.

Channel Metadata

Channel metadata may include sensitive labels:

max_channels = min(image.getSizeC(), 16)
for index, channel in enumerate(image.getChannels()):
    if index >= max_channels:
        break
    print(
        {
            "index": index,
            "label_redacted": True,
            "color": channel.getColor().getRGB(),
            "lut": channel.getLut(),
            "reverse_intensity": channel.isInverted(),
        }
    )

Raw pixel channel indices are zero-based. BlitzGateway rendering channel selectors are one-based. Keep this conversion explicit.

Raw lines 111 to 133
134## Physical Dimensions
136Physical sizes can be absent:
138```python
139for axis, value in (
140 ("x", image.getPixelSizeX(units=True)),
141 ("y", image.getPixelSizeY(units=True)),
142 ("z", image.getPixelSizeZ(units=True)),
143):
144 if value is not None:
145 print(axis, value.getValue(), value.getSymbol())
146```
148Preserve the unit. Do not assume an unwrapped numeric value has the unit needed
149by downstream analysis.
151Changing pixel sizes mutates the server model and must be a separately
152reviewed write. Do not “correct” missing metadata automatically.

Physical Dimensions

Physical sizes can be absent:

for axis, value in (
    ("x", image.getPixelSizeX(units=True)),
    ("y", image.getPixelSizeY(units=True)),
    ("z", image.getPixelSizeZ(units=True)),
):
    if value is not None:
        print(axis, value.getValue(), value.getSymbol())

Preserve the unit. Do not assume an unwrapped numeric value has the unit needed by downstream analysis.

Changing pixel sizes mutates the server model and must be a separately reviewed write. Do not “correct” missing metadata automatically.

Raw lines 134 to 153
154## Thumbnail Bytes
156`getThumbnail()` returns encoded image bytes using current rendering settings:
158```python
159from io import BytesIO
160from PIL import Image
162thumbnail_bytes = image.getThumbnail(size=(96, 96))
163thumbnail = Image.open(BytesIO(thumbnail_bytes))
164thumbnail.load()
165print(thumbnail.size)
166```
168To save, use a caller-selected filename and refuse collisions/symlinks:
170```python
171from pathlib import Path
173destination = Path("./image-123-thumbnail.png")
174if destination.exists() or destination.is_symlink():
175 raise FileExistsError(destination)
176thumbnail.save(destination, format="PNG")
177```
179Do not derive `destination` from `image.getName()`.

Thumbnail Bytes

getThumbnail() returns encoded image bytes using current rendering settings:

from io import BytesIO
from PIL import Image

thumbnail_bytes = image.getThumbnail(size=(96, 96))
thumbnail = Image.open(BytesIO(thumbnail_bytes))
thumbnail.load()
print(thumbnail.size)

To save, use a caller-selected filename and refuse collisions/symlinks:

from pathlib import Path

destination = Path("./image-123-thumbnail.png")
if destination.exists() or destination.is_symlink():
    raise FileExistsError(destination)
thumbnail.save(destination, format="PNG")

Do not derive destination from image.getName().

Raw lines 154 to 180
181## Rendering
183`renderImage(z, t, compression=0.9)` returns a Pillow image:
185```python
186z = image.getSizeZ() // 2
187t = 0
188rendered = image.renderImage(z, t, compression=0.9)
189```
191The rendered result reflects the current rendering model, active channels,
192colors, windows, LUTs, and defaults. Record those settings when a reproducible
193figure depends on them.
195Current official examples set active rendering channels with one-based
196indices:
198```python
199image.setActiveChannels(
200 [1, 2],
201 [[20.0, 300.0], [50.0, 500.0]],
202 ["00FF00", "FF0000"],
204rendered = image.renderImage(z, t)
205```
207This initializes a stateful rendering engine. Keep the rendering scope short.
208Closing the BlitzGateway closes its tracked services; if using low-level
209stateful services directly, close each in `finally`. Do not depend on private
210attributes such as `image._re` in durable code.
212`saveDefaults()` or other persistence calls change server rendering settings.
213Do not call them in a read/render helper. Rendering locally does not authorize
214persisting new defaults.

Rendering

renderImage(z, t, compression=0.9) returns a Pillow image:

z = image.getSizeZ() // 2
t = 0
rendered = image.renderImage(z, t, compression=0.9)

The rendered result reflects the current rendering model, active channels, colors, windows, LUTs, and defaults. Record those settings when a reproducible figure depends on them.

Current official examples set active rendering channels with one-based indices:

image.setActiveChannels(
    [1, 2],
    [[20.0, 300.0], [50.0, 500.0]],
    ["00FF00", "FF0000"],
)
rendered = image.renderImage(z, t)

This initializes a stateful rendering engine. Keep the rendering scope short. Closing the BlitzGateway closes its tracked services; if using low-level stateful services directly, close each in finally. Do not depend on private attributes such as image._re in durable code.

saveDefaults() or other persistence calls change server rendering settings. Do not call them in a read/render helper. Rendering locally does not authorize persisting new defaults.

Raw lines 181 to 215
216## Histograms and Statistics
218Histograms and min/max statistics can be large or expensive across many
219channels/planes. Restrict:
221- one explicit image;
222- an allowlisted channel list;
223- bin count;
224- Z/T;
225- number of returned arrays.
227Do not use whole-dataset histograms as a connectivity test.

Histograms and Statistics

Histograms and min/max statistics can be large or expensive across many channels/planes. Restrict:

  • one explicit image;
  • an allowlisted channel list;
  • bin count;
  • Z/T;
  • number of returned arrays.

Do not use whole-dataset histograms as a connectivity test.

Raw lines 216 to 228
229## Derived Images Are Writes
231`BlitzGateway.createImageFromNumpySeq(...)` creates a server image:
233```python
234result = conn.createImageFromNumpySeq(
235 plane_iterator,
236 "reviewed-derived-image",
237 sizeZ=1,
238 sizeC=source.getSizeC(),
239 sizeT=source.getSizeT(),
240 description="Method and source IDs recorded separately",
241 dataset=target_dataset,
242 sourceImageId=source.getId(),
244```
246Before execution:
2481. use T-fastest plane order: `for z ...: for c ...: for t ...`, with
249 exactly `sizeZ * sizeC * sizeT` arrays of shape `(sizeY, sizeX)`;
2502. validate each shape and dtype;
2513. cap source planes and memory;
2524. confirm target dataset and group;
2535. confirm output name/description contains no secrets;
2546. decide cleanup for a partial write;
2557. copy physical dimensions only when semantically valid.
257For a maximum-intensity projection, the derived image has one Z plane.
258Do not copy a source Z spacing that no longer describes the data.

Derived Images Are Writes

BlitzGateway.createImageFromNumpySeq(...) creates a server image:

result = conn.createImageFromNumpySeq(
    plane_iterator,
    "reviewed-derived-image",
    sizeZ=1,
    sizeC=source.getSizeC(),
    sizeT=source.getSizeT(),
    description="Method and source IDs recorded separately",
    dataset=target_dataset,
    sourceImageId=source.getId(),
)

Before execution:

  1. use T-fastest plane order: for z ...: for c ...: for t ..., with exactly sizeZ * sizeC * sizeT arrays of shape (sizeY, sizeX);
  2. validate each shape and dtype;
  3. cap source planes and memory;
  4. confirm target dataset and group;
  5. confirm output name/description contains no secrets;
  6. decide cleanup for a partial write;
  7. copy physical dimensions only when semantically valid.

For a maximum-intensity projection, the derived image has one Z plane. Do not copy a source Z spacing that no longer describes the data.

Raw lines 229 to 259
260## Dtype Handling
262Keep the source dtype unless the algorithm requires conversion:
264```python
265import numpy as np
267plane_float = plane.astype(np.float32)
268# Perform reviewed numerical processing.
269result = np.clip(plane_float, 0, np.iinfo(np.uint16).max).astype(np.uint16)
270```
272Document clipping, scaling, normalization, and rounding. Never cast a float
273array to an integer type without checking range and non-finite values.

Dtype Handling

Keep the source dtype unless the algorithm requires conversion:

import numpy as np

plane_float = plane.astype(np.float32)
# Perform reviewed numerical processing.
result = np.clip(plane_float, 0, np.iinfo(np.uint16).max).astype(np.uint16)

Document clipping, scaling, normalization, and rounding. Never cast a float array to an integer type without checking range and non-finite values.

Raw lines 260 to 274
275## Rendering and Pixel Checklist
277- Explicit image ID and group
278- Dimensions inspected before retrieval
279- Z/C/T and coordinates range-checked
280- Plane/tile count and total pixels bounded
281- Raw channel indexing distinguished from rendering indexing
282- Labels and pixel-derived values classified for export
283- Caller-selected non-symlink output with collision refusal
284- Rendering services short-lived
285- No rendering-default save in read-only workflows
286- Derived-image creation separately approved
287- Connection and stateful services closed

Rendering and Pixel Checklist

  • Explicit image ID and group
  • Dimensions inspected before retrieval
  • Z/C/T and coordinates range-checked
  • Plane/tile count and total pixels bounded
  • Raw channel indexing distinguished from rendering indexing
  • Labels and pixel-derived values classified for export
  • Caller-selected non-symlink output with collision refusal
  • Rendering services short-lived
  • No rendering-default save in read-only workflows
  • Derived-image creation separately approved
  • Connection and stateful services closed
Raw lines 275 to 287