Skill Tracer
Skills › Scientific Agent Skills › geopandas › spatial-analysis.md

spatial-analysis.md

295 lines · skills/geopandas/references/spatial-analysis.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Spatial joins, overlay, clip, and dissolve
2
3Spatial operations can multiply, split, or merge records. Define geometry
4semantics and expected cardinality before running them, then audit both.
5

Spatial joins, overlay, clip, and dissolve

Spatial operations can multiply, split, or merge records. Define geometry semantics and expected cardinality before running them, then audit both.

Raw lines 1 to 5
6## Shared preflight
7
8For each input:
9
101. preserve a non-null stable feature-ID column;
112. count duplicate IDs and duplicate pandas indices;
123. require CRS and reproject to a common, justified CRS;
134. count null, empty, invalid, mixed, and Z/M geometries;
145. validate precision/accuracy compatibility;
156. state whether boundary contact should count;
167. state the expected one-to-one, one-to-many, many-to-one, or many-to-many
17 relationship.
19GeoPandas operations are planar and ignore Z. Geographic longitude/latitude is
20not suitable for distance/nearest work.

Shared preflight

For each input:

  1. preserve a non-null stable feature-ID column;
  2. count duplicate IDs and duplicate pandas indices;
  3. require CRS and reproject to a common, justified CRS;
  4. count null, empty, invalid, mixed, and Z/M geometries;
  5. validate precision/accuracy compatibility;
  6. state whether boundary contact should count;
  7. state the expected one-to-one, one-to-many, many-to-one, or many-to-many relationship.

GeoPandas operations are planar and ignore Z. Geographic longitude/latitude is not suitable for distance/nearest work.

Raw lines 6 to 21
22## Attribute joins
24Call `merge` from the spatial object so geometry dtype and CRS are retained:
26```python
27result = zones.merge(
28 attributes,
29 on="zone_id",
30 how="left",
31 validate="one_to_one",
32 indicator=True,
34```
36Use pandas `validate=` whenever the key contract is known. Before merging, audit
37null/duplicate keys on both sides. Afterward, count `_merge` categories and
38verify row count. A pandas index is not a feature key.

Attribute joins

Call merge from the spatial object so geometry dtype and CRS are retained:

result = zones.merge(
    attributes,
    on="zone_id",
    how="left",
    validate="one_to_one",
    indicator=True,
)

Use pandas validate= whenever the key contract is known. Before merging, audit null/duplicate keys on both sides. Afterward, count _merge categories and verify row count. A pandas index is not a feature key.

Raw lines 22 to 39
40## Binary-predicate spatial joins
42GeoPandas 1.2.0 signature:
44```python
45joined = left.sjoin(
46 right,
47 how="inner",
48 predicate="intersects",
49 distance=None,
50 on_attribute=None,
52```
54`predicate` is evaluated directionally from each left geometry against right
55geometries. Query available values from
56`left.sindex.valid_query_predicates`.
58Common choices:
60- point-in-polygon strict interior: points on polygon boundaries do not satisfy
61 `within`;
62- point in polygon including boundary: use `covered_by` for points on the
63 left and polygons on the right; reversing the frames permits `covers`;
64- any shared boundary/interior: `intersects`;
65- boundary-only relationships: `touches`;
66- distance threshold: `dwithin`.
68`dwithin` requires `distance`. It may be a scalar or a one-dimensional array
69with one value per **left row**; values are CRS units:
71```python
72joined = left.sjoin(
73 right,
74 predicate="dwithin",
75 distance=500,
77```
79`on_attribute="category"` (or a list/tuple) adds equality on columns present in
80both frames after the spatial predicate. It is a restriction, not a substitute
81for checking nulls, normalization, and duplicates in the attribute.

Binary-predicate spatial joins

GeoPandas 1.2.0 signature:

joined = left.sjoin(
    right,
    how="inner",
    predicate="intersects",
    distance=None,
    on_attribute=None,
)

predicate is evaluated directionally from each left geometry against right geometries. Query available values from left.sindex.valid_query_predicates.

Common choices:

  • point-in-polygon strict interior: points on polygon boundaries do not satisfy within;
  • point in polygon including boundary: use covered_by for points on the left and polygons on the right; reversing the frames permits covers;
  • any shared boundary/interior: intersects;
  • boundary-only relationships: touches;
  • distance threshold: dwithin.

dwithin requires distance. It may be a scalar or a one-dimensional array with one value per left row; values are CRS units:

joined = left.sjoin(
    right,
    predicate="dwithin",
    distance=500,
)

on_attribute="category" (or a list/tuple) adds equality on columns present in both frames after the spatial predicate. It is a restriction, not a substitute for checking nulls, normalization, and duplicates in the attribute.

Raw lines 40 to 82
83### Geometry and index retention
85- `how="left"` keeps left keys and only left geometry.
86- `how="right"` keeps right keys and only right geometry.
87- `how="inner"` keeps matching pairs and only left geometry.
88- GeoPandas 1.0 preserves a named right index as that output column name;
89 otherwise output often uses `index_right`. Do not hard-code that name as a
90 permanent ID.
92Every qualifying pair produces a row. One left feature intersecting three right
93features yields three rows. Empty/null geometries do not produce predicate
94matches.

Geometry and index retention

  • how="left" keeps left keys and only left geometry.
  • how="right" keeps right keys and only right geometry.
  • how="inner" keeps matching pairs and only left geometry.
  • GeoPandas 1.0 preserves a named right index as that output column name; otherwise output often uses index_right. Do not hard-code that name as a permanent ID.

Every qualifying pair produces a row. One left feature intersecting three right features yields three rows. Empty/null geometries do not produce predicate matches.

Raw lines 83 to 95
96### Cardinality audit
98Add internal unique row numbers before joining; never expose user IDs in a
99public report:
101```python
102left_work = left.reset_index(drop=True).assign(_left_row=lambda x: range(len(x)))
103right_work = right.reset_index(drop=True).assign(_right_row=lambda x: range(len(x)))
105pairs = left_work[["_left_row", left_work.geometry.name]].sjoin(
106 right_work[["_right_row", right_work.geometry.name]],
107 predicate="intersects",
108 how="inner",
111left_multiplicity = pairs.groupby("_left_row").size()
112right_multiplicity = pairs.groupby("_right_row").size()
113```
115Report pair count, matched/unmatched counts on both sides, and counts with
116multiple matches. Compare these to the declared contract. The bundled
117`scripts/spatial_join_audit.py` implements this with redacted aggregate output.

Cardinality audit

Add internal unique row numbers before joining; never expose user IDs in a public report:

left_work = left.reset_index(drop=True).assign(_left_row=lambda x: range(len(x)))
right_work = right.reset_index(drop=True).assign(_right_row=lambda x: range(len(x)))

pairs = left_work[["_left_row", left_work.geometry.name]].sjoin(
    right_work[["_right_row", right_work.geometry.name]],
    predicate="intersects",
    how="inner",
)

left_multiplicity = pairs.groupby("_left_row").size()
right_multiplicity = pairs.groupby("_right_row").size()

Report pair count, matched/unmatched counts on both sides, and counts with multiple matches. Compare these to the declared contract. The bundled scripts/spatial_join_audit.py implements this with redacted aggregate output.

Raw lines 96 to 118
Runs code scripts/spatial_join_audit.py :117
119## Nearest spatial joins
121```python
122nearest = left.sjoin_nearest(
123 right,
124 how="left",
125 max_distance=1_000,
126 distance_col="distance_crs_units",
127 exclusive=False,
129```
131Key semantics:
133- distance and `max_distance` use CRS units;
134- geographic CRS results are inaccurate;
135- `max_distance > 0` can substantially reduce work and limits accepted matches;
136- all equidistant nearest or intersecting neighbors are returned, so one input
137 can produce multiple rows;
138- `exclusive=True` excludes geometrically equal nearest candidates;
139- GeoPandas `sjoin_nearest` has no `k=` argument. For k-nearest behavior use a
140 separately designed spatial-index/neighbor workflow and define tie handling.
142Never silently keep the first tie. Report tie counts and specify a deterministic
143domain rule if one result must be selected.

Nearest spatial joins

nearest = left.sjoin_nearest(
    right,
    how="left",
    max_distance=1_000,
    distance_col="distance_crs_units",
    exclusive=False,
)

Key semantics:

  • distance and max_distance use CRS units;
  • geographic CRS results are inaccurate;
  • max_distance > 0 can substantially reduce work and limits accepted matches;
  • all equidistant nearest or intersecting neighbors are returned, so one input can produce multiple rows;
  • exclusive=True excludes geometrically equal nearest candidates;
  • GeoPandas sjoin_nearest has no k= argument. For k-nearest behavior use a separately designed spatial-index/neighbor workflow and define tie handling.

Never silently keep the first tie. Report tie counts and specify a deterministic domain rule if one result must be selected.

Raw lines 119 to 144
145## Overlay
147```python
148result = left.overlay(
149 right,
150 how="intersection",
151 keep_geom_type=False,
152 make_valid=False,
154```
156Modes:
158| `how` | Result |
159|---|---|
160| `intersection` | Areas/parts shared by both |
161| `union` | All partitioned parts with attributes from either/both |
162| `identity` | All left parts split by right |
163| `difference` | Left minus right |
164| `symmetric_difference` | Parts in exactly one input |
166Constraints and choices:
168- Each input must have a uniform supported family: (Multi)Polygon,
169 (Multi)Point, or line/LinearRing family.
170- Inputs need the same CRS.
171- `make_valid=True` repairs invalid inputs and may change types; pre-audit repair
172 is more traceable. With `False`, invalid inputs raise.
173- `keep_geom_type=None` behaves as `True` and warns when dropping other result
174 types. Set it explicitly and count dropped/type-changed results.
175- Union-like modes place `NaN` in attributes absent from one side.
176- Near-coincident boundaries and precision mismatch produce slivers. Use a
177 justified precision model, quantify small parts, and validate area balance.
179Always keep source IDs from both inputs. Overlay can split one source feature
180into many derived records.

Overlay

result = left.overlay(
    right,
    how="intersection",
    keep_geom_type=False,
    make_valid=False,
)

Modes:

howResult
intersectionAreas/parts shared by both
unionAll partitioned parts with attributes from either/both
identityAll left parts split by right
differenceLeft minus right
symmetric_differenceParts in exactly one input

Constraints and choices:

  • Each input must have a uniform supported family: (Multi)Polygon, (Multi)Point, or line/LinearRing family.
  • Inputs need the same CRS.
  • make_valid=True repairs invalid inputs and may change types; pre-audit repair is more traceable. With False, invalid inputs raise.
  • keep_geom_type=None behaves as True and warns when dropping other result types. Set it explicitly and count dropped/type-changed results.
  • Union-like modes place NaN in attributes absent from one side.
  • Near-coincident boundaries and precision mismatch produce slivers. Use a justified precision model, quantify small parts, and validate area balance.

Always keep source IDs from both inputs. Overlay can split one source feature into many derived records.

Raw lines 145 to 181
182## Clip
184```python
185clipped = gpd.clip(
186 features,
187 mask,
188 keep_geom_type=False,
189 sort=False,
191```
193- Both layers must share CRS.
194- Multiple mask geometries are dissolved before intersection, so mask
195 attributes are not transferred.
196- A four-number `(minx, miny, maxx, maxy)` mask activates a fast rectangle
197 path. It is possibly dirty, does not guarantee valid output, and can omit a
198 line that collapses to a point.
199- `keep_geom_type=False` retains mixed-dimensional outputs; set deliberately.
200- `sort=False` does not promise source order. Preserve IDs and sort explicitly
201 if order is part of the contract.
203Validate clip output. Use overlay intersection when mask attributes or more
204auditable topology are required.

Clip

clipped = gpd.clip(
    features,
    mask,
    keep_geom_type=False,
    sort=False,
)
  • Both layers must share CRS.
  • Multiple mask geometries are dissolved before intersection, so mask attributes are not transferred.
  • A four-number (minx, miny, maxx, maxy) mask activates a fast rectangle path. It is possibly dirty, does not guarantee valid output, and can omit a line that collapses to a point.
  • keep_geom_type=False retains mixed-dimensional outputs; set deliberately.
  • sort=False does not promise source order. Preserve IDs and sort explicitly if order is part of the contract.

Validate clip output. Use overlay intersection when mask attributes or more auditable topology are required.

Raw lines 182 to 205
206## Dissolve
208`dissolve` performs `groupby.agg` on attributes and `union_all` on geometry:
210```python
211dissolved = parcels.dissolve(
212 by="region_id",
213 aggfunc={
214 "population": "sum",
215 "source_date": "max",
216 },
217 as_index=False,
218 dropna=False,
219 method="unary",
220 grid_size=0.01,
222```
224Avoid the default `aggfunc="first"` for meaningful attributes. Specify every
225aggregation and its units. Decide whether null group keys should be dropped
226(`dropna=True`) or retained as a group (`False`).
228Union methods:
230- `unary`: robust general default; supports `grid_size`;
231- `coverage`: fast only for proven non-overlapping, edge-matched polygons and
232 may produce invalid output otherwise;
233- `disjoint_subset`: Shapely >=2.1, useful for disjoint partitions.
235For coverage mode, run `is_valid_coverage()` first. After dissolve, compare
236group counts, summed attributes, validity, empty output, and area in a suitable
237projected CRS.

Dissolve

dissolve performs groupby.agg on attributes and union_all on geometry:

dissolved = parcels.dissolve(
    by="region_id",
    aggfunc={
        "population": "sum",
        "source_date": "max",
    },
    as_index=False,
    dropna=False,
    method="unary",
    grid_size=0.01,
)

Avoid the default aggfunc="first" for meaningful attributes. Specify every aggregation and its units. Decide whether null group keys should be dropped (dropna=True) or retained as a group (False).

Union methods:

  • unary: robust general default; supports grid_size;
  • coverage: fast only for proven non-overlapping, edge-matched polygons and may produce invalid output otherwise;
  • disjoint_subset: Shapely >=2.1, useful for disjoint partitions.

For coverage mode, run is_valid_coverage() first. After dissolve, compare group counts, summed attributes, validity, empty output, and area in a suitable projected CRS.

Raw lines 206 to 238
239## Spatial index
241GeoPandas uses Shapely's spatial index automatically for joins, clip, and
242overlay. Direct queries are candidate/predicate operations:
244```python
245predicate_names = gdf.sindex.valid_query_predicates
246indices = gdf.sindex.query(query_geometry, predicate="intersects")
247```
249GeoPandas 1.0 removed `sindex.query_bulk`; use `query`. GeoPandas 1.1 supports
250indices, dense boolean, and optional SciPy sparse boolean output formats. For array input, `output_format="indices"` returns a `(2, n)` integer array:
251row 0 indexes input/query geometries and row 1 indexes the tree geometries.
252Dense/sparse boolean output is `(len(tree), len(query))`; scalar queries yield
253a one-dimensional result. Set the format explicitly and preserve this orientation.
255Spatial indexing does not fix CRS, invalid geometry, distance units, predicate
256direction, or cardinality.

Spatial index

GeoPandas uses Shapely's spatial index automatically for joins, clip, and overlay. Direct queries are candidate/predicate operations:

predicate_names = gdf.sindex.valid_query_predicates
indices = gdf.sindex.query(query_geometry, predicate="intersects")

GeoPandas 1.0 removed sindex.query_bulk; use query. GeoPandas 1.1 supports indices, dense boolean, and optional SciPy sparse boolean output formats. For array input, output_format="indices" returns a (2, n) integer array: row 0 indexes input/query geometries and row 1 indexes the tree geometries. Dense/sparse boolean output is (len(tree), len(query)); scalar queries yield a one-dimensional result. Set the format explicitly and preserve this orientation.

Spatial indexing does not fix CRS, invalid geometry, distance units, predicate direction, or cardinality.

Raw lines 239 to 257
258## Area and distance checks
260For planar metrics:
262```python
263if gdf.crs is None or gdf.crs.is_geographic:
264 raise ValueError("Use a justified projected CRS")
266area = gdf.geometry.area
267length = gdf.geometry.length
268distance = gdf.geometry.distance(reference_geometry)
269```
271Confirm axis unit and conversion factor before labeling values. For
272large/global or cross-zone work, use a geodesic design instead.

Area and distance checks

For planar metrics:

if gdf.crs is None or gdf.crs.is_geographic:
    raise ValueError("Use a justified projected CRS")

area = gdf.geometry.area
length = gdf.geometry.length
distance = gdf.geometry.distance(reference_geometry)

Confirm axis unit and conversion factor before labeling values. For large/global or cross-zone work, use a geodesic design instead.

Raw lines 258 to 273
274## Provenance checklist
276Record:
278- source hashes, layer names, stable IDs, input row/state counts;
279- source and operation CRS, units, and transform pipeline;
280- predicate direction, boundary semantics, distance/max-distance;
281- join type, attribute restrictions, expected and observed cardinality;
282- validity repair, precision grid, overlay/union method;
283- mask dissolve/rectangle choice, dissolve aggregations;
284- output row/type/state counts and new artifact hash.

Provenance checklist

Record:

  • source hashes, layer names, stable IDs, input row/state counts;
  • source and operation CRS, units, and transform pipeline;
  • predicate direction, boundary semantics, distance/max-distance;
  • join type, attribute restrictions, expected and observed cardinality;
  • validity repair, precision grid, overlay/union method;
  • mask dissolve/rectangle choice, dissolve aggregations;
  • output row/type/state counts and new artifact hash.
Raw lines 274 to 285
286## Sources (verified 2026-10-01)
288- [GeoPandas merging data guide](https://geopandas.org/en/stable/docs/user_guide/mergingdata.html).
289- [geopandas.sjoin](https://geopandas.org/en/stable/docs/reference/api/geopandas.sjoin.html).
290- [geopandas.sjoin_nearest](https://geopandas.org/en/stable/docs/reference/api/geopandas.sjoin_nearest.html).
291- [geopandas.overlay](https://geopandas.org/en/stable/docs/reference/api/geopandas.overlay.html).
292- [geopandas.clip](https://geopandas.org/en/stable/docs/reference/api/geopandas.clip.html).
293- [GeoDataFrame.dissolve](https://geopandas.org/en/stable/docs/reference/api/geopandas.GeoDataFrame.dissolve.html).
294- [GeoPandas set operations guide](https://geopandas.org/en/stable/docs/user_guide/set_operations.html).
295- [GeoPandas aggregation with dissolve](https://geopandas.org/en/stable/docs/user_guide/aggregation_with_dissolve.html).