lgtm
This commit is contained in:
parent
2efa4d9f47
commit
5e73287eaf
99 changed files with 6392 additions and 1462 deletions
|
|
@ -79,7 +79,9 @@ The output of `process_oa` is `list[(postcode, polygon)]` — the per-OA fragmen
|
|||
|
||||
**Fragment merging** (`output.py:merge_fragments`): Groups all fragments by postcode, unions them. If the result is a MultiPolygon (meaning the postcode has disconnected pieces — either from spanning OAs with a gap, or algorithm artifacts), applies a 5m buffer-then-unbuffer to close tiny gaps from floating-point mismatches at OA boundary edges. If still a MultiPolygon after that, keeps the largest part **plus any other part ≥ `_MIN_DETACHED_PART_AREA` (100 m²)** (`_keep_polygon_parts`); only sub-100 m² noise slivers are dropped. Keeping substantial detached parts matters because a postcode genuinely split across an OA seam (by a railway, river, or main road wider than the 5m buffer) would otherwise lose a chunk — measured at ~1.8% of merged area left as uncovered gaps (often 3000–5000 m² building blocks) before this change.
|
||||
|
||||
**GeoJSON output** (`output.py:write_district_geojson`): Two passes. Pass 1 converts every postcode from BNG to WGS84 (pyproj), simplifies with 1m tolerance (Douglas-Peucker), and snaps to 6 decimal places (~0.1m precision); multi-part postcodes become `MultiPolygon` (`to_wgs84_geojson_multi`, each part handled independently), single-part stay `Polygon`. The whole set is then made a **partition** (`_resolve_overlaps`): each postcode is trimmed by the union of its higher-priority overlapping neighbours, where **priority = ascending area** (smaller postcodes win contested ground). That single rule handles both seam overlap *and* containment — an enclosed postcode is always smaller than its container, so it keeps its area while the container gets a hole (the query uses both the `overlaps` and `contains` predicates, since `overlaps` alone excludes containment). This runs last, so nothing re-introduces overlap; a postcode that would be emptied keeps its original geometry, so no active postcode is dropped. Pass 2 groups postcodes by district (the outward code, e.g. `SW1A` from `SW1A 1AA`), rounds coordinates to 6dp, and writes a `{district}.geojson` FeatureCollection. Each Feature has `postcodes` (formatted like `"SW1A 1AA"`) and `mapit_code` (no space: `"SW1A1AA"`) in its properties.
|
||||
**Greenspace subtraction is connectivity-preserving** (`greenspace.py:subtract_greenspace`): park/water polygons are subtracted from each postcode, but greenspace that *crosses* a postcode (a river, a strip of parkland, a golf course through a village) would otherwise split it into scattered pieces. When the subtraction disconnects a postcode, `_reconnect_split` re-adds the narrowest removed necks — a morphological closing (`_RECONNECT_BRIDGE_M`, 25 m) clipped to the original postcode footprint — so parts ≤ ~50 m apart stay joined by a thin bridge of the postcode's own land (no address moves); genuinely wide barriers stay subtracted and the postcode legitimately splits.
|
||||
|
||||
**GeoJSON output** (`output.py:write_district_geojson`): three passes. Pass 1 converts every postcode from BNG to WGS84 (pyproj), simplifies with 1m tolerance (Douglas-Peucker), and snaps to 6 decimal places (~0.1m precision); multi-part postcodes become `MultiPolygon` (`to_wgs84_geojson_multi`, each part handled independently), single-part stay `Polygon`. The whole set is then made a **partition** (`_resolve_overlaps`): each postcode is trimmed by the union of its higher-priority overlapping neighbours, where **priority = ascending area** (smaller postcodes win contested ground). That single rule handles both seam overlap *and* containment — an enclosed postcode is always smaller than its container, so it keeps its area while the container gets a hole (the query uses both the `overlaps` and `contains` predicates, since `overlaps` alone excludes containment). This runs last, so nothing re-introduces overlap; a postcode that would be emptied keeps its original geometry, so no active postcode is dropped. Pass 2 **de-fragments** the partition (`_eliminate_small_detached_parts`): a detached part that is *both* small in absolute terms (< `_ELIM_ABS_MAX_M2`, 3000 m²) *and* a minor fraction (< `_ELIM_FRAC_MAX`, 15%) of its postcode is absorbed into the neighbouring postcode it shares the most boundary with — the classic GIS *eliminate*. This removes the Voronoi/INSPIRE/seam *scatter* that left ~1/3 of postcodes non-contiguous, while a genuine bisection (two substantial parts split by a river/railway) keeps both parts. The land is **reassigned**, never dropped, so the output stays a gapless partition and coverage is conserved; the largest part of every postcode is always retained, so no active postcode is dropped (a tiny neighbour-less sliver in removed greenspace is dropped, a larger isolated patch is kept). Pass 3 groups postcodes by district (the outward code, e.g. `SW1A` from `SW1A 1AA`), rounds coordinates to 6dp, and writes a `{district}.geojson` FeatureCollection. Each Feature has `postcodes` (formatted like `"SW1A 1AA"`) and `mapit_code` (no space: `"SW1A1AA"`) in its properties.
|
||||
|
||||
## Memory architecture
|
||||
|
||||
|
|
@ -107,6 +109,7 @@ Key design choices:
|
|||
2. **Every postcode that exists in the UPRN data gets a polygon** — unless all its UPRNs share coordinates with another postcode's UPRNs (handled by jitter) or it has zero UPRNs
|
||||
3. **Postcode polygons never extend outside their OA(s)** — all geometry is clipped to OA boundaries
|
||||
4. **A postcode split across an OA seam keeps all its substantial parts** — `merge_fragments` keeps every part ≥ 100 m² and the output is emitted as a `MultiPolygon` (the Rust server `postcodes.rs` and `loader.py` both parse MultiPolygon); only sub-100 m² noise slivers are dropped
|
||||
5. **Postcodes are contiguous unless genuinely split** — most non-contiguity is *scatter* (a unit drawn as many disconnected specks) from point-Voronoi over sparse/interleaved UPRNs, greenspace cutting across a unit, and overlap/seam slivers. Connectivity-preserving greenspace subtraction + the `_eliminate_small_detached_parts` de-fragmentation pass absorb that scatter into neighbours (coverage-conserving), cutting the share of multi-part postcodes roughly in half (~30% → ~14% measured on the worst rural/coastal districts) without dropping any postcode or leaving coverage gaps. Genuine bisections (river/railway/major road, or a detached part above the absolute+fraction thresholds) are preserved.
|
||||
|
||||
## Module structure
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ from shapely import make_valid, wkb
|
|||
from shapely.geometry import MultiPolygon, Polygon
|
||||
from shapely.strtree import STRtree
|
||||
|
||||
from .geometry import safe_difference, safe_union
|
||||
from .geometry import _SNAP_GRID, _poly_valid, safe_difference, safe_intersection, safe_union
|
||||
|
||||
|
||||
def load_greenspace(path: Path) -> tuple[STRtree, list]:
|
||||
|
|
@ -36,6 +36,42 @@ def load_greenspace(path: Path) -> tuple[STRtree, list]:
|
|||
|
||||
MAX_REMOVAL_FRACTION = 0.9 # Keep original if >90% would be removed
|
||||
|
||||
# Greenspace that merely trims the edge of a postcode is fine, but greenspace
|
||||
# that CROSSES it (a river, a strip of parkland, a golf course running through a
|
||||
# village) splits the postcode into a MultiPolygon -- one the map then draws as
|
||||
# several disconnected pieces. When subtraction disconnects a postcode, re-add
|
||||
# the postcode's OWN removed land along the narrowest necks (a morphological
|
||||
# closing clipped to the original footprint) so the parts stay joined by a thin
|
||||
# bridge. Parts left more than ~2x this width apart (a genuinely wide barrier)
|
||||
# stay split. Because the bridge is the postcode's own land, no address moves and
|
||||
# only a thin sliver of green is kept back.
|
||||
_RECONNECT_BRIDGE_M = 25.0
|
||||
|
||||
|
||||
def _reconnect_split(
|
||||
result: Polygon | MultiPolygon, postcode_geom: Polygon | MultiPolygon
|
||||
) -> Polygon | MultiPolygon:
|
||||
"""Re-join postcode parts that greenspace subtraction pulled apart by re-adding
|
||||
the narrow removed necks (within the original postcode), leaving wide barriers
|
||||
intact."""
|
||||
if result.geom_type != "MultiPolygon":
|
||||
return result
|
||||
closed = result.buffer(_RECONNECT_BRIDGE_M).buffer(-_RECONNECT_BRIDGE_M)
|
||||
if not closed.is_valid:
|
||||
closed = make_valid(closed)
|
||||
# The closing material that lies inside the original postcode but outside the
|
||||
# subtraction result == the thin green necks linking the parts. The exact
|
||||
# overlay path can leave line/point debris (coincident edges) that is
|
||||
# zero-area but NOT is_empty; `_poly_valid` strips it to polygons only, so the
|
||||
# is_empty guard works and the union can never return a GeometryCollection
|
||||
# (which `to_wgs84_geojson_multi` would silently truncate to a single part).
|
||||
bridges = _poly_valid(
|
||||
safe_difference(safe_intersection(closed, postcode_geom), result), _SNAP_GRID
|
||||
)
|
||||
if bridges.is_empty:
|
||||
return result
|
||||
return _poly_valid(safe_union([result, bridges]), _SNAP_GRID)
|
||||
|
||||
|
||||
def subtract_greenspace(
|
||||
postcode_geom: Polygon | MultiPolygon,
|
||||
|
|
@ -48,6 +84,10 @@ def subtract_greenspace(
|
|||
of intersecting greenspace from the postcode polygon. If subtraction
|
||||
would remove >90% of the area, keeps the original (the postcode
|
||||
genuinely covers that land, e.g. churchyards, riverside addresses).
|
||||
|
||||
If the subtraction disconnects the postcode (greenspace crossing it),
|
||||
:func:`_reconnect_split` re-adds the narrowest removed necks so the postcode
|
||||
stays a single piece rather than shipping as scattered fragments.
|
||||
"""
|
||||
candidate_idxs = tree.query(postcode_geom)
|
||||
if len(candidate_idxs) == 0:
|
||||
|
|
@ -74,4 +114,4 @@ def subtract_greenspace(
|
|||
if original_area > 0 and result.area / original_area < (1 - MAX_REMOVAL_FRACTION):
|
||||
return postcode_geom
|
||||
|
||||
return result
|
||||
return _reconnect_split(result, postcode_geom)
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
import json
|
||||
import math
|
||||
import shutil
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
|
|
@ -383,6 +384,164 @@ def _resolve_overlaps(
|
|||
return [(pc, out[i]) for i, (pc, _) in enumerate(items)]
|
||||
|
||||
|
||||
# A detached (non-largest) part of a postcode is absorbed into the neighbouring
|
||||
# postcode it shares the most boundary with, when the part is BOTH small in
|
||||
# absolute terms AND a minor fraction of its postcode. This removes the
|
||||
# Voronoi/INSPIRE/overlap-seam SCATTER that otherwise left ~1/3 of postcodes
|
||||
# non-contiguous (a single unit drawn as many disconnected specks), while keeping
|
||||
# genuine splits: a postcode bisected by a river/railway into two SUBSTANTIAL
|
||||
# parts has both parts above these thresholds and is left alone. The land is
|
||||
# REASSIGNED to an adjacent postcode wherever a neighbour exists, so coverage is
|
||||
# conserved and the output stays a partition; only a tiny neighbour-less sliver
|
||||
# (< _ELIM_DROP_BELOW_M2, snap/overlap debris floating in removed greenspace) is
|
||||
# dropped, and a larger isolated part (a genuine detached hamlet) is kept. The
|
||||
# largest part of every postcode is always retained, so no postcode is dropped.
|
||||
_ELIM_ABS_MAX_M2 = 3000.0 # absolute size below which a minor part reads as scatter
|
||||
_ELIM_FRAC_MAX = 0.15 # ...and only when it is < this fraction of the postcode
|
||||
_ELIM_DROP_BELOW_M2 = 200.0 # a neighbour-less sliver this small is snap debris -> drop
|
||||
# WGS84-degree distances at UK latitudes (1e-6 deg ~ 0.11 m at ~53N)
|
||||
_ELIM_SHARED_EPS_DEG = 5e-7 # ~0.05 m: thin probe whose overlap area ~ shared-edge length
|
||||
# Candidate-gather + nearest-neighbour fallback radius, in degrees so it needs no
|
||||
# per-part reprojection. The metric reach is mildly anisotropic (~2.2 m N-S,
|
||||
# ~1.3-1.4 m E-W at England's latitudes); this only bounds the rare gapped-seam
|
||||
# fallback (the dominant border-sharing path is unaffected), so the approximation
|
||||
# is harmless. Gather and accept use the SAME radius, so there is no dead band.
|
||||
_ELIM_NEAREST_MAX_DEG = 2e-5
|
||||
_M_PER_DEG_LAT = 111_320.0
|
||||
_ELIM_ITERATIONS = 2
|
||||
|
||||
|
||||
def _approx_area_m2(geom_deg) -> float:
|
||||
"""Metric area of a WGS84 polygon via a latitude-scaled planar approximation.
|
||||
|
||||
Accurate to a few percent at the few-hundred-to-few-thousand m^2 scale these
|
||||
thresholds work at, and far cheaper than a per-part CRS transform over the
|
||||
full ~1.8M postcodes.
|
||||
"""
|
||||
area_deg2 = geom_deg.area
|
||||
if area_deg2 == 0.0:
|
||||
return 0.0
|
||||
centroid = geom_deg.centroid
|
||||
if centroid.is_empty:
|
||||
lat = (geom_deg.bounds[1] + geom_deg.bounds[3]) / 2
|
||||
else:
|
||||
lat = centroid.y
|
||||
return area_deg2 * _M_PER_DEG_LAT * _M_PER_DEG_LAT * math.cos(math.radians(lat))
|
||||
|
||||
|
||||
def _geom_parts(geom):
|
||||
return list(geom.geoms) if geom.geom_type == "MultiPolygon" else [geom]
|
||||
|
||||
|
||||
def _eliminate_small_detached_parts(
|
||||
items: list[tuple[str, Polygon | MultiPolygon]],
|
||||
) -> list[tuple[str, Polygon | MultiPolygon]]:
|
||||
"""De-fragment the partition: absorb small detached parts into the neighbour
|
||||
they border most (see the module note above).
|
||||
|
||||
Runs on the de-overlapped WGS84 geometries as the last shaping step. The
|
||||
largest part of every postcode is always retained, so no active postcode is
|
||||
dropped; parts are moved between postcodes (or, for tiny neighbour-less
|
||||
slivers, dropped), so total covered area is conserved to within rounding.
|
||||
"""
|
||||
geoms: dict[str, Polygon | MultiPolygon] = {pc: g for pc, g in items}
|
||||
for _ in range(_ELIM_ITERATIONS):
|
||||
recs: list[tuple[str, Polygon, float]] = []
|
||||
total: dict[str, float] = defaultdict(float)
|
||||
largest: dict[str, float] = defaultdict(float)
|
||||
for pc, geom in geoms.items():
|
||||
for part in _geom_parts(geom):
|
||||
if part.is_empty:
|
||||
continue
|
||||
area = _approx_area_m2(part)
|
||||
recs.append((pc, part, area))
|
||||
total[pc] += area
|
||||
if area > largest[pc]:
|
||||
largest[pc] = area
|
||||
|
||||
tree = STRtree([part for _, part, _ in recs])
|
||||
assign: dict[int, str | None] = {}
|
||||
for i, (pc, part, area) in enumerate(recs):
|
||||
if area >= largest[pc]:
|
||||
continue # the largest part always stays -> a postcode never vanishes
|
||||
if not (area < _ELIM_ABS_MAX_M2 and area < _ELIM_FRAC_MAX * total[pc]):
|
||||
continue # substantial or major-fraction part = genuine split, keep
|
||||
|
||||
best_pc = None
|
||||
best_score = 0.0
|
||||
nearest_pc = None
|
||||
nearest_d = float("inf")
|
||||
# Gather candidates out to the nearest-fallback radius (not just the
|
||||
# smaller border-probe radius), so a gapped seam in the
|
||||
# (border, nearest] band can actually be reassigned rather than dropped.
|
||||
for j in tree.query(part.buffer(_ELIM_NEAREST_MAX_DEG)):
|
||||
if j == i:
|
||||
continue
|
||||
other_pc, other_geom, _ = recs[j]
|
||||
if other_pc == pc:
|
||||
continue
|
||||
try:
|
||||
score = part.buffer(_ELIM_SHARED_EPS_DEG).intersection(other_geom).area
|
||||
except GEOSException:
|
||||
score = 0.0
|
||||
# Ties broken by the lexicographically smaller postcode so the
|
||||
# result is independent of STRtree traversal order (matches the
|
||||
# stable tie-break in _resolve_overlaps; keeps output byte-stable
|
||||
# across shapely/GEOS upgrades for content-hash caching).
|
||||
if score > best_score or (
|
||||
score == best_score and best_pc is not None and other_pc < best_pc
|
||||
):
|
||||
best_score = score
|
||||
best_pc = other_pc
|
||||
dist = part.distance(other_geom)
|
||||
if dist < nearest_d or (
|
||||
dist == nearest_d
|
||||
and nearest_pc is not None
|
||||
and other_pc < nearest_pc
|
||||
):
|
||||
nearest_d = dist
|
||||
nearest_pc = other_pc
|
||||
|
||||
if best_pc is not None and best_score > 0:
|
||||
assign[i] = best_pc # shares a border -> absorb into that neighbour
|
||||
elif nearest_pc is not None and nearest_d <= _ELIM_NEAREST_MAX_DEG:
|
||||
assign[i] = nearest_pc # gapped seam -> nearest neighbour
|
||||
elif area < _ELIM_DROP_BELOW_M2:
|
||||
assign[i] = None # neighbour-less snap/overlap sliver -> drop
|
||||
# else: keep with its own postcode (a genuine isolated patch)
|
||||
|
||||
if not assign:
|
||||
break
|
||||
|
||||
new_parts: dict[str, list] = defaultdict(list)
|
||||
for i, (pc, part, _) in enumerate(recs):
|
||||
if i in assign:
|
||||
target = assign[i]
|
||||
if target is None:
|
||||
continue
|
||||
new_parts[target].append(part)
|
||||
else:
|
||||
new_parts[pc].append(part)
|
||||
|
||||
rebuilt: dict[str, Polygon | MultiPolygon] = {}
|
||||
for pc, parts in new_parts.items():
|
||||
if len(parts) == 1:
|
||||
rebuilt[pc] = parts[0]
|
||||
else:
|
||||
merged = safe_union(parts, grid=_OUTPUT_PRECISION_DEG)
|
||||
if not merged.is_empty:
|
||||
rebuilt[pc] = merged
|
||||
# Carry forward any postcode that contributed no parts to recs (e.g. an
|
||||
# all-empty input geometry, which the part loop skips): never drop a
|
||||
# postcode just because reassignment fired elsewhere in this pass.
|
||||
for pc, geom in geoms.items():
|
||||
if pc not in rebuilt:
|
||||
rebuilt[pc] = geom
|
||||
geoms = rebuilt
|
||||
|
||||
return list(geoms.items())
|
||||
|
||||
|
||||
def _round_coords(coords, ndigits=6):
|
||||
if coords and isinstance(coords[0], (int, float)):
|
||||
return [round(coords[0], ndigits), round(coords[1], ndigits)]
|
||||
|
|
@ -501,6 +660,11 @@ def write_district_geojson(
|
|||
# Remove overlap strips so the output is a clean partition.
|
||||
converted = _resolve_overlaps(converted)
|
||||
|
||||
# De-fragment: absorb small detached parts (Voronoi/INSPIRE/seam scatter) into
|
||||
# the neighbour they border most, so postcodes stop shipping as many
|
||||
# disconnected specks. Coverage-preserving; runs on the final partition.
|
||||
converted = _eliminate_small_detached_parts(converted)
|
||||
|
||||
by_district: dict[str, list[tuple[str, Polygon | MultiPolygon]]] = defaultdict(list)
|
||||
for pc, geom in converted:
|
||||
parts = pc.split()
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@ Each test targets a specific bug or edge case identified during code review.
|
|||
"""
|
||||
|
||||
import json
|
||||
import math
|
||||
|
||||
import numpy as np
|
||||
import polars as pl
|
||||
|
|
@ -21,6 +22,8 @@ from .inspire import build_inspire_index
|
|||
from .oa_boundaries import parse_gpkg_geometry
|
||||
from .greenspace import subtract_greenspace
|
||||
from .output import (
|
||||
_approx_area_m2,
|
||||
_eliminate_small_detached_parts,
|
||||
_fill_holes,
|
||||
merge_fragments,
|
||||
to_wgs84_geojson,
|
||||
|
|
@ -1830,3 +1833,231 @@ class TestFragmentsCache:
|
|||
cache.write_text("c")
|
||||
# arcgis is optional/absent — it cannot have invalidated the cache.
|
||||
assert fragments_cache_is_fresh(cache, [tmp_path / "absent.parquet"]) is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# De-fragmentation: small detached parts are absorbed into their best neighbour
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
# WGS84 helper: build a box from METRE offsets around a fixed England anchor, so
|
||||
# the eliminate pass (which works in WGS84 degrees and measures area with a
|
||||
# latitude-scaled approximation) sees realistic coordinates and areas.
|
||||
_ANCHOR_LON, _ANCHOR_LAT = -0.1, 51.5
|
||||
_M_PER_DEG_LAT = 111_320.0
|
||||
_M_PER_DEG_LON = 111_320.0 * math.cos(math.radians(_ANCHOR_LAT))
|
||||
|
||||
|
||||
def _mbox(x0, y0, x1, y1):
|
||||
return box(
|
||||
_ANCHOR_LON + x0 / _M_PER_DEG_LON,
|
||||
_ANCHOR_LAT + y0 / _M_PER_DEG_LAT,
|
||||
_ANCHOR_LON + x1 / _M_PER_DEG_LON,
|
||||
_ANCHOR_LAT + y1 / _M_PER_DEG_LAT,
|
||||
)
|
||||
|
||||
|
||||
def _as_dict(items):
|
||||
return dict(items)
|
||||
|
||||
|
||||
class TestEliminateSmallDetachedParts:
|
||||
"""A small detached part should be absorbed into the postcode it borders most,
|
||||
de-fragmenting the unit while conserving coverage and never dropping a
|
||||
postcode. Genuine large splits must survive."""
|
||||
|
||||
def test_small_island_absorbed_by_surrounding_neighbour(self):
|
||||
# A: a big main blob (left) plus a small 30x30m island sitting INSIDE B's
|
||||
# territory; B fills the middle/right with a hole where the island is.
|
||||
main_a = _mbox(0, 0, 100, 200) # 20000 m²
|
||||
island = _mbox(150, 90, 180, 120) # 900 m², surrounded by B
|
||||
a = MultiPolygon([main_a, island])
|
||||
b = _mbox(100, 0, 300, 200).difference(island) # hole around the island
|
||||
before = unary_union([a, b]).area
|
||||
|
||||
out = _as_dict(_eliminate_small_detached_parts([("A", a), ("B", b)]))
|
||||
|
||||
assert "A" in out and "B" in out, "no postcode may be dropped"
|
||||
assert out["A"].geom_type == "Polygon", "A's island should be absorbed away"
|
||||
# The island area moves to B (the surrounding postcode); coverage is kept.
|
||||
assert out["B"].contains(island.representative_point())
|
||||
after = unary_union([out["A"], out["B"]]).area
|
||||
assert after == pytest.approx(before, rel=1e-6), "coverage must be conserved"
|
||||
|
||||
def test_genuine_large_split_is_kept(self):
|
||||
# Two substantial parts (both 40000 m²) far apart — a real bisection, not
|
||||
# scatter. Neither is below the absolute/fraction thresholds, so both stay.
|
||||
a = MultiPolygon([_mbox(0, 0, 200, 200), _mbox(400, 0, 600, 200)])
|
||||
out = _as_dict(_eliminate_small_detached_parts([("A", a)]))
|
||||
assert out["A"].geom_type == "MultiPolygon"
|
||||
assert len(out["A"].geoms) == 2
|
||||
|
||||
def test_midsize_minor_part_kept_when_above_abs_threshold(self):
|
||||
# A big main plus a 10000 m² detached part: above the 3000 m² absolute
|
||||
# threshold, so it is a genuine secondary piece and must NOT be eliminated,
|
||||
# even though it is a small fraction of the postcode.
|
||||
a = MultiPolygon([_mbox(0, 0, 300, 300), _mbox(1000, 0, 1100, 100)])
|
||||
out = _as_dict(_eliminate_small_detached_parts([("A", a)]))
|
||||
assert out["A"].geom_type == "MultiPolygon"
|
||||
|
||||
def test_tiny_neighbourless_sliver_is_dropped(self):
|
||||
# A main blob plus a 10x10m (100 m²) sliver far from anything: below the
|
||||
# drop threshold with no neighbour to absorb it -> dropped as snap debris.
|
||||
a = MultiPolygon([_mbox(0, 0, 200, 200), _mbox(1000, 1000, 1010, 1010)])
|
||||
out = _as_dict(_eliminate_small_detached_parts([("A", a)]))
|
||||
assert "A" in out, "the postcode itself must survive"
|
||||
assert out["A"].geom_type == "Polygon"
|
||||
|
||||
def test_largest_part_always_retained_no_postcode_dropped(self):
|
||||
# Even a postcode that is ENTIRELY a tiny sliver keeps its (largest) part —
|
||||
# active postcodes must never be dropped (validate_outputs is zero-tolerance).
|
||||
tiny = _mbox(500, 500, 505, 505) # 25 m², the whole postcode
|
||||
big = _mbox(0, 0, 300, 300)
|
||||
out = _as_dict(
|
||||
_eliminate_small_detached_parts([("TINY", tiny), ("BIG", big)])
|
||||
)
|
||||
assert "TINY" in out and not out["TINY"].is_empty
|
||||
assert "BIG" in out
|
||||
|
||||
def test_no_overlaps_introduced(self):
|
||||
# Absorbing parts must keep the output a partition (no double-coverage).
|
||||
main_a = _mbox(0, 0, 100, 200)
|
||||
island = _mbox(150, 90, 180, 120)
|
||||
a = MultiPolygon([main_a, island])
|
||||
b = _mbox(100, 0, 300, 200).difference(island)
|
||||
out = _as_dict(_eliminate_small_detached_parts([("A", a), ("B", b)]))
|
||||
overlap = out["A"].intersection(out["B"]).area
|
||||
assert overlap < (1.0 / (_M_PER_DEG_LAT * _M_PER_DEG_LON)), "no overlap"
|
||||
|
||||
def test_empty_postcode_not_dropped_when_reassignment_fires(self):
|
||||
# Regression: an empty-geometry postcode must survive even when a
|
||||
# reassignment fires elsewhere in the same pass (the rebuild used to drop
|
||||
# any key absent from `recs`). No active postcode may ever be dropped.
|
||||
main_a = _mbox(0, 0, 100, 200)
|
||||
island = _mbox(150, 90, 180, 120) # B absorbs this -> reassignment fires
|
||||
a = MultiPolygon([main_a, island])
|
||||
b = _mbox(100, 0, 300, 200).difference(island)
|
||||
empty = Polygon() # degenerate input that contributes no parts
|
||||
out = _as_dict(
|
||||
_eliminate_small_detached_parts([("A", a), ("B", b), ("EMPTY", empty)])
|
||||
)
|
||||
assert "EMPTY" in out, "an empty-geometry postcode must not be dropped"
|
||||
assert "A" in out and "B" in out
|
||||
|
||||
def test_tiebreak_is_deterministic_smaller_postcode_wins(self):
|
||||
# An island bordered by TWO postcodes with EQUAL shared edges must go to
|
||||
# the lexicographically smaller postcode, independent of STRtree order.
|
||||
a_main = _mbox(0, 0, 100, 100) # A's main, far from the island
|
||||
island = _mbox(300, 50, 320, 70) # 400 m², A's detached part
|
||||
a = MultiPolygon([a_main, island])
|
||||
bbb = _mbox(250, 0, 300, 120) # borders island's left edge
|
||||
ccc = _mbox(320, 0, 370, 120) # borders island's right edge (equal)
|
||||
out = _as_dict(
|
||||
_eliminate_small_detached_parts([("A", a), ("BBB", bbb), ("CCC", ccc)])
|
||||
)
|
||||
pt = island.representative_point()
|
||||
assert out["BBB"].contains(pt), "tie must go to the smaller postcode string"
|
||||
assert not out["CCC"].contains(pt)
|
||||
assert out["A"].geom_type == "Polygon"
|
||||
|
||||
def test_gapped_sliver_within_nearest_radius_is_reassigned_not_dropped(self):
|
||||
# A 300 m² sliver 1.2 m from its only neighbour N — beyond the border probe
|
||||
# but inside the nearest-fallback radius. Before the gather buffer was
|
||||
# widened to the accept radius, the old (smaller) gather buffer never
|
||||
# returned N, so the sliver fell through to the drop branch. Now it is
|
||||
# reassigned to N, conserving coverage.
|
||||
x_main = _mbox(0, 0, 100, 100) # far from the sliver
|
||||
x_sliver = _mbox(278.8, 50, 298.8, 65) # 300 m², right edge at x=298.8
|
||||
x = MultiPolygon([x_main, x_sliver])
|
||||
n = _mbox(300, 0, 400, 200) # left edge at x=300 -> 1.2 m gap
|
||||
before = unary_union([x, n]).area
|
||||
out = _as_dict(_eliminate_small_detached_parts([("X", x), ("N", n)]))
|
||||
assert out["X"].geom_type == "Polygon", "sliver should leave X"
|
||||
assert out["N"].contains(x_sliver.representative_point()), "absorbed into N"
|
||||
after = unary_union([out["X"], out["N"]]).area
|
||||
assert after == pytest.approx(before, rel=1e-6), "coverage conserved, not dropped"
|
||||
|
||||
def test_approx_area_matches_real_metric_area(self):
|
||||
# The latitude-scaled area approximation should be within a few percent of
|
||||
# the true projected area for a building-scale polygon.
|
||||
import pyproj
|
||||
from shapely.ops import transform as transform_geometry
|
||||
|
||||
poly = _mbox(0, 0, 50, 60) # ~3000 m²
|
||||
to_bng = pyproj.Transformer.from_crs(
|
||||
"EPSG:4326", "EPSG:27700", always_xy=True
|
||||
)
|
||||
true_m2 = transform_geometry(to_bng.transform, poly).area
|
||||
assert _approx_area_m2(poly) == pytest.approx(true_m2, rel=0.02)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Greenspace subtraction is connectivity-preserving
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestGreenspaceReconnect:
|
||||
"""Greenspace that CROSSES a postcode must not shatter it: a narrow strip is
|
||||
bridged back (postcode stays one piece), a wide barrier is left subtracted
|
||||
(postcode genuinely splits)."""
|
||||
|
||||
def test_narrow_strip_reconnects_postcode(self):
|
||||
from shapely.strtree import STRtree
|
||||
|
||||
postcode = box(0, 0, 200, 100)
|
||||
strip = box(95, 0, 105, 100) # 10 m wide green strip crossing the postcode
|
||||
# Sanity: a plain difference WOULD split it into two parts.
|
||||
assert postcode.difference(strip).geom_type == "MultiPolygon"
|
||||
|
||||
result = subtract_greenspace(postcode, STRtree([strip]), [strip])
|
||||
assert result.geom_type == "Polygon", "narrow strip should be bridged"
|
||||
assert result.is_valid
|
||||
|
||||
def test_wide_barrier_keeps_postcode_split(self):
|
||||
from shapely.strtree import STRtree
|
||||
|
||||
postcode = box(0, 0, 200, 100)
|
||||
barrier = box(60, 0, 130, 100) # 70 m wide — beyond 2x the bridge width
|
||||
result = subtract_greenspace(postcode, STRtree([barrier]), [barrier])
|
||||
assert result.geom_type == "MultiPolygon", "wide barrier should stay split"
|
||||
assert len(result.geoms) == 2
|
||||
|
||||
def test_edge_greenspace_still_trimmed(self):
|
||||
# Greenspace on the edge (not crossing) is trimmed as before; reconnect is
|
||||
# a no-op because the result stays a single piece.
|
||||
from shapely.strtree import STRtree
|
||||
|
||||
postcode = box(0, 0, 100, 100) # 10000 m²
|
||||
park = box(60, 0, 100, 100) # 4000 m² on the right edge
|
||||
result = subtract_greenspace(postcode, STRtree([park]), [park])
|
||||
assert result.geom_type == "Polygon"
|
||||
assert result.area == pytest.approx(6000, rel=0.01)
|
||||
|
||||
def test_result_is_always_polygonal(self):
|
||||
# Regression: _reconnect_split must never return a GeometryCollection
|
||||
# (line/point debris) — downstream to_wgs84_geojson_multi would truncate a
|
||||
# GC to a single part, silently dropping substantial pieces. Sweep many
|
||||
# strip widths/offsets (incl. coincident-edge-prone integer geometries).
|
||||
from shapely.strtree import STRtree
|
||||
|
||||
postcode = box(0, 0, 200, 120)
|
||||
for w in (2, 5, 8, 10, 25, 49, 50, 51, 60, 90):
|
||||
for x0 in (40, 70, 95, 100, 130):
|
||||
strip = box(x0, -10, x0 + w, 130)
|
||||
result = subtract_greenspace(postcode, STRtree([strip]), [strip])
|
||||
assert result.geom_type in ("Polygon", "MultiPolygon"), (
|
||||
f"w={w} x0={x0} produced {result.geom_type}"
|
||||
)
|
||||
assert result.is_valid and not result.is_empty
|
||||
|
||||
def test_wide_barrier_preserves_all_substantial_parts(self):
|
||||
# The motivating case for the GC fix: a wide barrier genuinely splits the
|
||||
# postcode; ALL substantial parts must survive (not be truncated to one).
|
||||
from shapely.strtree import STRtree
|
||||
|
||||
postcode = box(0, 0, 300, 100)
|
||||
barrier = box(120, 0, 200, 100) # 80 m wide -> beyond 2x bridge
|
||||
result = subtract_greenspace(postcode, STRtree([barrier]), [barrier])
|
||||
assert result.geom_type == "MultiPolygon"
|
||||
areas = sorted(p.area for p in result.geoms)
|
||||
assert areas == pytest.approx([10000, 12000], rel=0.01) # both banks kept
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue