andras/perfect-postcode
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages 2 Wiki Activity Actions

..

Browse source
This commit is contained in:
Andras Schmelczer 2026-07-03 18:39:34 +01:00
parent 1ee796b282
commit ab688243d7
36 changed files with 307 additions and 135 deletions
Download patch file Download diff file Expand all files Collapse all files

6
pipeline/check_travel_times.py
View file

@ -5,11 +5,11 @@ computation failed or was interrupted, leaving either zero rows or only
the origin postcode. We detect this by an absolute, structural criterion:
a file is corrupt only when it is unreadable or has a row count at or below
CORRUPT_ROW_FLOOR. Per-mode percentile/median/range figures are reported
for context only — they never drive the deletable set, so repeated runs
for context only. They never drive the deletable set, so repeated runs
(including with --delete) are idempotent and never erode legitimate
small-catchment (rural/island) origins.
Duplicates arise when places.parquet is rebuilt between R5 runs — each
Duplicates arise when places.parquet is rebuilt between R5 runs. Each
place gets a new numeric index prefix, so the skip-completed logic
doesn't recognize previous results. --dedup keeps only the largest
file per slug and removes the rest.
@ -105,7 +105,7 @@ def find_bad_files(base_dir: Path) -> tuple[list[BadFile], dict[str, dict]]:
if not row_counts:
continue
# Reporting statistics only — these never decide what gets deleted.
# Reporting statistics only; these never decide what gets deleted.
p5 = percentile(row_counts, 5)
median = percentile(row_counts, 50)