..

analyses/online_listings_buy.ipynb

@@ -5,9 +5,9 @@
    "id": "21f27a93",
    "metadata": {},
    "source": [
-    "# Online Buy Listings — data quality & cleanup\n",
+    "# Online Buy Listings: data quality & cleanup\n",
     "\n",
-    "Source: `finder/data/online_listings_buy.parquet` — ~112k UK *for-sale* property listings\n",
+    "Source: `finder/data/online_listings_buy.parquet`, ~112k UK *for-sale* property listings\n",
     "(Greater London) scraped from **Rightmove**, **OnTheMarket** and **Zoopla** and merged by the\n",
     "`finder` pipeline (`transform.py` / `onthemarket.py` / `zoopla.py` → `storage.py`).\n",
     "\n",
@@ -453,7 +453,7 @@
    "id": "7ea9293c",
    "metadata": {},
    "source": [
-    "### 3.1 · 🔴 Mislabeled column — `Number of bedrooms & living rooms` is actually `Bedrooms + Bathrooms`\n",
+    "### 3.1 · 🔴 Mislabeled column: `Number of bedrooms & living rooms` is actually `Bedrooms + Bathrooms`\n",
     "\n",
     "The column name promises *living rooms / receptions*, but `storage.py` recomputes it as\n",
     "`Bedrooms + Bathrooms` for **every** row of **every** portal (Zoopla's `+ receptions` line is\n",
@@ -529,7 +529,7 @@
     "### 3.2 · 🔴 Price integrity\n",
     "\n",
     "- **Shared-ownership / part-buy** listings store the *share* price as `Asking price`, not full value\n",
-    "  (median ~£146k vs ~£550k) — corrupts any price / £-per-sqm aggregate.\n",
+    "  (median ~£146k vs ~£550k). Corrupts any price / £-per-sqm aggregate.\n",
     "- **38% of rows carry a non-firm qualifier** (Guide / Offers Over / OIEO / From / Shared); `Asking price`\n",
     "  is stored identically regardless.\n",
     "- A `<£10k` tail of auction land/garage lots + two `£1` placeholders; 364 nulls (price ≤ 0 → null)."
@@ -657,7 +657,7 @@
     "\n",
     "- ~49% of `Total floor area (sqm)` is null (OnTheMarket ~87%).\n",
     "- **Impossible tiny areas**: rows under 20 m² with ≥2 bedrooms (single-room dimensions parsed as total).\n",
-    "- **Suspiciously large areas** (>400 m²) — likely sq ft never converted to m².\n",
+    "- **Suspiciously large areas** (>400 m²): likely sq ft never converted to m².\n",
     "- `Asking price per sqm` is mechanically correct but faithfully amplifies bad areas (max £410,959/m²)."
    ]
   },
@@ -755,7 +755,7 @@
     }
    ],
    "source": [
-    "# Floor area vs bedrooms — red points are physically impossible (area < beds * 8 m²)\n",
+    "# Floor area vs bedrooms: red points are physically impossible (area < beds * 8 m²)\n",
     "s = raw.filter(fa.is_not_null() & (fa < 160)).select(\"Bedrooms\", \"Total floor area (sqm)\")\n",
     "rng = np.random.default_rng(0)\n",
     "xb = s[\"Bedrooms\"].to_numpy().astype(float)\n",
@@ -946,7 +946,7 @@
     }
    ],
    "source": [
-    "# Listings by provider (downsampled) — shows London footprint + centroid clusters\n",
+    "# Listings by provider (downsampled): shows London footprint + centroid clusters\n",
     "g = raw.sample(min(25_000, raw.height), seed=0).select(\"lat\", \"lon\", \"provider\")\n",
     "colors = {\"Rightmove\": \"#2563eb\", \"OnTheMarket\": \"#16a34a\", \"Zoopla\": \"#dc2626\"}\n",
     "fig, ax = plt.subplots(figsize=(8, 7))\n",
@@ -971,7 +971,7 @@
     "\n",
     "No cross-source identity resolution: the same physical property appears across portals, and is\n",
     "re-listed multiple times within a portal (especially Zoopla). `UPRN` (Zoopla-only, 1.8% coverage)\n",
-    "is not 1:1 — 405 UPRNs repeat up to 7×, breaking the intended exact EPC join."
+    "is not 1:1. 405 UPRNs repeat up to 7×, breaking the intended exact EPC join."
    ]
   },
   {
@@ -1024,7 +1024,7 @@
     "- `Listing status` is a constant `\"For sale\"` (dead column).\n",
     "- `Property sub-type` has 79 values with portal-spelling variants (`Apartment`↔`Flat`, `X House`↔`X`, …).\n",
     "- `Price qualifier` has case-only duplicate pairs (Rightmove TitleCase vs OTM sentence-case).\n",
-    "- Missing values are encoded inconsistently — empty-string in some columns, `null` in others."
+    "- Missing values are encoded inconsistently: empty-string in some columns, `null` in others."
    ]
   },
   {
@@ -1155,7 +1155,7 @@
     "### 3.8 · 🔴/🟡 Listing date\n",
     "\n",
     "`Listing date` is **null for 100% of OnTheMarket & Zoopla** (Rightmove-only `firstVisibleDate`), so any\n",
-    "recency analysis is biased to 82% of the data — and it reaches back to **2011** in a live for-sale set."
+    "recency analysis is biased to 82% of the data, and it reaches back to **2011** in a live for-sale set."
    ]
   },
   {
@@ -1293,7 +1293,7 @@
    "id": "2491e545",
    "metadata": {},
    "source": [
-    "**Step A** — drop dead/redundant columns, normalise empty-string → null, canonicalise `Price qualifier`, derive `price_basis` + `is_shared_ownership`."
+    "**Step A**: drop dead/redundant columns, normalise empty-string → null, canonicalise `Price qualifier`, derive `price_basis` + `is_shared_ownership`."
    ]
   },
   {
@@ -1392,7 +1392,7 @@
    "id": "58767c84",
    "metadata": {},
    "source": [
-    "**Step B** — floor-area sanity (null impossibly-small, flag suspiciously-large) + recompute `Asking price per sqm` (excluding shared-ownership); null sentinel-`0` beds/baths for dwellings; canonicalise sub-types; add `is_residential`, `location_precision`; strip fake unit postcodes from outcode-only rows."
+    "**Step B**: floor-area sanity (null impossibly-small, flag suspiciously-large) + recompute `Asking price per sqm` (excluding shared-ownership); null sentinel-`0` beds/baths for dwellings; canonicalise sub-types; add `is_residential`, `location_precision`; strip fake unit postcodes from outcode-only rows."
    ]
   },
   {
@@ -1481,7 +1481,7 @@
    "id": "33e81cf5",
    "metadata": {},
    "source": [
-    "**Step C** — de-duplicate to one row per physical property. Heuristic key: `(lat, lon, Asking price, Bedrooms)`, keeping the most-recently-listed row. This collapses both cross-portal and intra-portal duplicates; `raw` is retained for provenance."
+    "**Step C**: de-duplicate to one row per physical property. Heuristic key: `(lat, lon, Asking price, Bedrooms)`, keeping the most-recently-listed row. This collapses both cross-portal and intra-portal duplicates; `raw` is retained for provenance."
    ]
   },
   {
@@ -1554,7 +1554,7 @@
    "id": "7d17dd05",
    "metadata": {},
    "source": [
-    "## 5 · After cleanup — re-show the stats"
+    "## 5 · After cleanup: re-show the stats"
    ]
   },
   {
@@ -1737,7 +1737,7 @@
    "id": "0a16d168",
    "metadata": {},
    "source": [
-    "**Missingness after cleanup** — beds/baths now legitimately null where unknown; empty-strings gone; per-sqm follows cleaned area:"
+    "**Missingness after cleanup**: beds/baths now legitimately null where unknown; empty-strings gone; per-sqm follows cleaned area:"
    ]
   },
   {
@@ -1924,7 +1924,7 @@
    "id": "e833c578",
    "metadata": {},
    "source": [
-    "**Categoricals normalised** — sub-type variants collapsed, qualifier casing merged:"
+    "**Categoricals normalised**: sub-type variants collapsed, qualifier casing merged:"
    ]
   },
   {
@@ -2007,7 +2007,7 @@
    "id": "94ac0585",
    "metadata": {},
    "source": [
-    "**Before/after — key metrics chart:**"
+    "**Before/after key metrics chart:**"
    ]
   },
   {
@@ -2076,7 +2076,7 @@
     "\n",
     "- **`clean` keeps every row** (issues are flagged, not dropped) so it stays comparable to `raw`; use the\n",
     "  `is_residential` / `is_shared_ownership` / `floor_area_suspect_*` / `location_precision` flags to filter.\n",
-    "- **Shared-ownership full prices cannot be recovered** here — those rows are flagged and excluded from\n",
+    "- **Shared-ownership full prices cannot be recovered** here: those rows are flagged and excluded from\n",
     "  `£/sqm`, but their `Asking price` is still a share. Likewise **suspiciously-large areas are flagged,\n",
     "  not auto-converted** from sq ft (the conversion factor isn't certain per-row).\n",
     "- **De-dup is heuristic** (`lat,lon,price,beds`); it can over-collapse distinct units sharing a\n",