andras/perfect-postcode
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages 2 Wiki Activity Actions

ok

Browse source
This commit is contained in:
Andras Schmelczer 2026-07-03 18:01:10 +01:00
parent c2070693fb
commit 909e241907
55 changed files with 594 additions and 223 deletions
Download patch file Download diff file Expand all files Collapse all files

4
finder/postcode_cache.py
View file

@ -2,7 +2,7 @@
Each portal recovers a listing's true postcode (Rightmove/OnTheMarket) or full
geo dict (Zoopla) from its detail page. That value never changes for a given
listing id, yet the in-memory caches are discarded at the end of every run — so
listing id, yet the in-memory caches are discarded at the end of every run, so
each run re-fetches every listing's detail page from scratch. Persisting the
cache to disk means a steady-state run only fetches NEWLY-appeared listings,
typically a small fraction of the market, which is the single biggest saving
@ -27,7 +27,7 @@ log = logging.getLogger("rightmove")
def load_cache(path: str | Path) -> dict:
"""Load a persisted detail cache. Returns ``{}`` when absent or unreadable.
A corrupt or non-object file is treated as empty rather than fatal — a bad
A corrupt or non-object file is treated as empty rather than fatal: a bad
cache must never block a scrape; the worst case is re-fetching details."""
p = Path(path)
if not p.exists():