andras/schmelczer-dev
1
0
Fork 0
Code Issues Pull requests 6 Projects Releases Packages Wiki Activity Actions

Fable clean up

Browse source
This commit is contained in:
Andras Schmelczer 2026-06-11 21:35:45 +01:00
parent 3441a7e4af
commit 4ce8a4f41d
46 changed files with 642 additions and 911 deletions
Download patch file Download diff file Expand all files Collapse all files

49
src/content/work/life-towers-immutable-tries.md
View file

@ -1,6 +1,6 @@
---
title: Syncing State with an Immutable Trie
description: 'A visual goal tracker whose lasting idea was the sync model: an immutable trie so structural diffs are trivial and only deltas cross the wire.'
title: The Sync Protocol I Designed and the One I Shipped
description: 'A 2019 goal tracker. I designed delta sync over an immutable trie; what shipped was the whole tree plus a version counter, and that was the better idea.'
date: 2026-05-05
period: 'August-September 2019'
thumbnail:
@ -13,52 +13,45 @@ article:
featuredOrder: 4
tags: ['systems', 'web', 'tools']
role: Full-stack author
stack: ['Python', 'Angular', 'TypeScript', 'Custom sync protocol']
stack: ['Python', 'Angular', 'TypeScript', 'FastAPI', 'SQLite']
scale: Multi-device goal and task state shared between clients and a server
outcome: A working sync protocol where structural sharing made the delta tiny
outcome: A goal tracker still in use, and a lesson about when clever sync isn't worth it
audience: recruiter-relevant
media:
- type: image
src: ./_assets/towers.jpg
alt: Screenshot of a life tracking web interface represented with tower-like visual structures.
caption: The interface was a 2019 weekend experiment. The trie underneath aged better.
caption: Towers of finished tasks, one column per goal. Done blocks fall into place with a small gravity animation.
project:
title: Life Towers
description: A multi-device goal tracker. The trie underneath made the sync diff free; the towers were just the UI.
description: A multi-device goal tracker where the clever trie lost to send-the-whole-tree with a version counter. The towers were the UI; the protocol was the lesson.
selected: true
technologies: ['Python', 'Angular', 'TypeScript', 'Immutable trie']
technologies: ['Python', 'Angular', 'TypeScript', 'Immutable trees']
---
In August 2019 I wanted a goal tracker I'd actually open, on whichever device was nearest, without watching it disagree with itself. Nothing off the shelf fit, so I built one over a couple of weekends. The tower metaphor was the part friends saw; the part that aged well was the sync model that fell out of needing the same state in three places at once.
In August 2019 I wanted a goal tracker I'd actually open, on whichever device was nearest, without it disagreeing with itself. Nothing off the shelf fit, so I built one over a couple of weekends: an Angular frontend where every goal is a tower, and every finished task is a block that falls into it with a little gravity animation. A block's difficulty (1 to 100) decides how many tiles it adds, six blocks to a row, so a productive month literally raises the skyline. Friends saw the towers. I was more attached to what was underneath.
## The problem in one paragraph
## The clever part
Pick any non-trivial mutable object graph, sync it across devices, and you end up either sending the whole thing on every change (wasteful) or writing ad-hoc diff logic per shape (brittle). I wanted a representation where the _shape_ of the data made the diff fall out for free.
The state (pages of towers of blocks) is a tree, and I stored it as an immutable one. The commit that introduced `Root`, `InnerNode`, and `Node` landed at the end of the first month. Updates produced new structure and shared everything untouched by reference, so "where I was" and "where I am" were two pointers, and the difference between them could be walked in time proportional to the change, not the state. I had the whole plan in my head: diff the roots, ship one tiny op per changed leaf, rebase in-flight edits on top of whatever came back. A delta-sync protocol where the data structure does the hard part.
## The trie, concretely
## The part that shipped
A goal in Life Towers is a path of strings. `Health / Running / 5k`. Tasks under a goal hang off the leaf. A user's whole state is a tree, and a trie is exactly the data structure that makes that tree's _identity_ manipulable.
What the client actually sends, to this day, is the whole tree.
Two properties did the heavy lifting:
`PUT /api/v1/data` carries every page, tower, and block, with an `If-Match` header holding the last revision number this device saw. If the server's counter has moved on, the request gets a 409 and the client refetches and adopts the server's version, discarding its own un-pushed edit. A server-sent-events stream pushes new revision numbers so other devices know to refetch; a 750 ms debounce keeps typing from becoming a PUT storm.
- **Structural sharing.** When you tick off a task under `Health / Running / 5k`, the new root reuses every untouched subtree by reference. The `Career` branch and the `Reading` branch are the same objects they were before. Comparing the old and new roots is mostly pointer equality; only the path that actually changed gets walked.
- **Immutability.** Updates produce new structure instead of mutating. "Where I was" and "where I am" become two pointers, not two snapshots. The diff between them is whatever's not shared, and that walk is O(changes), not O(state).
I never built the delta protocol, and the honest reason isn't that I ran out of time. The numbers refused to justify it. A person's life plans are a few kilobytes of JSON; shipping all of them costs nothing perceptible, while the diff machinery would have saved bytes nobody was paying for and added a rebase layer that is exactly where the subtle bugs would have lived. Even server-wins, which sounds brutal, is close to the right semantics here, because the only concurrent writer is me on another device, usually hours apart.
The sync loop falls out:
When I rewrote the backend this spring (FastAPI and SQLite now, and the frontend moved to Angular signals), I kept the protocol exactly as it was. That's the strongest endorsement I know how to give it.
1. Client holds the last root the server acknowledged plus its own current root.
2. To send: walk only the unshared paths, emit one op per changed leaf. In practice that's a handful of bytes for a typical edit, no matter how large the rest of the tree is.
3. Server applies, returns its new root.
4. Client rebases any in-flight edits by replaying them on top.
## What the trie was actually for
There's no conflict resolution layer because the operations commute on the structure. Two clients adding tasks under different branches produce non-overlapping deltas that compose trivially. The hard cases (two clients editing the same leaf) are tiny and obvious, because they're the _only_ place the deltas touch the same path.
The immutability wasn't wasted; it just never needed to cross the wire. Inside the client it made the everyday things simple: undo is keeping an old pointer, change detection is reference equality, and nothing ever observes a half-applied update. The structure earned its keep locally and stayed an implementation detail, which is a perfectly good career for a data structure.
The wish for a real merge, where two devices edit the same thing and both edits survive, didn't die either. It turned out to belong to a different problem. Years later it became [reconcile-text](/articles/reconcile-text-3-way-merge/), where the data is prose and losing an edit actually hurts.
## What I'd change
- **Property tests around the rebase.** The reconcile path is exactly where a generator finds bugs that hand-written tests never think to write. I had hand-written cases; I'd start with `proptest` now.
- **A standalone spec for the wire format.** The part worth lifting out was the protocol, not the goal tracker. A short spec would let me (or anyone) reimplement it in a different stack without re-deriving everything from the Python source.
- **Strip the visual experiment.** The tower visualisation was fun but it bound the storage to a UI metaphor. The sync model should be a library; the towers should be a separate toy.
## If you take one idea from this
Most sync problems are diff problems pretending to be transport problems. Pick the data structure that makes the diff free, and the protocol almost writes itself. The corollary: if you're writing a lot of "if this changed, send that" code, you're using the wrong structure.
- **Tell the loser.** Server-wins is a fine policy; silent server-wins is rude. The device whose edit got discarded should at least say so out loud.
- **Write the protocol down at the time.** I spent years remembering this as "the delta-sync project", because that was the design in my head. The code disagreed. One README paragraph written in 2019 would have kept my memory honest.