4.4 KiB
| title | description | date | period | thumbnail | links | article | project | ||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| The Sync Protocol I Designed and the One I Shipped | A 2019 goal tracker. I designed delta sync over an immutable trie; what shipped was the whole tree plus a version counter, and that was the better idea. | 2026-05-05 | August-September 2019 |
|
|
|
|
In August 2019 I wanted a goal tracker I'd actually open, on whichever device was nearest, without it disagreeing with itself. Nothing off the shelf fit, so I built one over a couple of weekends: an Angular frontend where every goal is a tower, and every finished task is a block that falls into it with a little gravity animation. A block's difficulty (1 to 100) decides how many tiles it adds, six blocks to a row, so a productive month literally raises the skyline. Friends saw the towers. I was more attached to what was underneath.
The clever part
The state (pages of towers of blocks) is a tree, and I stored it as an immutable one. The commit that introduced Root, InnerNode, and Node landed at the end of the first month. Updates produced new structure and shared everything untouched by reference, so "where I was" and "where I am" were two pointers, and the difference between them could be walked in time proportional to the change, not the state. I had the whole plan in my head: diff the roots, ship one tiny op per changed leaf, rebase in-flight edits on top of whatever came back. A delta-sync protocol where the data structure does the hard part.
The part that shipped
What the client actually sends, to this day, is the whole tree.
PUT /api/v1/data carries every page, tower, and block, with an If-Match header holding the last revision number this device saw. If the server's counter has moved on, the request gets a 409 and the client refetches and adopts the server's version, discarding its own un-pushed edit. A server-sent-events stream pushes new revision numbers so other devices know to refetch; a 750 ms debounce keeps typing from becoming a PUT storm.
I never built the delta protocol, and the honest reason isn't that I ran out of time. The numbers refused to justify it. A person's life plans are a few kilobytes of JSON; shipping all of them costs nothing perceptible, while the diff machinery would have saved bytes nobody was paying for and added a rebase layer that is exactly where the subtle bugs would have lived. Even server-wins, which sounds brutal, is close to the right semantics here, because the only concurrent writer is me on another device, usually hours apart.
When I rewrote the backend this spring (FastAPI and SQLite now, and the frontend moved to Angular signals), I kept the protocol exactly as it was. That's the strongest endorsement I know how to give it.
What the trie was actually for
The immutability wasn't wasted; it just never needed to cross the wire. Inside the client it made the everyday things simple: undo is keeping an old pointer, change detection is reference equality, and nothing ever observes a half-applied update. The structure earned its keep locally and stayed an implementation detail, which is a perfectly good career for a data structure.
The wish for a real merge, where two devices edit the same thing and both edits survive, didn't die either. It turned out to belong to a different problem. Years later it became reconcile-text, where the data is prose and losing an edit actually hurts.
What I'd change
- Tell the loser. Server-wins is a fine policy; silent server-wins is rude. The device whose edit got discarded should at least say so out loud.
- Write the protocol down at the time. I spent years remembering this as "the delta-sync project", because that was the design in my head. The code disagreed. One README paragraph written in 2019 would have kept my memory honest.