---
title: An Obsidian Sync Built Around the Merger I Already Had
description: 'VaultLink: self-hosted Obsidian sync. Edit in any editor, online or off, then come back to a converged vault. The app that justified reconcile-text.'
date: 2026-05-30
period: '2025-2026'
thumbnail:
  src: ./_assets/vault-link.svg
  alt: 'The VaultLink logo: a chain-link mark in a soft gradient.'
links:
  - label: Source
    url: https://github.com/schmelczer/vault-link
  - label: Docs
    url: https://vault-link.schmelczer.dev
article:
  tags: ['systems', 'web', 'tools']
  role: Sync engine and server author
  stack:
    [
      'Rust',
      'axum',
      'sqlx',
      'SQLite',
      'WebSockets',
      'TypeScript',
      'Obsidian plugin',
      'ts-rs',
      'wasm-bindgen',
      'reconcile-text',
    ]
  scale: One Rust server, one TypeScript sync engine, three published consumers (Obsidian plugin, CLI, fuzz/deterministic test harnesses)
  outcome: A self-hosted Obsidian sync I trust enough to use as my primary vault transport
  audience: technical
project:
  title: VaultLink
  description: 'I refuse to give up the editor: Obsidian, Vim, VS Code, sed. Self-hosted sync that survives all four, built on reconcile-text underneath.'
  selected: true
---

I refuse to give up my editors. Obsidian on the phone, Vim on the laptop, VS Code at work, the occasional headless `sed` across the whole vault. None of them know about each other, none of them are going to learn to, and I'm not switching to whichever sync product picks a favourite. VaultLink is what that refusal costs: one Rust server, one TypeScript sync engine, an Obsidian plugin, a CLI, and two test harnesses. The merge primitive underneath everything is [reconcile-text](/articles/reconcile-text-3-way-merge/), which I had written first, on the bet that a question like this would eventually ask for it.

## The constraint that picks the algorithm

The consequence of that refusal is that the server never sees keystrokes. It sees end states: a file as it stood when sync caught it. That kills CRDTs (which need every operation) and OT-as-it's-usually-implemented (same). It leaves you with one primitive: 3-way merge given a parent, a left, and a right. Which is reconcile-text. Which I'd written exactly because no existing tool took three independently-edited file states and gave one back.

The other consequence is that the _path placement_ is its own problem. Two clients might both move the same file. A file might land on a slot another file already occupies. A rename and a content edit might race. That's the part I underestimated.

## Two loops, separate invariants

The sync engine is two loops, deliberately disentangled:

- **Wire loop** (`syncer.ts`). Drains the single-consumer FIFO of pending HTTP and WebSocket ops. Updates a document's record fields (`remoteRelativePath`, `parentVersionId`, `remoteHash`) and writes content to whatever path the record currently holds. _Never moves files for path placement._
- **Path reconciler** (`reconciler.ts`). Runs after every drained event. Best-effort pass that moves files on disk so `localPath === remoteRelativePath`. The move graph is topologically sorted. Records with pending local events are skipped; the reconciler only operates on settled ones. Failures (slot occupied by something untracked) are silent skips; the next pass retries.

This split is the decision the rest of the engine leans on. It used to be one loop with both responsibilities, and the bug catalogue from that era reads like a confession: slot-collision stashes, "conflict-uuid" hacks, `MoveOnConflict.NEW`/`EXISTING` policy enums. Separating wire transport from path placement made most of it vanish, because the wire loop can now freely write `remoteRelativePath` to whatever the server returned, even when it disagrees with the file on disk, knowing the reconciler will never move anything out from under a queued user rename.

Cycles in the move graph (A→B, B→C, C→A) are found with Tarjan's strongly-connected-components pass and resolved by reading every file in the cycle into memory and writing each back to its new slot; no tmp files. A write-ahead marker at `.vaultlink/swap-<uuid>.json` lists each leg along with the hash the source file is expected to have, so after a crash the reconciler can hash each `from`, work out which legs already ran, and replay only the rest. `.vaultlink/**` is hardcoded into the internal ignore pattern so the swap markers never themselves get synced.

## Pending creates are Promises, not strings

When the user creates a file locally and _then_ immediately edits or renames it before the create has been acknowledged, the engine doesn't know the document's id yet; the server assigns it. So queued events for that doc carry a `Promise<DocumentId>` in their `documentId` slot, threaded back to the still-in-flight `LocalCreate`. When the server acks the create, `resolveCreate` fulfils the promise and `replacePendingDocumentId` walks the queue swapping the resolved string into every dependent event.

If you're walking `events[]` and comparing docIds with `===`, you'll silently fail to match until the swap happens. There's a comment in `sync-event-queue.ts` that warns about exactly that, in slightly more alarmed punctuation. The shape is unusual but the alternative (synchronously waiting for the create ack before letting the user type more) is the kind of thing that makes a notes app feel like a 1998 webform.

## MinCovered: the watermark that doesn't lie

The catch-up handshake says "give me everything newer than `lastSeenUpdateId`." If the client advances that id as it receives a stream of RemoteChange ids out of order, it'll publish a too-high cursor, and the next reconnect will request from a point past events it never actually applied. Permanent gap. Replay-forever bug, with extra steps.

The fix is a small data structure called `MinCovered`: a contiguous-prefix tracker over a stream of integers. It advances the public minimum only when the next consecutive id has been processed; out-of-order arrivals stash without bumping the cursor. The implementation is one screen of code, the test file is slightly longer than the implementation, and an entire category of confusing data-loss bugs stopped existing.

## reconcile-text on the server

The merge sits on the server. When two clients submit edits against the same `parent_version_id`, the second submission triggers a 3-way merge against the parent and the freshly-committed first edit. Three strings in, one out. No conflict markers. The engine commits the merged result, increments the version, and broadcasts the new state to every connected client.

Two restrictions, neither hidden:

- **Only `.md` and `.txt` get merged.** Markdown that fails UTF-8 validation is treated as binary, same as PNGs and PDFs.
- **Last-write-wins for everything else.** Concurrent edits to a `.docx` lose one of the writes. The real fix is "don't edit binaries concurrently", which is unsatisfying and true.

Merge quality is exactly what reconcile-text gives me. Word-level tokenisation turns most prose conflicts into two adjacent edits that coexist. If the merge looks slightly clumsy now and then, the alternative is a `<<<<<<< HEAD` block in my notes, and I'd take the clumsy sentence every time.

## Two test harnesses, one workflow

A distributed-sync bug is bewildering the first time you see it and gone by the time you've attached a debugger. My answer is two harnesses with different jobs:

- **`test-client` (fuzz).** N parallel processes hammering random operations against a shared server for minutes at a time. It catches the bugs nobody would have thought to write a test for, and its reproductions are unreadable.
- **`deterministic-tests`.** Scripted multi-client scenarios, over a hundred of them by now, written in a small step grammar (`pause-server`, `pause-websocket`, `barrier`, `assert-consistent`) and run with an in-memory filesystem against the real server binary.

The workflow connects them: the fuzzer finds something, I sift the logs for a root cause, write the smallest deterministic scenario that fails the same way, then fix until both stay green. Without the second harness, every fix would be an act of faith.

## Smaller calls

- **TS types are generated from Rust via `ts-rs`.** The HTTP/WS API has one source of truth: the Serde types in the server. `scripts/update-api-types.sh` re-emits `frontend/sync-client/src/services/types/`. Hand-edits to those files are explicitly banned.
- **`sqlx::query!` macros over a checked-in `.sqlx` cache.** SQL is verified against the schema at compile time. Touching SQL means re-running `cargo sqlx prepare --workspace`; if you forget, CI catches it.
- **One sync engine, four consumers.** `sync-client` is the engine. Obsidian plugin, standalone CLI, fuzz harness, and deterministic harness all depend on it via `file:../sync-client`. Bugs are fixed once and inherited everywhere.
- **`record.localPath` mutates in place across awaits.** The watcher can rename a doc while a wire-loop handler is mid-HTTP. Snapshotting `localPath` into a local at function entry and reading it after the await reads a vacated slot. Read it live; only snapshot when you deliberately want to compare _before_ and _after_ the await.
- **Watermark advancement can fail in both directions.** A branch that skips a remote event without advancing `lastSeenUpdateId` creates a permanent gap that re-delivers forever; a branch that advances without applying the content loses data. The rule that survives code review: advance only if you applied the event or deliberately discarded it.

## The race I haven't structurally fixed

Pause-or-disable-sync mid-flight is the one left. An HTTP that committed server-side but whose response was dropped leaves the server holding a doc the client never recorded. On resume, the offline scan finds the file again, uploads it as a new create, and server-side dedupe merges the duplicate into the existing doc. If the merge produces a deconflict file (two real divergences), the user picks up an extra file in their vault. Not data loss, but a small ugliness.

The two-loop split doesn't fix this and probably shouldn't. The honest path is something like a persisted client-side "have I acked this op?" log, sitting in the same SQLite the engine already uses. It's on my list, below several things I want more.

## What I'd change

- **Move the merge to the client.** Right now reconcile-text runs on the server. Putting it in the WASM build of reconcile-text on each client, and letting the server be a dumb commit log, would let the merge benefit from device-specific tokenisers (Markdown-aware on the desktop, word-level on mobile). It would also stop the server from needing to understand the file format at all.
- **Property tests for the move graph.** The cycle resolver is the part I trust least under crash. Snapshot tests can't go where proptest can; I should be generating arbitrary move-graph + interruption combinations.
- **A first-class "pause" with a write-ahead op log.** See above.
- **More than `.md` and `.txt`.** A canvas-aware merge for Obsidian's `.canvas` files is one reconcile-text tokeniser away. Not because anyone asked, but because the asymmetry annoys me.

The way I think about VaultLink now: reconcile-text was the bet, and VaultLink is what I built once the bet looked like paying off. The bet said three independently edited files can become one without anyone telling the system about the keystrokes that produced them. What building the application taught me is that the merge is the easy half; the real work is everything around it, arranged so the rest of the system can't undo what the merge got right.
