ai

frontend/sync-client/ARCHITECTURE.md

@@ -0,0 +1,197 @@
+# Sync Client Architecture
+
+## Overview
+
+The sync client synchronizes Obsidian vault files between clients via a central server. It handles offline edits, concurrent multi-client changes, crash recovery, and real-time updates via WebSocket.
+
+## Architecture Layers
+
+```
+SyncClient (public API — unchanged)
+    │
+    ├── Syncer (event router + reconciliation orchestrator)
+    │       │
+    │       ├── SyncEventQueue (per-document coalescing FIFO)
+    │       │       │
+    │       │       └── executor callback → sync-actions functions
+    │       │
+    │       └── VirtualFilesystem (document identity + state tracking)
+    │
+    ├── WebSocketManager (connection, message serialization)
+    ├── FileOperations (filesystem abstraction, 3-way merge on write)
+    ├── SyncService (HTTP client for server REST API)
+    ├── CursorTracker (collaborative cursor positions)
+    └── ContentCache (LRU cache for diff computation)
+```
+
+## Key Design Decisions
+
+### 1. Sequential Processing
+
+All sync operations run one at a time. No concurrent sync operations, no locks, no deadlock prevention, no generation counters. The server uses SQLite which serializes writes anyway, so client-side parallelism provided no real benefit while creating enormous complexity.
+
+The only concurrency that remains is between the sync queue and the WebSocket message handler, but both funnel events into the same sequential queue.
+
+### 2. Virtual Filesystem (VFS)
+
+Replaces the old `Database` class. Documents have explicit states as a discriminated union:
+
+```typescript
+type VirtualDocument =
+    | PendingDocument       // Created locally, server doesn't know yet
+    | TrackedDocument       // Synced with server
+    | DeletedLocallyDocument // Deleted locally, server not yet notified
+```
+
+Three internal indexes replace the old flat array + `parallelVersion` system:
+- `pathIndex` — at most one live document per path
+- `documentIdIndex` — all documents with a server-assigned ID
+- `idempotencyKeyIndex` — pending documents only
+
+No more inferring state from `metadata === undefined` + `isDeleted` + `parentVersionId === 0`. The state field is the discriminator.
+
+### 3. Per-Document Event Coalescing
+
+Events from file watchers and WebSocket broadcasts are grouped by document identity and coalesced:
+- 10 rapid edits → 1 sync operation (content read at execution time)
+- create then delete → noop (file never reached the server)
+- move A→B then B→C → move A→C
+
+This replaces the old opaque-closure FIFO where every event was independent.
+
+### 4. Server Protocol Unchanged
+
+The server still does 3-way merging via `reconcile-text`. Response types (`FastForwardUpdate`, `MergingUpdate`) are unchanged. The client content cache remains for diff computation (needed for mobile bandwidth). Idempotency keys remain for crash-safe creates. No server changes were made.
+
+## Module Descriptions
+
+### `persistence/vfs.ts` — Virtual Filesystem
+
+Tracks document identity across creates, moves, deletes. Provides:
+- State transitions: `createPending()`, `confirmCreate()`, `assignDocumentId()`, `updateTracked()`, `deleteLocally()`, `confirmDelete()`
+- Queries: `getByPath()`, `getByDocumentId()`, `getByIdempotencyKey()`
+- Disk reconciliation: `reconcileWithDisk()` returns a pure result comparing VFS state against filesystem
+- Persistence: serializes to `StoredDatabase` format (backward compatible)
+
+### `sync-operations/sync-events.ts` — Event Types + Coalescing
+
+Pure functions with no side effects. Defines `SyncEvent` (6 types from file watchers and WebSocket) and `CoalescedAction` (8 possible merged actions). The `coalesce()` function implements a 48-entry transition table.
+
+### `sync-operations/sync-event-queue.ts` — Event Queue
+
+Per-document coalescing FIFO. Maps each event to a document key (documentId for tracked docs, `path:<relativePath>` for pending docs). Processes one document at a time via an injected executor. Supports key migration when pending docs receive a documentId.
+
+On reset (WebSocket disconnect): remote events are cleared (server replays on reconnect), local events are preserved (unsynced user actions).
+
+### `sync-operations/sync-actions.ts` — Sync Action Implementations
+
+Extracted from the old `unrestricted-syncer.ts`. Each function takes explicit dependencies (`SyncDeps`) and a VFS document:
+
+- `executeSyncCreate()` — POST to server with idempotencyKey, handle response
+- `executeSyncUpdate()` — compute diff from cache, PUT to server
+- `executeSyncDelete()` — DELETE on server, confirm in VFS
+- `executeRemoteUpdate()` — download content, write to disk, update VFS
+- `applyServerResponse()` — handle MergingUpdate/FastForwardUpdate, path changes, idempotent returns
+
+### `sync-operations/syncer.ts` — Orchestrator
+
+Thin layer that:
+- Converts file change events → `SyncEvent` objects → enqueue
+- Converts WebSocket broadcasts → `SyncEvent` objects → enqueue
+- Sets up the executor that dispatches `CoalescedAction` → sync-actions functions
+- Runs offline reconciliation: resolve idempotency keys → scan filesystem → enqueue results
+- Manages the `scheduleSyncForOfflineChanges` lifecycle
+
+## Offline Reconciliation Algorithm
+
+Runs on startup and WebSocket reconnect:
+
+1. **Resolve idempotency keys** — call server for pending creates whose responses were lost
+2. **Clean up orphans** — remove pending docs whose files no longer exist
+3. **Scan filesystem** — `vfs.reconcileWithDisk()` compares VFS state vs actual files
+4. **Apply moves** — update VFS for detected file moves (content hash matching)
+5. **Enqueue events** in order:
+   - Interrupted deletes (VFS says deleted-locally, file gone, server not notified)
+   - Moves (detected via hash matching)
+   - Updates (file content changed)
+   - Creates (new files with no VFS entry)
+   - Delete candidates (VFS entry but file missing, not matched as a move)
+
+Creates run before delete candidates so the server can merge creates with existing documents (preserving documentIds).
+
+## Document Lifecycle
+
+```
+[File created locally]
+    → VFS: createPending(path) → PendingDocument
+    → Queue: enqueue local-create
+    → Action: POST /documents with idempotencyKey
+    → Server: returns FastForwardUpdate or MergingUpdate
+    → VFS: confirmCreate() → TrackedDocument
+
+[File edited locally]
+    → Queue: enqueue local-update
+    → Action: compute diff, PUT /documents/:id/text
+    → Server: returns FastForwardUpdate or MergingUpdate
+    → VFS: updateTracked()
+
+[File deleted locally]
+    → VFS: deleteLocally() → DeletedLocallyDocument (or removed if pending)
+    → Queue: enqueue local-delete
+    → Action: DELETE /documents/:id
+    → VFS: confirmDelete() → removed
+
+[Remote update via WebSocket]
+    → Queue: enqueue remote-update
+    → Action: fetch content, write to disk
+    → VFS: updateTracked()
+
+[Crash during create → restart]
+    → VFS loads PendingDocument from disk (idempotencyKey preserved)
+    → resolveIdempotencyKeys() maps key → documentId
+    → VFS: assignDocumentId() → TrackedDocument with serverVersion=0
+    → Queue: enqueue local-create (retry)
+    → Server: returns existing document (idempotent)
+    → VFS: updateTracked() with real serverVersion
+```
+
+## Invariants
+
+1. **All state mutations go through the sequential queue.** No document state can change while a sync operation is running. File-change handlers and WebSocket handlers only enqueue events.
+
+2. **Content cache stores server content after merges.** The cache is used for diff computation: `diff(cached_server_content, new_local_content)`. The server applies diffs against its content at `parentVersionId`.
+
+3. **Idempotency keys survive crashes.** VFS persists pending documents with their keys. On restart, `resolveIdempotencyKeys` maps keys to documentIds. The key is preserved on TrackedDocument when `serverVersion === 0` so retry creates remain idempotent.
+
+4. **Write file before updating metadata.** If the write fails, metadata still points to the old version. On recovery, the stale `serverVersion` triggers a re-fetch from server.
+
+5. **Local events survive reset.** When the WebSocket disconnects, remote events are cleared (server replays on reconnect) but local events are preserved in the queue as unsynced user actions.
+
+6. **Creates run before delete candidates** in the reconciliation ordering. A create may adopt a "deleted" document's identity via server-side merge.
+
+## What Was Removed
+
+- **PQueue** — configurable concurrency queue (replaced by sequential event queue)
+- **Locks** — per-document multi-key locks with alphabetical ordering
+- **Generation counters** — `resetGeneration` for stale operation detection
+- **`containsDocument` guards** — 11 guards after async operations for concurrent-delete protection
+- **`parentVersionIdForUpdate` snapshots** — mutable reference protection
+- **`parallelVersion`** — collision tracking for multiple docs at same path
+- **`UnrestrictedSyncer`** — 1,169-line class with nested if/else (replaced by sync-actions.ts with explicit dispatch)
+- **`Database` class usage** — replaced by VFS everywhere (class still exists for type exports)
+
+## Files
+
+```
+persistence/
+  vfs.ts                    779 lines  — Virtual Filesystem
+  database.ts               535 lines  — Type definitions only (StoredDatabase, RelativePath, etc.)
+
+sync-operations/
+  syncer.ts                 615 lines  — Orchestrator
+  sync-actions.ts          1229 lines  — Action implementations
+  sync-event-queue.ts       242 lines  — Per-document coalescing queue
+  sync-events.ts            297 lines  — Event types + coalescing logic
+  unrestricted-syncer.ts   1169 lines  — DEAD CODE (not imported, to be deleted)
+  cursor-tracker.ts         273 lines  — Cursor position tracking
+```