# CLAUDE.md This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. ## Project Overview VaultLink is a self-hosted Obsidian plugin for real-time collaborative file syncing. The project consists of a Rust-based sync server and a TypeScript frontend with four main components: an Obsidian plugin, a sync client library, a test client, and a standalone CLI client. ## Architecture ### Core Components - **sync-server/**: Rust-based WebSocket server with SQLite database for document versioning and real-time synchronization - **frontend/sync-client/**: TypeScript library providing core sync functionality, WebSocket management, and file operations - **frontend/obsidian-plugin/**: Obsidian plugin that integrates the sync client with Obsidian's API - **frontend/test-client/**: CLI testing tool for simulating multiple concurrent users - **frontend/local-client-cli/**: Standalone CLI for VaultLink sync client - **frontend/history-ui/**: Svelte 5 web UI for browsing vault history, viewing diffs, and restoring versions ### Key Technologies - **Backend**: Rust with Axum framework, SQLite with SQLx, WebSockets for real-time sync - **Frontend**: TypeScript, Webpack for bundling, Node.js native test runner - **History UI**: Svelte 5 with runes, Vite for bundling, embedded in server binary via `rust-embed` - **Sync Algorithm**: Uses reconcile-text library for operational transformation ### Architectural Patterns **Server Architecture:** - `AppState`: Central state container holding `Database`, `Cursors`, and `Broadcasts` - `Database`: SQLite-backed document versioning with SQLx for compile-time query verification - `Broadcasts`: WebSocket broadcast system for real-time updates to connected clients - `Cursors`: Tracks user cursor positions across documents with background cleanup task **Client Architecture (Serial Event Queue Model):** - `SyncClient`: Main entry point, orchestrates all sync operations - `SyncService`: HTTP API client for CRUD operations on documents - `WebSocketManager`: Manages WebSocket connection and real-time updates - `Syncer`: Coordinates file synchronisation via a serial drain loop over a `SyncEventQueue` - `SyncEventQueue`: Intent queue that coalesces events and tracks path→documentId mappings - `CursorTracker`: Manages local and remote cursor positions - `Database`: Client-side document metadata cache (persisted via `PersistenceProvider`) - `FileOperations`: Abstraction layer for filesystem operations (3-way merge on write) **Dual-Bundle Strategy:** The sync-client builds two separate bundles: - `sync-client.web.js`: Browser-compatible UMD bundle (excludes `ws` package) - `sync-client.node.js`: Node.js CommonJS bundle with WebSocket support **History UI Architecture:** The history UI (`frontend/history-ui/`) is a standalone Svelte 5 SPA that provides read-only vault history browsing. It communicates with the server via the same REST API used by sync clients, plus three additional endpoints: - `GET /vaults/:vault_id/documents/:document_id/versions` — all versions of a document (without content) - `GET /vaults/:vault_id/history?limit=&before_update_id=` — paginated vault-wide version history (cursor-based) - `POST /vaults/:vault_id/documents/:document_id/restore` — restore a document to a historical version (creates a new version with old content) Server-side implementation: - Database methods: `get_document_versions()` and `get_vault_history()` in `database.rs`, plus a `VaultHistoryRow` helper struct for `sqlx::query_as!` - Handlers: `fetch_document_versions.rs`, `fetch_vault_history.rs`, `restore_document_version.rs` - Response type: `VaultHistoryResponse { versions, hasMore }` in `responses.rs` - SPA serving: `rust-embed` embeds `frontend/history-ui/dist/` into the binary; `index.rs` serves the SPA at `/` and assets at `/assets/*` Client-side component hierarchy: - `App.svelte` — session restore, routing - `Login.svelte` — vault name + token auth via `/ping` - `Dashboard.svelte` — main layout: file tree sidebar, activity feed, time-travel slider - `DocumentDetail.svelte` — version timeline, content preview, diff view, restore - `DiffView.svelte` — unified diff with LCS algorithm - `FileTree.svelte` — recursive tree built from flat `relativePath` values - `ActivityFeed.svelte` — git-log-style feed with action pills (created/updated/renamed/deleted/restored) - `TimeSlider.svelte` — scrubs through `vaultUpdateId` range, reconstructs vault state at any point State is managed with Svelte 5 runes (`$state`, `$derived`, `$effect`) in `lib/stores.svelte.ts`. Auth is stored in `sessionStorage`. The API client (`lib/api.ts`) sets `Authorization: Bearer` and `device-id: history-ui` headers on all requests. ## Development Commands ### Initial Setup **Node.js (requires version 25):** ```bash curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash nvm install 25 nvm use 25 nvm alias default 25 # Optional: set as system default ``` **Rust:** ```bash curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh cargo install sqlx-cli cargo-machete cargo-edit cargo-insta ``` **Frontend:** ```bash cd frontend npm install ``` ### Server Development ```bash cd sync-server cargo run config-e2e.yml # Start development server cargo test --verbose # Run all Rust tests cargo test # Run specific test cargo clippy --all-targets --all-features # Lint Rust code cargo clippy --all-targets --all-features --fix --allow-dirty --allow-staged # Auto-fix clippy warnings cargo fmt --all -- --check # Check Rust formatting cargo fmt --all # Auto-format Rust code cargo machete --with-metadata # Detect unused dependencies ``` ### Frontend Development ```bash cd frontend npm run dev # Start development mode (watches sync-client and obsidian-plugin) npm run build # Build all workspaces npm run build -w sync-client # Build specific workspace npm run test # Run all tests across all workspaces npm run test -w sync-client # Run tests for specific workspace npm run lint # Lint and format TypeScript code with ESLint + Prettier ``` ### History UI Development ```bash cd frontend npm run dev -w history-ui # Start Vite dev server (localhost:5173, proxies API to localhost:3000) npm run build -w history-ui # Build for production (output: frontend/history-ui/dist/) ``` The history UI is a Svelte 5 SPA embedded in the server binary via `rust-embed`. The build flow is: 1. `npm run build -w history-ui` produces `frontend/history-ui/dist/` 2. The Rust server embeds these files at compile time (`sync-server/src/server/index.rs`) 3. The server serves `index.html` at `GET /` and static assets at `GET /assets/*` 4. If the dist directory doesn't exist at Rust compile time, `build.rs` creates a placeholder During development, run the Vite dev server separately and use its proxy to forward API calls to the running sync server. ### Database Operations ```bash cd sync-server # Create/reset database for development rm -rf db.sqlite* sqlx database create --database-url sqlite://db.sqlite3 sqlx migrate run --source src/app_state/database/migrations --database-url sqlite://db.sqlite3 cargo sqlx prepare --workspace # Add new migration sqlx migrate add --source src/app_state/database/migrations sqlx migrate run --source src/app_state/database/migrations --database-url sqlite://db.sqlite3 ``` ### Project Scripts - `scripts/check.sh`: Full CI check (builds, lints, tests both server and frontend). **Run before pushing.** - `scripts/check.sh --fix`: Same as above but auto-fixes linting and formatting issues - `scripts/e2e.sh`: End-to-end testing (e.g., `scripts/e2e.sh 8` for 8 concurrent clients) - `scripts/clean-up.sh`: Clean logs and database files - `scripts/bump-version.sh patch`: Publish new version (options: patch, minor, major) - `scripts/update-api-types.sh`: Update TypeScript bindings from Rust types (uses ts-rs) ## Code Structure ### Workspace Configuration The frontend uses npm workspaces with five packages: - `sync-client`: Core synchronization logic (builds dual bundles for web and Node.js) - `obsidian-plugin`: Obsidian-specific integration - `test-client`: Testing utilities for E2E tests - `local-client-cli`: Standalone CLI for VaultLink sync client - `history-ui`: Svelte 5 SPA for vault history browsing (built with Vite, embedded in server binary) ### Type Generation and API Updates Rust structs generate TypeScript types via ts-rs crate: 1. Rust structs annotated with `#[derive(TS)]` export to `sync-server/bindings/` 2. Run `scripts/update-api-types.sh` to copy bindings to `frontend/sync-client/src/services/types/` 3. Frontend imports these types for type-safe API communication ### Important Implementation Details **SQLx Compile-Time Verification:** - SQLx verifies SQL queries at compile time against the database schema - Run `cargo sqlx prepare --workspace` after schema changes to update `.sqlx/` directory - CI builds require prepared query metadata to avoid needing a live database ## Testing ### Running Tests **Server:** ```bash cargo test --verbose # All tests cargo test # Specific test ``` **Frontend:** ```bash npm run test # All workspaces npm run test -w sync-client # Specific workspace ``` **E2E:** ```bash scripts/e2e.sh 8 # 8 concurrent clients scripts/clean-up.sh # Clean up after tests ``` ### Test Structure - **Rust**: Unit tests alongside source files, uses `cargo-insta` for snapshot testing - **TypeScript**: `.test.ts` files using Node.js native test runner (not Jest) - **E2E**: Uses `test-client` to simulate multiple concurrent users with random operations - **Deterministic**: Step-by-step sync scenario tests in `frontend/deterministic-tests/` ### Deterministic Tests (`frontend/deterministic-tests/`) Controlled, step-by-step sync scenario tests that exercise specific edge cases. Each test defines a sequence of operations (create, update, rename, delete, enable/disable sync, pause/resume server) and asserts convergence across multiple agents. **Running:** ```bash cd frontend/deterministic-tests npx webpack --config webpack.config.js # Build (required after changes) node dist/cli.js # Run all tests node dist/cli.js --filter=write-write # Run tests matching a name/key ``` Requires the server binary at `sync-server/target/release/sync_server` and `sync-server/config-e2e.yml`. The harness starts/stops servers automatically. **Architecture:** - `DeterministicAgent` extends `InMemoryFileSystem` — wraps a real `SyncClient` with an in-memory filesystem - `TestRunner` executes `TestStep[]` sequentially, manages agent lifecycle - `ServerControl` manages server processes (start/stop/SIGSTOP/SIGCONT) - Tests that use `pause-server`/`resume-server` get dedicated server instances; regular tests share one - Each test gets a unique vault name (UUID) for isolation **Writing Tests — Step Types:** ```typescript { type: "create", client: 0, path: "A.md", content: "hello" } { type: "update", client: 0, path: "A.md", content: "updated" } { type: "rename", client: 0, oldPath: "A.md", newPath: "B.md" } { type: "delete", client: 0, path: "A.md" } { type: "enable-sync", client: 0 } // Connects WS, triggers reconciliation { type: "disable-sync", client: 0 } // Disconnects WS { type: "sync", client: 0 } // Wait for specific client to settle { type: "sync" } // Wait for ALL clients to settle { type: "barrier" } // Wait for convergence + check consistency { type: "pause-server" } // SIGSTOP the server process { type: "resume-server" } // SIGCONT + wait for readiness { type: "assert-consistent", verify?: (state: AssertableState) => void } ``` **Critical Rules When Writing Tests:** 1. **Agents start with sync DISABLED.** Do not `disable-sync` on an agent that hasn't been `enable-sync`'d — it's already off. 2. **Do not put `{ type: "sync" }` before `{ type: "barrier" }`.** The barrier already calls `waitAllAgentsSettled()` (2 rounds of `waitForSync` on all agents). Adding a `sync` before it is pure redundancy. Use targeted `{ type: "sync", client: N }` only when you need a specific client to finish before another client acts. 3. **`enable-sync` blocks until WebSocket connects.** If the server is paused (SIGSTOP), `enable-sync` will hang for 10 seconds then fail. Never `enable-sync` while the server is paused. Tests that need to stall in-flight requests should enable sync FIRST, then pause the server. 4. **File operations while sync is disabled are queued.** When `createFile` is called on the agent, `enqueueSync(syncLocallyCreatedFile)` fires immediately but the fetch is disabled. The `scheduleSyncForOfflineChanges` reconciliation scans the filesystem and re-enqueues all pending changes on the next `enable-sync`. 5. **`barrier` retries for up to 60 seconds.** It calls `waitAllAgentsSettled`, checks consistency, and if clients disagree, sleeps 500ms and retries. Tests that need more settling time should add targeted `sync` steps before the barrier (e.g., `{ type: "sync", client: 0 }` to ensure client 0's operations complete first). 6. **No comments in test files.** The test name/description and step types are self-documenting. Keep test files comment-free. 7. **Keep tests minimal.** Each test should reproduce exactly one edge case with the fewest steps possible. Don't add `assert-consistent` after `barrier` unless it has a `verify` callback (barrier already checks consistency). Always use inline arrow functions for `verify` callbacks rather than separate named functions. 8. **Treat sync as a black box in test names/descriptions.** Don't reference internal implementation details (VFS, coalescing, idempotency keys, reconciliation, parentVersionId, etc.). Describe the observable scenario and expected outcome from the user's perspective. **Test Patterns for Common Edge Cases:** *Two clients create at same path (offline):* ```typescript steps: [ { type: "create", client: 0, path: "A.md", content: "hello" }, { type: "create", client: 1, path: "A.md", content: "world" }, { type: "enable-sync", client: 0 }, { type: "enable-sync", client: 1 }, { type: "barrier" }, { type: "assert-consistent", verify: verifyMergedContent } ] ``` *Client edits while other client is offline:* ```typescript steps: [ { type: "create", client: 0, path: "A.md", content: "original" }, { type: "enable-sync", client: 0 }, { type: "enable-sync", client: 1 }, { type: "barrier" }, // Client 1 goes offline, client 0 edits { type: "disable-sync", client: 1 }, { type: "update", client: 0, path: "A.md", content: "edited" }, { type: "sync", client: 0 }, // Client 1 reconnects { type: "enable-sync", client: 1 }, { type: "barrier" }, { type: "assert-consistent" } ] ``` *Testing behavior during server pause (stalled HTTP requests):* ```typescript steps: [ // Setup FIRST — both clients must be online before pausing { type: "enable-sync", client: 0 }, { type: "enable-sync", client: 1 }, { type: "barrier" }, // NOW pause — in-flight requests from subsequent operations will stall { type: "pause-server" }, { type: "create", client: 0, path: "A.md", content: "hello" }, { type: "resume-server" }, { type: "barrier" }, { type: "assert-consistent" } ] ``` **Verify Functions and `AssertableState`:** The `verify` callback on `assert-consistent` receives an `AssertableState` object (defined in `utils/assertable-state.ts`) with chainable assertion methods: ```typescript state.assertFileCount(2) // exact file count state.assertFileExists("A.md") // file must exist state.assertFileNotExists("old.md") // file must not exist state.assertContent("A.md", "hello") // exact content match state.assertContains("A.md", "hello", "world") // all substrings present state.assertContainsAny("A.md", "hello", "world") // at least one substring state.assertAnyFileContains("content-a") // substring in any file state.assertSubstringCount("A.md", "hello", 1) // occurrence count state.assertContentInAtMostOneFile("original") // no duplicate content state.ifFileExists("A.md", (s) => ...) // conditional assertion state.getContent("A.md") // raw content access ``` All methods return `this` for chaining. The object also exposes `files` and `clientFiles` for custom logic. For conflict-resolution tests where the outcome is genuinely ambiguous (delete vs update, rename ordering), use `ifFileExists`. For merges where both sides MUST be preserved, use `assertContains`. When the empty-parent merge (invariant #15) is involved, word boundaries may be garbled — check for fragments, not exact substrings. ```typescript function verify(state: AssertableState): void { state.ifFileExists("A.md", (s) => s.assertContent("A.md", "expected content")); } function verify(state: AssertableState): void { state.assertContains("A.md", "edit from 0", "edit from 1"); } ``` **Adding a New Test:** 1. Create `frontend/deterministic-tests/src/tests/your-test-name.test.ts` 2. Export a `TestDefinition` with `clients` and `steps` (the test name is derived from the registry key) 3. Import and register in `test-registry.ts` 4. Build with `npx webpack --config webpack.config.js` 5. Run with `node dist/cli.js --filter=your-test-name` **Known Limitations:** - Cannot test VFS.move failures — the in-memory filesystem never fails - Cannot `enable-sync` while the server is paused — the WebSocket connection will time out - The empty-parent 3-way merge (used for smart creates) can produce garbled word boundaries — check for fragments, not exact substrings - The test harness can hang during shared server cleanup when transitioning to server-pause tests ## Code Style and Formatting ### Rust - Extensive Clippy lints (see `Cargo.toml`) - Pedantic linting rules enabled - Forbids unsafe code - Uses `rustfmt.toml` for formatting configuration (4 spaces, Unix line endings) - Run `cargo fmt --all` to format ### TypeScript - **Prettier**: 4-space indentation, no trailing commas, LF line endings - **YAML/Markdown override**: 2-space indentation (via prettier config) - **ESLint**: Strict rules with unused imports detection - Configuration in `frontend/package.json` - Run `npm run lint` to format and fix issues ### Svelte (History UI) - Uses Svelte 5 runes syntax (`$state`, `$derived`, `$effect`, `$props`) - Vite as bundler with `@sveltejs/vite-plugin-svelte` - Excluded from the main ESLint config (Svelte files need different linting); `history-ui/**` is in the eslint ignores list - CSS is component-scoped via Svelte's `