This commit is contained in:
Andras Schmelczer 2026-04-06 13:01:47 +01:00
parent 0e3e5a99cd
commit d034ad5cb3
50 changed files with 6515 additions and 1492 deletions

9
.gitignore vendored
View file

@ -7,15 +7,18 @@ node_modules
# Frontend build folders
frontend/*/dist
sync-server/db.sqlite3*
sync-server/databases
# Rust build folders
sync-server/target
sync-server/artifacts
sync-server/bindings/*.ts
# build folders
sync-server/db.sqlite3*
**/databases
*.log
*.sqlx
target
.task

562
CLAUDE.md
View file

@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
## Project Overview
VaultLink is a self-hosted Obsidian plugin for real-time collaborative file syncing. The project consists of a Rust-based sync server and a TypeScript frontend with three main components: an Obsidian plugin, a sync client library, and a test client.
VaultLink is a self-hosted Obsidian plugin for real-time collaborative file syncing. The project consists of a Rust-based sync server and a TypeScript frontend with four main components: an Obsidian plugin, a sync client library, a test client, and a standalone CLI client.
## Architecture
@ -13,21 +13,104 @@ VaultLink is a self-hosted Obsidian plugin for real-time collaborative file sync
- **sync-server/**: Rust-based WebSocket server with SQLite database for document versioning and real-time synchronization
- **frontend/sync-client/**: TypeScript library providing core sync functionality, WebSocket management, and file operations
- **frontend/obsidian-plugin/**: Obsidian plugin that integrates the sync client with Obsidian's API
- **frontend/test-client/**: CLI testing tool for the sync functionality
- **frontend/test-client/**: CLI testing tool for simulating multiple concurrent users
- **frontend/local-client-cli/**: Standalone CLI for VaultLink sync client
- **frontend/history-ui/**: Svelte 5 web UI for browsing vault history, viewing diffs, and restoring versions
### Key Technologies
- **Backend**: Rust with Axum framework, SQLite with SQLx, WebSockets for real-time sync
- **Frontend**: TypeScript, Webpack for bundling, Jest for testing
- **Frontend**: TypeScript, Webpack for bundling, Node.js native test runner
- **History UI**: Svelte 5 with runes, Vite for bundling, embedded in server binary via `rust-embed`
- **Sync Algorithm**: Uses reconcile-text library for operational transformation
### Architectural Patterns
**Server Architecture:**
- `AppState`: Central state container holding `Database`, `Cursors`, and `Broadcasts`
- `Database`: SQLite-backed document versioning with SQLx for compile-time query verification
- `Broadcasts`: WebSocket broadcast system for real-time updates to connected clients
- `Cursors`: Tracks user cursor positions across documents with background cleanup task
**Client Architecture (Serial Event Queue Model):**
- `SyncClient`: Main entry point, orchestrates all sync operations
- `SyncService`: HTTP API client for CRUD operations on documents
- `WebSocketManager`: Manages WebSocket connection and real-time updates
- `Syncer`: Coordinates file synchronisation via a serial drain loop over a `SyncEventQueue`
- `SyncEventQueue`: Intent queue that coalesces events and tracks path→documentId mappings
- `CursorTracker`: Manages local and remote cursor positions
- `Database`: Client-side document metadata cache (persisted via `PersistenceProvider`)
- `FileOperations`: Abstraction layer for filesystem operations (3-way merge on write)
**Dual-Bundle Strategy:**
The sync-client builds two separate bundles:
- `sync-client.web.js`: Browser-compatible UMD bundle (excludes `ws` package)
- `sync-client.node.js`: Node.js CommonJS bundle with WebSocket support
**History UI Architecture:**
The history UI (`frontend/history-ui/`) is a standalone Svelte 5 SPA that provides read-only vault history browsing. It communicates with the server via the same REST API used by sync clients, plus three additional endpoints:
- `GET /vaults/:vault_id/documents/:document_id/versions` — all versions of a document (without content)
- `GET /vaults/:vault_id/history?limit=&before_update_id=` — paginated vault-wide version history (cursor-based)
- `POST /vaults/:vault_id/documents/:document_id/restore` — restore a document to a historical version (creates a new version with old content)
Server-side implementation:
- Database methods: `get_document_versions()` and `get_vault_history()` in `database.rs`, plus a `VaultHistoryRow` helper struct for `sqlx::query_as!`
- Handlers: `fetch_document_versions.rs`, `fetch_vault_history.rs`, `restore_document_version.rs`
- Response type: `VaultHistoryResponse { versions, hasMore }` in `responses.rs`
- SPA serving: `rust-embed` embeds `frontend/history-ui/dist/` into the binary; `index.rs` serves the SPA at `/` and assets at `/assets/*`
Client-side component hierarchy:
- `App.svelte` — session restore, routing
- `Login.svelte` — vault name + token auth via `/ping`
- `Dashboard.svelte` — main layout: file tree sidebar, activity feed, time-travel slider
- `DocumentDetail.svelte` — version timeline, content preview, diff view, restore
- `DiffView.svelte` — unified diff with LCS algorithm
- `FileTree.svelte` — recursive tree built from flat `relativePath` values
- `ActivityFeed.svelte` — git-log-style feed with action pills (created/updated/renamed/deleted/restored)
- `TimeSlider.svelte` — scrubs through `vaultUpdateId` range, reconstructs vault state at any point
State is managed with Svelte 5 runes (`$state`, `$derived`, `$effect`) in `lib/stores.svelte.ts`. Auth is stored in `sessionStorage`. The API client (`lib/api.ts`) sets `Authorization: Bearer` and `device-id: history-ui` headers on all requests.
## Development Commands
### Initial Setup
**Node.js (requires version 25):**
```bash
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash
nvm install 25
nvm use 25
nvm alias default 25 # Optional: set as system default
```
**Rust:**
```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
cargo install sqlx-cli cargo-machete cargo-edit cargo-insta
```
**Frontend:**
```bash
cd frontend
npm install
```
### Server Development
```bash
cd sync-server
cargo run config-e2e.yml # Start development server
cargo test --verbose # Run Rust tests
cargo test --verbose # Run all Rust tests
cargo test <test_name> # Run specific test
cargo clippy --all-targets --all-features # Lint Rust code
cargo clippy --all-targets --all-features --fix --allow-dirty --allow-staged # Auto-fix clippy warnings
cargo fmt --all -- --check # Check Rust formatting
@ -36,75 +119,474 @@ cargo machete --with-metadata # Detect unused dependencies
```
### Frontend Development
```bash
cd frontend
npm run dev # Start development mode (watches sync-client and obsidian-plugin)
npm run build # Build all workspaces
npm run test # Run all tests
npm run lint # Lint and format TypeScript code
npm run build -w sync-client # Build specific workspace
npm run test # Run all tests across all workspaces
npm run test -w sync-client # Run tests for specific workspace
npm run lint # Lint and format TypeScript code with ESLint + Prettier
```
### Database Setup (Development)
### History UI Development
```bash
cd frontend
npm run dev -w history-ui # Start Vite dev server (localhost:5173, proxies API to localhost:3000)
npm run build -w history-ui # Build for production (output: frontend/history-ui/dist/)
```
The history UI is a Svelte 5 SPA embedded in the server binary via `rust-embed`. The build flow is:
1. `npm run build -w history-ui` produces `frontend/history-ui/dist/`
2. The Rust server embeds these files at compile time (`sync-server/src/server/index.rs`)
3. The server serves `index.html` at `GET /` and static assets at `GET /assets/*`
4. If the dist directory doesn't exist at Rust compile time, `build.rs` creates a placeholder
During development, run the Vite dev server separately and use its proxy to forward API calls to the running sync server.
### Database Operations
```bash
cd sync-server
# Create/reset database for development
rm -rf db.sqlite*
sqlx database create --database-url sqlite://db.sqlite3
sqlx migrate run --source src/app_state/database/migrations --database-url sqlite://db.sqlite3
cargo sqlx prepare --workspace
# Add new migration
sqlx migrate add --source src/app_state/database/migrations <migration_name>
sqlx migrate run --source src/app_state/database/migrations --database-url sqlite://db.sqlite3
```
### Initial Setup
```bash
# Install required cargo tools
cargo install sqlx-cli cargo-machete cargo-edit
```
### Project Scripts
### Scripts
- `scripts/check.sh`: Full CI check (builds, lints, tests both server and frontend)
- `scripts/check.sh`: Full CI check (builds, lints, tests both server and frontend). **Run before pushing.**
- `scripts/check.sh --fix`: Same as above but auto-fixes linting and formatting issues
- `scripts/e2e.sh`: End-to-end testing
- `scripts/e2e.sh`: End-to-end testing (e.g., `scripts/e2e.sh 8` for 8 concurrent clients)
- `scripts/clean-up.sh`: Clean logs and database files
- `scripts/bump-version.sh patch`: Publish new version
- `scripts/update-api-types.sh`: Update TypeScript bindings from Rust types
- `scripts/bump-version.sh patch`: Publish new version (options: patch, minor, major)
- `scripts/update-api-types.sh`: Update TypeScript bindings from Rust types (uses ts-rs)
## Code Structure
### Workspace Configuration
The frontend uses npm workspaces with four packages:
- `sync-client`: Core synchronization logic
The frontend uses npm workspaces with five packages:
- `sync-client`: Core synchronization logic (builds dual bundles for web and Node.js)
- `obsidian-plugin`: Obsidian-specific integration
- `test-client`: Testing utilities
- `test-client`: Testing utilities for E2E tests
- `local-client-cli`: Standalone CLI for VaultLink sync client
- `history-ui`: Svelte 5 SPA for vault history browsing (built with Vite, embedded in server binary)
### Type Generation
Rust structs generate TypeScript types via ts-rs crate, stored in `sync-server/bindings/` and used by frontend packages.
### Type Generation and API Updates
### Key Files
- `sync-server/src/`: Rust server implementation with WebSocket handlers
- `frontend/sync-client/src/sync-client.ts`: Main sync client entry point
- `frontend/obsidian-plugin/src/vault-link-plugin.ts`: Main Obsidian plugin class
- `frontend/sync-client/src/services/sync-service.ts`: Core synchronization logic
Rust structs generate TypeScript types via ts-rs crate:
1. Rust structs annotated with `#[derive(TS)]` export to `sync-server/bindings/`
2. Run `scripts/update-api-types.sh` to copy bindings to `frontend/sync-client/src/services/types/`
3. Frontend imports these types for type-safe API communication
### Important Implementation Details
**SQLx Compile-Time Verification:**
- SQLx verifies SQL queries at compile time against the database schema
- Run `cargo sqlx prepare --workspace` after schema changes to update `.sqlx/` directory
- CI builds require prepared query metadata to avoid needing a live database
## Testing
### Running Tests
- Server: `cargo test --verbose`
- Frontend: `npm run test` (runs Jest across all workspaces)
- E2E: `scripts/e2e.sh`
**Server:**
```bash
cargo test --verbose # All tests
cargo test <test_name> # Specific test
```
**Frontend:**
```bash
npm run test # All workspaces
npm run test -w sync-client # Specific workspace
```
**E2E:**
```bash
scripts/e2e.sh 8 # 8 concurrent clients
scripts/clean-up.sh # Clean up after tests
```
### Test Structure
- Rust: Unit tests alongside source files
- TypeScript: `.test.ts` files using Jest
- E2E: Uses test-client to simulate multiple concurrent users
## Code Style
- **Rust**: Unit tests alongside source files, uses `cargo-insta` for snapshot testing
- **TypeScript**: `.test.ts` files using Node.js native test runner (not Jest)
- **E2E**: Uses `test-client` to simulate multiple concurrent users with random operations
- **Deterministic**: Step-by-step sync scenario tests in `frontend/deterministic-tests/`
### Deterministic Tests (`frontend/deterministic-tests/`)
Controlled, step-by-step sync scenario tests that exercise specific edge cases. Each test defines a sequence of operations (create, update, rename, delete, enable/disable sync, pause/resume server) and asserts convergence across multiple agents.
**Running:**
```bash
cd frontend/deterministic-tests
npx webpack --config webpack.config.js # Build (required after changes)
node dist/cli.js # Run all tests
node dist/cli.js --filter=write-write # Run tests matching a name/key
```
Requires the server binary at `sync-server/target/release/sync_server` and `sync-server/config-e2e.yml`. The harness starts/stops servers automatically.
**Architecture:**
- `DeterministicAgent` extends `InMemoryFileSystem` — wraps a real `SyncClient` with an in-memory filesystem
- `TestRunner` executes `TestStep[]` sequentially, manages agent lifecycle
- `ServerControl` manages server processes (start/stop/SIGSTOP/SIGCONT)
- Tests that use `pause-server`/`resume-server` get dedicated server instances; regular tests share one
- Each test gets a unique vault name (UUID) for isolation
**Writing Tests — Step Types:**
```typescript
{ type: "create", client: 0, path: "A.md", content: "hello" }
{ type: "update", client: 0, path: "A.md", content: "updated" }
{ type: "rename", client: 0, oldPath: "A.md", newPath: "B.md" }
{ type: "delete", client: 0, path: "A.md" }
{ type: "enable-sync", client: 0 } // Connects WS, triggers reconciliation
{ type: "disable-sync", client: 0 } // Disconnects WS
{ type: "sync", client: 0 } // Wait for specific client to settle
{ type: "sync" } // Wait for ALL clients to settle
{ type: "barrier" } // Wait for convergence + check consistency
{ type: "pause-server" } // SIGSTOP the server process
{ type: "resume-server" } // SIGCONT + wait for readiness
{ type: "assert-consistent", verify?: (state: AssertableState) => void }
```
**Critical Rules When Writing Tests:**
1. **Agents start with sync DISABLED.** Do not `disable-sync` on an agent that hasn't been `enable-sync`'d — it's already off.
2. **Do not put `{ type: "sync" }` before `{ type: "barrier" }`.** The barrier already calls `waitAllAgentsSettled()` (2 rounds of `waitForSync` on all agents). Adding a `sync` before it is pure redundancy. Use targeted `{ type: "sync", client: N }` only when you need a specific client to finish before another client acts.
3. **`enable-sync` blocks until WebSocket connects.** If the server is paused (SIGSTOP), `enable-sync` will hang for 10 seconds then fail. Never `enable-sync` while the server is paused. Tests that need to stall in-flight requests should enable sync FIRST, then pause the server.
4. **File operations while sync is disabled are queued.** When `createFile` is called on the agent, `enqueueSync(syncLocallyCreatedFile)` fires immediately but the fetch is disabled. The `scheduleSyncForOfflineChanges` reconciliation scans the filesystem and re-enqueues all pending changes on the next `enable-sync`.
5. **`barrier` retries for up to 60 seconds.** It calls `waitAllAgentsSettled`, checks consistency, and if clients disagree, sleeps 500ms and retries. Tests that need more settling time should add targeted `sync` steps before the barrier (e.g., `{ type: "sync", client: 0 }` to ensure client 0's operations complete first).
6. **No comments in test files.** The test name/description and step types are self-documenting. Keep test files comment-free.
7. **Keep tests minimal.** Each test should reproduce exactly one edge case with the fewest steps possible. Don't add `assert-consistent` after `barrier` unless it has a `verify` callback (barrier already checks consistency). Always use inline arrow functions for `verify` callbacks rather than separate named functions.
8. **Treat sync as a black box in test names/descriptions.** Don't reference internal implementation details (VFS, coalescing, idempotency keys, reconciliation, parentVersionId, etc.). Describe the observable scenario and expected outcome from the user's perspective.
**Test Patterns for Common Edge Cases:**
*Two clients create at same path (offline):*
```typescript
steps: [
{ type: "create", client: 0, path: "A.md", content: "hello" },
{ type: "create", client: 1, path: "A.md", content: "world" },
{ type: "enable-sync", client: 0 },
{ type: "enable-sync", client: 1 },
{ type: "barrier" },
{ type: "assert-consistent", verify: verifyMergedContent }
]
```
*Client edits while other client is offline:*
```typescript
steps: [
{ type: "create", client: 0, path: "A.md", content: "original" },
{ type: "enable-sync", client: 0 },
{ type: "enable-sync", client: 1 },
{ type: "barrier" },
// Client 1 goes offline, client 0 edits
{ type: "disable-sync", client: 1 },
{ type: "update", client: 0, path: "A.md", content: "edited" },
{ type: "sync", client: 0 },
// Client 1 reconnects
{ type: "enable-sync", client: 1 },
{ type: "barrier" },
{ type: "assert-consistent" }
]
```
*Testing behavior during server pause (stalled HTTP requests):*
```typescript
steps: [
// Setup FIRST — both clients must be online before pausing
{ type: "enable-sync", client: 0 },
{ type: "enable-sync", client: 1 },
{ type: "barrier" },
// NOW pause — in-flight requests from subsequent operations will stall
{ type: "pause-server" },
{ type: "create", client: 0, path: "A.md", content: "hello" },
{ type: "resume-server" },
{ type: "barrier" },
{ type: "assert-consistent" }
]
```
**Verify Functions and `AssertableState`:**
The `verify` callback on `assert-consistent` receives an `AssertableState` object (defined in `utils/assertable-state.ts`) with chainable assertion methods:
```typescript
state.assertFileCount(2) // exact file count
state.assertFileExists("A.md") // file must exist
state.assertFileNotExists("old.md") // file must not exist
state.assertContent("A.md", "hello") // exact content match
state.assertContains("A.md", "hello", "world") // all substrings present
state.assertContainsAny("A.md", "hello", "world") // at least one substring
state.assertAnyFileContains("content-a") // substring in any file
state.assertSubstringCount("A.md", "hello", 1) // occurrence count
state.assertContentInAtMostOneFile("original") // no duplicate content
state.ifFileExists("A.md", (s) => ...) // conditional assertion
state.getContent("A.md") // raw content access
```
All methods return `this` for chaining. The object also exposes `files` and `clientFiles` for custom logic.
For conflict-resolution tests where the outcome is genuinely ambiguous (delete vs update, rename ordering), use `ifFileExists`. For merges where both sides MUST be preserved, use `assertContains`. When the empty-parent merge (invariant #15) is involved, word boundaries may be garbled — check for fragments, not exact substrings.
```typescript
function verify(state: AssertableState): void {
state.ifFileExists("A.md", (s) => s.assertContent("A.md", "expected content"));
}
function verify(state: AssertableState): void {
state.assertContains("A.md", "edit from 0", "edit from 1");
}
```
**Adding a New Test:**
1. Create `frontend/deterministic-tests/src/tests/your-test-name.test.ts`
2. Export a `TestDefinition` with `clients` and `steps` (the test name is derived from the registry key)
3. Import and register in `test-registry.ts`
4. Build with `npx webpack --config webpack.config.js`
5. Run with `node dist/cli.js --filter=your-test-name`
**Known Limitations:**
- Cannot test VFS.move failures — the in-memory filesystem never fails
- Cannot `enable-sync` while the server is paused — the WebSocket connection will time out
- The empty-parent 3-way merge (used for smart creates) can produce garbled word boundaries — check for fragments, not exact substrings
- The test harness can hang during shared server cleanup when transitioning to server-pause tests
## Code Style and Formatting
### Rust
- Uses extensive Clippy lints (see Cargo.toml)
- Follows pedantic linting rules
- Extensive Clippy lints (see `Cargo.toml`)
- Pedantic linting rules enabled
- Forbids unsafe code
- Uses cargo fmt with default settings
- Uses `rustfmt.toml` for formatting configuration (4 spaces, Unix line endings)
- Run `cargo fmt --all` to format
### TypeScript
- Prettier configuration: 4-space tabs, trailing commas removed, LF line endings
- ESLint with unused imports plugin
- Consistent across all three frontend packages
- **Prettier**: 4-space indentation, no trailing commas, LF line endings
- **YAML/Markdown override**: 2-space indentation (via prettier config)
- **ESLint**: Strict rules with unused imports detection
- Configuration in `frontend/package.json`
- Run `npm run lint` to format and fix issues
### Svelte (History UI)
- Uses Svelte 5 runes syntax (`$state`, `$derived`, `$effect`, `$props`)
- Vite as bundler with `@sveltejs/vite-plugin-svelte`
- Excluded from the main ESLint config (Svelte files need different linting); `history-ui/**` is in the eslint ignores list
- CSS is component-scoped via Svelte's `<style>` blocks with CSS custom properties defined in `app.css`
### EditorConfig
- `.editorconfig` at project root defines baseline formatting rules
- `rustfmt.toml` and Prettier config explicitly mirror these settings
- Both formatters enforce: 4-space indent (2 for YAML/MD), LF endings, final newline, trim trailing whitespace
## Sync Logic Deep Dive
### Architecture: Serial Event Queue (Command Sourcing)
The sync client uses a **serial event queue** pattern inspired by Dropbox's Nucleus rewrite. All sync decisions run through a single `drain()` loop — one event at a time. There are no per-document locks, no concurrent sync queues, and no PQueue. JS is single-threaded, so in-memory state changes between `await` points are impossible.
**Core components:**
- `SyncEventQueue` — intent queue that stores `SyncEvent[]` and a `documentIdsByPath` map (path → `{ documentId, hash, vaultUpdateId }`). Events are coalesced at dequeue time (not enqueue time)
- `Syncer.drain()` — serial loop that calls `next()` on the queue and processes one event at a time
- `Syncer.scheduleDrain()` — chains new drains after existing ones via `.then()` to ensure items enqueued during a drain are always processed by a subsequent drain
**Event types:**
| Event | Enqueued by | Processing |
|-------|-------------|------------|
| `create` | `syncLocallyCreatedFile` | Read file, POST to server, resolve promise-based documentId |
| `delete` | `syncLocallyDeletedFile`, auto (move-overwrite) | DELETE on server, mark `isDeleted` in database |
| `move` | `syncLocallyUpdatedFile` (with `oldPath`) | PUT with new path, server returns updated document |
| `local-content-update` | `syncLocallyUpdatedFile` (no `oldPath`) | Read file, compute diff, PUT to server |
| `remote-update` | `syncRemotelyUpdatedFile` (WebSocket) | Fetch from server, write locally, or merge |
**Event coalescing in `next()`:**
1. If there is an eventual `delete` for a document, all preceding events for that document are discarded — only the delete is returned
2. Multiple `local-content-update` / `remote-update` events for the same document coalesce to the last one (the file is re-read at processing time anyway)
3. `move` events act as barriers — updates cannot coalesce across a move since the path changed
4. `create` events are always returned immediately (FIFO) since they carry a `resolve` callback
**Promise-based documentIds for pending creates:**
When a `create` event is enqueued, `Promise.withResolvers<DocumentId>()` creates a promise. The `resolve` function is stored on the event. Subsequent events for the same path (move, delete, update) carry this promise as their `documentId`. When `processCreate` gets the server response, it calls `event.resolve(response.documentId)`, and all dependent events can then `await` the resolved ID.
### Move-to-Occupied-Path Handling
When the user renames a file to a path where another file already exists (overwrite semantics), the queue automatically enqueues a `delete` for the occupying document before the `move`. The `syncLocallyUpdatedFile` method also marks the displaced document as deleted in the database and clears it from the path to allow `database.move()` to succeed.
### Remote Delete vs Local Changes
When a remote delete arrives for a document that has been locally modified (hash differs from the last synced hash), the local changes survive. The client:
1. Removes the old document record from the database
2. Re-creates the file as a new document via `syncLocallyCreatedFile`
This preserves the user's edits. If the file has NOT been modified locally, the delete is applied normally.
The same logic applies in `handleMaybeMergingResponse`: if the server returns `isDeleted: true` for an update but the file still exists locally, the file is re-uploaded as a new document rather than being deleted.
### The Offline Sync Algorithm (`scheduleSyncForOfflineChanges`)
Runs on reconnect to detect what changed while offline:
1. Register all known documents from the database into the queue's path→documentId map
2. List all local files
3. For each file at a known path: **hash-check** against the database record. If the hash doesn't match AND matches a document missing from disk elsewhere, treat it as a rename (the file was moved to this path while offline, displacing the original document)
4. For each file without metadata at its path: try to match against "missing" DB records by content hash (detects moves). If no match, schedule as create
5. For DB records whose files don't exist locally: schedule as delete
6. Ordering: deletes first → updates/moves second → creates last. Creates run after deletes so the server can merge creates with existing documents at the same path
### Remote Update Processing
When the server broadcasts updates via WebSocket:
1. `scheduleSyncForOfflineChanges()` runs first (ensures local changes are queued)
2. Remote updates are enqueued with full `DocumentVersionWithoutContent` metadata (avoids an extra network call)
3. For each remote document:
- **Known document, already up-to-date** (`parentVersionId >= remoteVersion.vaultUpdateId`): skip
- **Known document, remote delete**: check for local changes; if present, re-create; if not, delete
- **Known document, remote update with local changes**: send local changes (server merges)
- **Known document, remote update without local changes**: fetch and apply
- **Unknown document, deleted**: skip
- **Unknown document, live**: download and create locally
### Drain Scheduling
`scheduleDrain()` chains drain promises: `this.draining = (this.draining ?? Promise.resolve()).then(() => this.drain())`. This ensures that events enqueued during an `await` inside a running drain are always processed by the next drain in the chain. The previous approach of storing a single `this.draining` promise and checking `undefined` had a race condition where events enqueued between the drain finishing and the `finally` block clearing `this.draining` were never processed.
### Server-Side Smart Create
When a client sends a create request for a path where a document already exists:
1. Server calls `merge_with_stored_version` instead of creating a new document
2. Content is 3-way merged using `reconcile-text` (for text files) or last-write-wins (for binary)
3. The merge parent is an empty string, not the existing content — this treats both sides as independent additions
4. The response uses the EXISTING document's `documentId` — the client adopts it
### Sync Reset and Recovery
A `SyncResetError` is thrown when the WebSocket disconnects or sync is toggled off. This:
- Clears the event queue (events + path map)
- On reconnect, `scheduleSyncForOfflineChanges()` runs a fresh filesystem scan
`runningScheduleSyncForOfflineChanges` is cleared on WebSocket disconnect so the next connection triggers a fresh scan.
**Important**: `SyncResetError` during `syncRemotelyUpdatedFile` must be caught and logged as INFO, not ERROR. The test client exits on ERROR-level logs, so logging SyncResetError as ERROR during expected resets causes false test failures.
### Critical Implementation Invariants
These invariants were discovered through testing. Violating them causes data loss, sync stalls, or test failures.
**Client-side invariants:**
**1. `handleMaybeMergingResponse` must write the file BEFORE updating metadata.**
Order: write file → re-read + re-hash → update metadata → update cache. If metadata is updated first and the write fails, the metadata points to a server version whose content was never written locally.
**2. After a MergingUpdate, cache the SERVER's content (`responseBytes`), not the local content.**
The content cache is used to compute diffs: `diff(cached, newFileContent)`. The server applies this diff against its content at `parentVersionId`. If the cache stores local content (which may differ due to the 3-way merge in `FileOperations.write`), subsequent diffs produce "Invalid diff" errors.
**3. After a MergingUpdate, re-read the file and re-hash.**
The 3-way merge in `operations.write()` may produce content different from `responseBytes`. The stored hash must match actual on-disk content, not the server's merged content.
**4. The drain must be chained, not gated.**
`scheduleDrain()` must chain via `.then()`: `this.draining = (this.draining ?? Promise.resolve()).then(() => this.drain())`. A gate pattern (`if (this.draining === undefined)`) has a race condition: events enqueued after the drain finishes its while loop but before the `finally` block clears `this.draining` are never processed.
**5. `syncLocallyUpdatedFile` must guard against unregistered paths.**
If sync is disabled when a rename/update event arrives, the queue's path map may not have the path registered. Check `getDocumentId(path) !== undefined` before calling `enqueue` for `move` or `local-content-update` events.
**6. Move-to-occupied-path must auto-delete the target.**
When a rename overwrites an existing file, `syncLocallyUpdatedFile` must mark the target document as deleted in the database, and the queue must enqueue a `delete` for the occupying document before the `move` event.
**7. Remote deletes must check for local changes before applying.**
In `processRemoteUpdateForExistingDocument`, when `remoteVersion.isDeleted`, read the local file and compare its hash to the stored hash. If the hash differs, the user made local edits — re-upload as a new document instead of deleting. If `handleMaybeMergingResponse` receives `isDeleted: true` but the file still exists locally, also re-upload rather than delete.
**8. `processDelete` must mark `isDeleted` on the database record.**
Call `database.delete(relativePath)` after `updateDocumentMetadata` so the record has `isDeleted: true`. This is needed so that subsequent `database.move()` calls to the same path don't throw "Document already exists".
**9. Offline scan must hash-check files at known paths.**
When a file exists at a path with a database record but `locallyPossiblyDeletedFiles` is non-empty, hash the file and check if it matches a missing document. If so, the file was renamed to this path — enqueue as a move from the original path, and add the displaced document to the "possibly deleted" list.
**Server-side invariants:**
**10. The server must not `expect()` / panic on UTF-8 conversion — return a client error.**
In `update_text`, use `.context(...).map_err(client_error)?` instead of `.expect()` on `str::from_utf8()`.
**11. The create-merge parent content must be empty (`&Vec::new()`), not `latest_version.content`.**
An empty parent causes `reconcile("", existing, new)` to treat both sides as independent additions and merge them.
**12. `retryForever` must not retry 4xx HTTP errors.**
4xx errors indicate the request itself is wrong. Only 5xx errors (transient failures) are retried.
**13. The broadcast channel's `RecvError::Lagged` must be handled explicitly.**
Handle `Lagged` with a `warn!` log and `break`, not silently exit.
**14. `merge_with_stored_version` must not short-circuit when an idempotency key is provided.**
The key must be persisted even if content is identical — the short-circuit only applies to keyless updates.
**15. The idempotency key check in `create_document` must skip deleted documents.**
Returning a deleted version causes the client to delete the user's local file.
### File Operations Abstraction
`FileOperations` has an `ensureClearPath` method that renames existing files to `(1).md`, `(2).md` etc. if a file already exists at the target path. This prevents data loss but can create apparent duplicates if the sync logic doesn't handle it.
The `write` method does a 3-way merge: `write(path, oldContent, newContent)`. It reads the current file, computes a diff from `oldContent` to `newContent`, and applies that diff to the current file content. This preserves local changes that happened between the read and write.
### E2E Test Configuration
The test client (`frontend/test-client/src/cli.ts`) runs 5 iterations of 9 test configurations per process. Tests assert: file system consistency between agents AND no duplicate content across files.
**Running E2E**: Requires a server running with `config-e2e.yml`. Always clean the server databases before running.
**Known issue**: The deterministic test harness can hang during shared server cleanup when transitioning from regular tests to server-pause tests. This is an infrastructure issue, not a sync bug.
Never ever run git commands
Never put a full stop at the end of a single sentence comment
always use British English spellings
Never ever do fallbacks
Style guide
Don't write `super::device_id_header::DeviceIdHeader` instead `use super::device_id_header::DeviceIdHeader;` and then just write `DeviceIdHeader`

View file

@ -0,0 +1,13 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>VaultLink2</title>
<link rel="icon" href="data:," />
</head>
<body>
<div id="app"></div>
<script type="module" src="/src/main.ts"></script>
</body>
</html>

View file

@ -0,0 +1,16 @@
{
"name": "history-ui",
"version": "0.14.0",
"private": true,
"type": "module",
"scripts": {
"dev": "vite dev --host 0.0.0.0",
"build": "vite build",
"test": "echo 'no tests yet'"
},
"devDependencies": {
"@sveltejs/vite-plugin-svelte": "^5.0.0",
"svelte": "^5.0.0",
"vite": "^6.0.0"
}
}

View file

@ -0,0 +1,78 @@
<script lang="ts">
import { auth, nav, toasts } from "./lib/stores.svelte";
import { listVaults } from "./lib/api";
import Login from "./components/Login.svelte";
import VaultPicker from "./components/VaultPicker.svelte";
import Dashboard from "./components/Dashboard.svelte";
import ToastContainer from "./components/ToastContainer.svelte";
let restoring = $state(true);
$effect(() => {
const saved = auth.tryRestore();
if (!saved) {
restoring = false;
return;
}
listVaults(saved.token)
.then((response) => {
auth.authenticate(
saved.token,
response.userName,
response.vaults
);
if (
saved.vaultId &&
response.vaults.some(
(v) => v.name === saved.vaultId
)
) {
auth.selectVault(saved.vaultId);
}
restoring = false;
})
.catch(() => {
restoring = false;
});
});
</script>
{#if restoring}
<div class="loading-screen">
<div class="spinner"></div>
</div>
{:else if !auth.token}
<Login />
{:else if !auth.isAuthenticated}
<VaultPicker />
{:else}
<Dashboard
selectedDocumentId={nav.current.kind === "document" ? nav.current.documentId : undefined}
/>
{/if}
<ToastContainer />
<style>
.loading-screen {
display: flex;
align-items: center;
justify-content: center;
height: 100%;
}
.spinner {
width: 32px;
height: 32px;
border: 3px solid var(--bg-tertiary);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
@keyframes spin {
to {
transform: rotate(360deg);
}
}
</style>

View file

@ -0,0 +1,101 @@
:root {
--bg: #0d1117;
--bg-secondary: #161b22;
--bg-tertiary: #21262d;
--bg-hover: #30363d;
--border: #30363d;
--border-light: #21262d;
--text: #e6edf3;
--text-muted: #8b949e;
--text-subtle: #6e7681;
--accent: #58a6ff;
--accent-hover: #79c0ff;
--green: #3fb950;
--green-bg: rgba(63, 185, 80, 0.15);
--red: #f85149;
--red-bg: rgba(248, 81, 73, 0.15);
--orange: #d29922;
--orange-bg: rgba(210, 153, 34, 0.15);
--purple: #bc8cff;
--purple-bg: rgba(188, 140, 255, 0.15);
--blue: #58a6ff;
--blue-bg: rgba(88, 166, 255, 0.15);
--mono: "SF Mono", "Fira Code", "Fira Mono", Menlo, Consolas, monospace;
--sans: -apple-system, BlinkMacSystemFont, "Segoe UI", Noto Sans, Helvetica, Arial, sans-serif;
--radius: 6px;
--radius-sm: 4px;
--shadow: 0 1px 3px rgba(0, 0, 0, 0.3);
}
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
html, body, #app {
height: 100%;
width: 100%;
overflow: hidden;
}
body {
font-family: var(--sans);
font-size: 14px;
line-height: 1.5;
color: var(--text);
background: var(--bg);
-webkit-font-smoothing: antialiased;
}
button {
font-family: inherit;
font-size: inherit;
cursor: pointer;
border: none;
background: none;
color: inherit;
}
input {
font-family: inherit;
font-size: inherit;
color: inherit;
background: var(--bg);
border: 1px solid var(--border);
border-radius: var(--radius);
padding: 8px 12px;
outline: none;
transition: border-color 0.15s;
}
input:focus {
border-color: var(--accent);
}
a {
color: var(--accent);
text-decoration: none;
}
a:hover {
text-decoration: underline;
}
::-webkit-scrollbar {
width: 8px;
height: 8px;
}
::-webkit-scrollbar-track {
background: transparent;
}
::-webkit-scrollbar-thumb {
background: var(--bg-tertiary);
border-radius: 4px;
}
::-webkit-scrollbar-thumb:hover {
background: var(--bg-hover);
}

View file

@ -0,0 +1,346 @@
<script lang="ts">
import type { VersionEvent } from "../lib/types";
import {
absoluteTime,
formatBytes
} from "../lib/stores.svelte";
interface Props {
versions: VersionEvent[];
loading: boolean;
hasMore: boolean;
onLoadMore: () => void;
onSelectDocument: (documentId: string) => void;
onTimeTravel: (vaultUpdateId: number) => void;
}
let {
versions,
loading,
hasMore,
onLoadMore,
onSelectDocument,
onTimeTravel
}: Props = $props();
function timeOfDay(dateStr: string): string {
return new Date(dateStr).toLocaleTimeString("en-US", {
hour: "numeric",
minute: "2-digit"
});
}
// Group by day
let grouped = $derived.by(() => {
const groups: { date: string; items: VersionEvent[] }[] = [];
const sortedDesc = [...versions].sort(
(a, b) => b.vaultUpdateId - a.vaultUpdateId
);
for (const v of sortedDesc) {
const date = new Date(v.updatedDate).toLocaleDateString(
"en-US",
{ month: "long", day: "numeric", year: "numeric" }
);
const last = groups.at(-1);
if (last && last.date === date) {
last.items.push(v);
} else {
groups.push({ date, items: [v] });
}
}
return groups;
});
const actionColors: Record<string, string> = {
created: "var(--green)",
updated: "var(--blue)",
renamed: "var(--orange)",
deleted: "var(--red)",
restored: "var(--purple)"
};
const actionBgColors: Record<string, string> = {
created: "var(--green-bg)",
updated: "var(--blue-bg)",
renamed: "var(--orange-bg)",
deleted: "var(--red-bg)",
restored: "var(--purple-bg)"
};
</script>
<div class="feed">
{#if loading && versions.length === 0}
<div class="feed-loading">Loading activity...</div>
{:else if versions.length === 0}
<div class="feed-empty">
No activity yet. Documents will appear here as sync clients
make changes.
</div>
{:else}
{#each grouped as group}
<div class="day-group">
<div class="day-header">{group.date}</div>
<div class="items-list">
{#each group.items as event}
<div class="feed-item">
<button
class="feed-item-main"
onclick={() =>
onSelectDocument(event.documentId)}
>
<div class="feed-timeline">
<div
class="timeline-dot"
style="background: {actionColors[
event.action
]}"
></div>
</div>
<div class="feed-content">
<div class="feed-header">
<span
class="action-pill"
style="color: {actionColors[
event.action
]}; background: {actionBgColors[
event.action
]}"
>
{event.action}
</span>
<span class="feed-path">
{#if event.action === "renamed" && event.previousPath}
<span class="prev-path"
>{event.previousPath}</span
>
<span class="arrow"
>&rarr;</span
>
{/if}
<span
class:deleted={event.action ===
"deleted"}
>
{event.relativePath}
</span>
</span>
</div>
<div class="feed-meta">
<span class="feed-user"
>{event.userId}</span
>
<span class="feed-dot"
>&middot;</span
>
<span class="feed-size"
>{formatBytes(
event.contentSize
)}</span
>
</div>
</div>
</button>
<button
class="feed-time-btn"
title="Time travel to {absoluteTime(event.updatedDate)}"
onclick={(e) => {
e.stopPropagation();
onTimeTravel(event.vaultUpdateId);
}}
>
{timeOfDay(event.updatedDate)}
</button>
</div>
{/each}
</div>
</div>
{/each}
{#if hasMore}
<div class="load-more">
<button class="load-more-btn" onclick={onLoadMore}>
Load older activity
</button>
</div>
{/if}
{/if}
</div>
<style>
.feed {
flex: 1;
overflow-y: auto;
padding: 0 0 16px;
}
.feed-loading,
.feed-empty {
padding: 48px 16px;
text-align: center;
color: var(--text-muted);
}
.day-group {
margin-bottom: 8px;
}
.day-header {
position: sticky;
top: 0;
z-index: 1;
padding: 8px 16px;
font-size: 12px;
font-weight: 600;
color: var(--text-muted);
background: var(--bg);
border-bottom: 1px solid var(--border-light);
text-transform: uppercase;
letter-spacing: 0.5px;
}
.feed-item {
display: flex;
align-items: stretch;
width: 100%;
transition: background 0.1s;
}
.feed-item:hover {
background: var(--bg-hover);
}
.feed-item-main {
display: flex;
gap: 12px;
flex: 1;
min-width: 0;
padding: 10px 0 10px 16px;
text-align: left;
}
.items-list {
position: relative;
}
.items-list::before {
content: "";
position: absolute;
left: 21px;
top: 0;
bottom: 0;
width: 2px;
background: var(--border);
}
.feed-timeline {
display: flex;
flex-direction: column;
align-items: center;
width: 12px;
flex-shrink: 0;
padding-top: 6px;
position: relative;
}
.timeline-dot {
width: 8px;
height: 8px;
border-radius: 50%;
flex-shrink: 0;
}
.feed-content {
flex: 1;
min-width: 0;
}
.feed-header {
display: flex;
align-items: center;
gap: 8px;
flex-wrap: wrap;
}
.action-pill {
font-size: 11px;
font-weight: 600;
padding: 1px 8px;
border-radius: 10px;
text-transform: uppercase;
letter-spacing: 0.3px;
flex-shrink: 0;
}
.feed-path {
font-family: var(--mono);
font-size: 13px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.prev-path {
color: var(--text-muted);
text-decoration: line-through;
}
.arrow {
color: var(--text-subtle);
margin: 0 4px;
}
.deleted {
text-decoration: line-through;
opacity: 0.6;
}
.feed-meta {
display: flex;
align-items: center;
gap: 6px;
margin-top: 4px;
font-size: 12px;
color: var(--text-muted);
}
.feed-dot {
color: var(--text-subtle);
}
.feed-time-btn {
display: flex;
align-items: center;
padding: 0 16px;
font-size: 12px;
font-family: var(--mono);
color: var(--text-muted);
white-space: nowrap;
flex-shrink: 0;
border-left: 1px solid transparent;
transition: color 0.15s, border-color 0.15s;
}
.feed-time-btn:hover {
color: var(--accent);
border-left-color: var(--border-light);
}
.load-more {
padding: 16px;
text-align: center;
}
.load-more-btn {
padding: 8px 20px;
font-size: 13px;
color: var(--accent);
border: 1px solid var(--border);
border-radius: var(--radius);
transition: background 0.15s;
}
.load-more-btn:hover {
background: var(--bg-hover);
}
</style>

View file

@ -0,0 +1,167 @@
<script lang="ts">
interface Props {
title: string;
message: string;
confirmLabel: string;
destructive?: boolean;
loading?: boolean;
onConfirm: () => void;
onCancel: () => void;
}
let {
title,
message,
confirmLabel,
destructive = false,
loading = false,
onConfirm,
onCancel
}: Props = $props();
function handleKeydown(e: KeyboardEvent) {
if (e.key === "Escape") onCancel();
}
</script>
<svelte:window on:keydown={handleKeydown} />
<!-- svelte-ignore a11y_click_events_have_key_events -->
<div class="backdrop" onclick={onCancel} role="presentation">
<!-- svelte-ignore a11y_click_events_have_key_events -->
<!-- svelte-ignore a11y_interactive_supports_focus -->
<div
class="dialog"
onclick={(e) => e.stopPropagation()}
role="dialog"
aria-label={title}
>
<h3 class="dialog-title">{title}</h3>
<p class="dialog-message">{message}</p>
<div class="dialog-actions">
<button class="btn-cancel" onclick={onCancel} disabled={loading}>
Cancel
</button>
<button
class="btn-confirm"
class:destructive
onclick={onConfirm}
disabled={loading}
>
{#if loading}
<span class="btn-spinner"></span>
{/if}
{confirmLabel}
</button>
</div>
</div>
</div>
<style>
.backdrop {
position: fixed;
inset: 0;
z-index: 1000;
background: rgba(0, 0, 0, 0.6);
display: flex;
align-items: center;
justify-content: center;
animation: fade-in 0.15s ease-out;
}
.dialog {
background: var(--bg-secondary);
border: 1px solid var(--border);
border-radius: 12px;
padding: 24px;
max-width: 480px;
width: calc(100% - 32px);
box-shadow: 0 8px 32px rgba(0, 0, 0, 0.4);
animation: scale-in 0.2s ease-out;
}
.dialog-title {
font-size: 16px;
font-weight: 600;
margin-bottom: 12px;
}
.dialog-message {
font-size: 14px;
color: var(--text-muted);
line-height: 1.5;
margin-bottom: 24px;
}
.dialog-actions {
display: flex;
justify-content: flex-end;
gap: 8px;
}
.btn-cancel {
padding: 8px 16px;
border: 1px solid var(--border);
border-radius: var(--radius);
color: var(--text-muted);
transition: background 0.15s;
}
.btn-cancel:hover:not(:disabled) {
background: var(--bg-hover);
color: var(--text);
}
.btn-confirm {
padding: 8px 16px;
background: var(--accent);
color: #fff;
font-weight: 600;
border-radius: var(--radius);
transition: background 0.15s;
display: flex;
align-items: center;
gap: 6px;
}
.btn-confirm:hover:not(:disabled) {
background: var(--accent-hover);
}
.btn-confirm.destructive {
background: var(--red);
}
.btn-confirm.destructive:hover:not(:disabled) {
background: #f97583;
}
.btn-confirm:disabled,
.btn-cancel:disabled {
opacity: 0.6;
cursor: not-allowed;
}
.btn-spinner {
width: 14px;
height: 14px;
border: 2px solid rgba(255, 255, 255, 0.3);
border-top-color: #fff;
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
@keyframes fade-in {
from { opacity: 0; }
to { opacity: 1; }
}
@keyframes scale-in {
from { transform: scale(0.95); opacity: 0; }
to { transform: scale(1); opacity: 1; }
}
@keyframes spin {
to { transform: rotate(360deg); }
}
</style>

View file

@ -0,0 +1,511 @@
<script lang="ts">
import {
auth,
nav,
toasts,
buildTree,
enrichVersions,
relativeTime,
formatBytes,
type View
} from "../lib/stores.svelte";
import type {
DocumentVersionWithoutContent,
VaultHistoryResponse,
VersionEvent,
TreeNode
} from "../lib/types";
import FileTree from "./FileTree.svelte";
import ActivityFeed from "./ActivityFeed.svelte";
import DocumentDetail from "./DocumentDetail.svelte";
import TimeSlider from "./TimeSlider.svelte";
import Header from "./Header.svelte";
interface Props {
selectedDocumentId?: string;
}
let { selectedDocumentId }: Props = $props();
// Data
let latestDocuments = $state<DocumentVersionWithoutContent[]>([]);
let historyVersions = $state<DocumentVersionWithoutContent[]>([]);
let historyHasMore = $state(false);
let loadingDocs = $state(true);
let loadingHistory = $state(true);
let showDeleted = $state(false);
let searchQuery = $state("");
let activeTab = $state<"activity" | "files">("activity");
// Time travel
let maxUpdateId = $state(0);
let minUpdateId = $state(0);
let timeSliderValue = $state<number | null>(null);
// Derived
let tree = $derived(buildTree(latestDocuments, showDeleted));
let enrichedHistory = $derived(enrichVersions(historyVersions));
let stats = $derived({
totalDocs: latestDocuments.filter((d) => !d.isDeleted).length,
deletedDocs: latestDocuments.filter((d) => d.isDeleted).length,
totalSize: latestDocuments
.filter((d) => !d.isDeleted)
.reduce((sum, d) => sum + d.contentSize, 0),
users: [...new Set(latestDocuments.map((d) => d.userId))]
});
let filteredTree = $derived.by(() => {
if (!searchQuery) return tree;
return filterTree(tree, searchQuery.toLowerCase());
});
function filterTree(node: TreeNode, query: string): TreeNode {
if (!node.isFolder) {
return node.name.toLowerCase().includes(query) ? node : { ...node, children: [] };
}
const filteredChildren = node.children
.map((c) => filterTree(c, query))
.filter((c) => c.isFolder ? c.children.length > 0 : true)
.filter((c) => !c.isFolder || c.children.length > 0);
return { ...node, children: filteredChildren };
}
// Time travel: compute vault state at a given updateId
let timeFilteredDocs = $derived.by(() => {
if (timeSliderValue === null || timeSliderValue >= maxUpdateId) {
return latestDocuments;
}
// From all history, find the latest version per documentId at or before timeSliderValue
const byDoc = new Map<string, DocumentVersionWithoutContent>();
for (const v of historyVersions) {
if (v.vaultUpdateId <= timeSliderValue) {
const existing = byDoc.get(v.documentId);
if (
!existing ||
v.vaultUpdateId > existing.vaultUpdateId
) {
byDoc.set(v.documentId, v);
}
}
}
return [...byDoc.values()];
});
let timeFilteredTree = $derived(
buildTree(
timeSliderValue !== null && timeSliderValue < maxUpdateId
? timeFilteredDocs
: latestDocuments,
showDeleted
)
);
let displayTree = $derived(
searchQuery ? filteredTree : timeFilteredTree
);
// Load data
async function loadData() {
const api = auth.api;
if (!api) return;
loadingDocs = true;
loadingHistory = true;
api.ping().then((ping) => {
auth.serverVersion = ping.serverVersion;
});
try {
const response = await api.fetchLatestDocuments();
latestDocuments = response.latestDocuments;
maxUpdateId = Number(response.lastUpdateId);
} catch (e) {
toasts.add("Failed to load documents", "error");
} finally {
loadingDocs = false;
}
try {
const response = await api.fetchVaultHistory(500);
historyVersions = response.versions;
historyHasMore = response.hasMore;
if (historyVersions.length > 0) {
minUpdateId = Math.min(
...historyVersions.map((v) => v.vaultUpdateId)
);
maxUpdateId = Math.max(
maxUpdateId,
Math.max(
...historyVersions.map((v) => v.vaultUpdateId)
)
);
}
} catch (e) {
toasts.add("Failed to load history", "error");
} finally {
loadingHistory = false;
}
}
async function loadMoreHistory() {
const api = auth.api;
if (!api || !historyHasMore) return;
const oldest = Math.min(
...historyVersions.map((v) => v.vaultUpdateId)
);
try {
const response = await api.fetchVaultHistory(500, oldest);
historyVersions = [...historyVersions, ...response.versions];
historyHasMore = response.hasMore;
minUpdateId = Math.min(
minUpdateId,
...response.versions.map((v) => v.vaultUpdateId)
);
} catch {
toasts.add("Failed to load more history", "error");
}
}
function selectDocument(documentId: string) {
nav.goto({ kind: "document", documentId });
}
function handleRefresh() {
loadData();
}
$effect(() => {
if (auth.isAuthenticated) {
loadData();
}
});
</script>
<div class="dashboard">
<Header
vaultId={auth.vaultId}
serverVersion={auth.serverVersion}
onRefresh={handleRefresh}
/>
<div class="main-layout">
<!-- Sidebar -->
<aside class="sidebar">
{#if !loadingDocs}
<div class="sidebar-stats">
<div class="stat">
<span class="stat-value">{stats.totalDocs}</span>
<span class="stat-label">files</span>
</div>
<div class="stat">
<span class="stat-value"
>{formatBytes(stats.totalSize)}</span
>
<span class="stat-label">total</span>
</div>
<div class="stat">
<span class="stat-value">{stats.users.length}</span>
<span class="stat-label"
>user{stats.users.length !== 1 ? "s" : ""}</span
>
</div>
</div>
{/if}
<div class="sidebar-search">
<input
type="text"
placeholder="Filter files..."
bind:value={searchQuery}
/>
</div>
<div class="sidebar-controls">
<label class="toggle-label">
<input
type="checkbox"
bind:checked={showDeleted}
/>
Show deleted
</label>
</div>
<div class="sidebar-tree">
{#if loadingDocs}
<div class="loading-placeholder">Loading...</div>
{:else}
<FileTree
node={displayTree}
selectedId={selectedDocumentId ?? null}
onSelect={selectDocument}
/>
{/if}
</div>
</aside>
<!-- Main content -->
<main class="content">
{#if maxUpdateId > 0}
<div class="time-slider-container">
<TimeSlider
min={minUpdateId}
max={maxUpdateId}
value={timeSliderValue}
versions={historyVersions}
onchange={(v) => {
timeSliderValue = v;
}}
/>
</div>
{/if}
{#if selectedDocumentId}
<DocumentDetail
documentId={selectedDocumentId}
onClose={() => nav.goHome()}
onRestore={handleRefresh}
/>
{:else}
<div class="tabs">
<button
class="tab"
class:active={activeTab === "activity"}
onclick={() => (activeTab = "activity")}
>
Activity
</button>
<button
class="tab"
class:active={activeTab === "files"}
onclick={() => (activeTab = "files")}
>
Files
</button>
</div>
{#if activeTab === "activity"}
<ActivityFeed
versions={enrichedHistory}
loading={loadingHistory}
hasMore={historyHasMore}
onLoadMore={loadMoreHistory}
onSelectDocument={selectDocument}
onTimeTravel={(id) => {
timeSliderValue = id >= maxUpdateId ? null : id;
}}
/>
{:else}
<div class="file-list">
{#each latestDocuments
.filter((d) => showDeleted || !d.isDeleted)
.sort((a, b) => b.vaultUpdateId - a.vaultUpdateId) as doc}
<button
class="file-row"
class:deleted={doc.isDeleted}
onclick={() =>
selectDocument(doc.documentId)}
>
<span class="file-icon"
>{doc.isDeleted
? "🗑"
: "📄"}</span
>
<span class="file-path"
>{doc.relativePath}</span
>
<span class="file-meta">
{formatBytes(doc.contentSize)}
&middot;
{doc.userId}
&middot;
{relativeTime(doc.updatedDate)}
</span>
</button>
{/each}
</div>
{/if}
{/if}
</main>
</div>
</div>
<style>
.dashboard {
display: flex;
flex-direction: column;
height: 100%;
}
.main-layout {
display: flex;
flex: 1;
overflow: hidden;
}
.sidebar {
width: 280px;
min-width: 280px;
border-right: 1px solid var(--border);
display: flex;
flex-direction: column;
background: var(--bg-secondary);
overflow: hidden;
}
.sidebar-stats {
display: flex;
gap: 12px;
padding: 12px 16px;
border-bottom: 1px solid var(--border);
}
.stat {
display: flex;
flex-direction: column;
align-items: center;
flex: 1;
}
.stat-value {
font-size: 16px;
font-weight: 600;
color: var(--text);
}
.stat-label {
font-size: 11px;
color: var(--text-muted);
text-transform: uppercase;
letter-spacing: 0.5px;
}
.sidebar-search {
padding: 8px 12px;
}
.sidebar-search input {
width: 100%;
font-size: 13px;
padding: 6px 10px;
}
.sidebar-controls {
padding: 4px 16px 8px;
}
.toggle-label {
font-size: 12px;
color: var(--text-muted);
display: flex;
align-items: center;
gap: 6px;
cursor: pointer;
}
.toggle-label input[type="checkbox"] {
width: auto;
accent-color: var(--accent);
}
.sidebar-tree {
flex: 1;
overflow-y: auto;
padding: 4px 0;
}
.loading-placeholder {
padding: 16px;
color: var(--text-muted);
text-align: center;
font-size: 13px;
}
.content {
flex: 1;
display: flex;
flex-direction: column;
overflow: hidden;
}
.time-slider-container {
padding: 8px 16px;
border-bottom: 1px solid var(--border);
background: var(--bg-secondary);
}
.tabs {
display: flex;
border-bottom: 1px solid var(--border);
background: var(--bg-secondary);
padding: 0 16px;
}
.tab {
padding: 10px 16px;
font-size: 13px;
font-weight: 500;
color: var(--text-muted);
border-bottom: 2px solid transparent;
transition: color 0.15s, border-color 0.15s;
}
.tab:hover {
color: var(--text);
}
.tab.active {
color: var(--text);
border-bottom-color: var(--accent);
}
.file-list {
flex: 1;
overflow-y: auto;
padding: 4px 0;
}
.file-row {
display: flex;
align-items: center;
gap: 8px;
width: 100%;
padding: 8px 16px;
text-align: left;
transition: background 0.1s;
}
.file-row:hover {
background: var(--bg-hover);
}
.file-row.deleted {
opacity: 0.5;
}
.file-row.deleted .file-path {
text-decoration: line-through;
}
.file-icon {
font-size: 16px;
flex-shrink: 0;
}
.file-path {
font-family: var(--mono);
font-size: 13px;
flex: 1;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.file-meta {
font-size: 12px;
color: var(--text-muted);
white-space: nowrap;
flex-shrink: 0;
}
</style>

View file

@ -0,0 +1,288 @@
<script lang="ts">
interface Props {
oldContent: string;
newContent: string;
oldLabel: string;
newLabel: string;
}
let { oldContent, newContent, oldLabel, newLabel }: Props = $props();
interface DiffLine {
type: "add" | "remove" | "context";
content: string;
oldLineNo: number | null;
newLineNo: number | null;
}
let diffLines = $derived.by((): DiffLine[] => {
const oldLines = oldContent.split("\n");
const newLines = newContent.split("\n");
// Simple line-by-line diff using LCS
const lines: DiffLine[] = [];
const lcs = computeLCS(oldLines, newLines);
let oi = 0;
let ni = 0;
let oldLineNo = 1;
let newLineNo = 1;
for (const match of lcs) {
// Remove lines before match
while (oi < match.oldIndex) {
lines.push({
type: "remove",
content: oldLines[oi],
oldLineNo: oldLineNo++,
newLineNo: null
});
oi++;
}
// Add lines before match
while (ni < match.newIndex) {
lines.push({
type: "add",
content: newLines[ni],
oldLineNo: null,
newLineNo: newLineNo++
});
ni++;
}
// Context line
lines.push({
type: "context",
content: oldLines[oi],
oldLineNo: oldLineNo++,
newLineNo: newLineNo++
});
oi++;
ni++;
}
// Remaining removes
while (oi < oldLines.length) {
lines.push({
type: "remove",
content: oldLines[oi],
oldLineNo: oldLineNo++,
newLineNo: null
});
oi++;
}
// Remaining adds
while (ni < newLines.length) {
lines.push({
type: "add",
content: newLines[ni],
oldLineNo: null,
newLineNo: newLineNo++
});
ni++;
}
return lines;
});
let stats = $derived({
added: diffLines.filter((l) => l.type === "add").length,
removed: diffLines.filter((l) => l.type === "remove").length
});
interface LCSMatch {
oldIndex: number;
newIndex: number;
}
function computeLCS(a: string[], b: string[]): LCSMatch[] {
const m = a.length;
const n = b.length;
// For large files, use a simpler approach
if (m * n > 1_000_000) {
return simpleDiff(a, b);
}
const dp: number[][] = Array.from({ length: m + 1 }, () =>
new Array(n + 1).fill(0)
);
for (let i = 1; i <= m; i++) {
for (let j = 1; j <= n; j++) {
if (a[i - 1] === b[j - 1]) {
dp[i][j] = dp[i - 1][j - 1] + 1;
} else {
dp[i][j] = Math.max(dp[i - 1][j], dp[i][j - 1]);
}
}
}
// Backtrack
const matches: LCSMatch[] = [];
let i = m;
let j = n;
while (i > 0 && j > 0) {
if (a[i - 1] === b[j - 1]) {
matches.unshift({ oldIndex: i - 1, newIndex: j - 1 });
i--;
j--;
} else if (dp[i - 1][j] > dp[i][j - 1]) {
i--;
} else {
j--;
}
}
return matches;
}
function simpleDiff(a: string[], b: string[]): LCSMatch[] {
// Hash-based matching for large files
const bMap = new Map<string, number[]>();
for (let j = 0; j < b.length; j++) {
const arr = bMap.get(b[j]);
if (arr) arr.push(j);
else bMap.set(b[j], [j]);
}
const matches: LCSMatch[] = [];
let lastJ = -1;
for (let i = 0; i < a.length; i++) {
const candidates = bMap.get(a[i]);
if (!candidates) continue;
for (const j of candidates) {
if (j > lastJ) {
matches.push({ oldIndex: i, newIndex: j });
lastJ = j;
break;
}
}
}
return matches;
}
</script>
<div class="diff-view">
<div class="diff-header">
<span class="diff-label">{oldLabel}</span>
<span class="diff-arrow">&rarr;</span>
<span class="diff-label">{newLabel}</span>
<span class="diff-stats">
<span class="diff-added">+{stats.added}</span>
<span class="diff-removed">-{stats.removed}</span>
</span>
</div>
<div class="diff-content">
{#each diffLines as line}
<div class="diff-line {line.type}">
<span class="line-no old-no">
{line.oldLineNo ?? ""}
</span>
<span class="line-no new-no">
{line.newLineNo ?? ""}
</span>
<span class="line-marker">
{#if line.type === "add"}+{:else if line.type === "remove"}-{:else}&nbsp;{/if}
</span>
<span class="line-content">{line.content}</span>
</div>
{/each}
</div>
</div>
<style>
.diff-view {
display: flex;
flex-direction: column;
height: 100%;
}
.diff-header {
display: flex;
align-items: center;
gap: 8px;
padding: 8px 16px;
background: var(--bg-secondary);
border-bottom: 1px solid var(--border);
flex-shrink: 0;
}
.diff-label {
font-family: var(--mono);
font-size: 12px;
color: var(--text-muted);
}
.diff-arrow {
color: var(--text-subtle);
}
.diff-stats {
margin-left: auto;
display: flex;
gap: 8px;
font-family: var(--mono);
font-size: 12px;
}
.diff-added {
color: var(--green);
}
.diff-removed {
color: var(--red);
}
.diff-content {
flex: 1;
overflow: auto;
font-family: var(--mono);
font-size: 13px;
line-height: 1.5;
}
.diff-line {
display: flex;
white-space: pre;
min-height: 20px;
}
.diff-line.add {
background: var(--green-bg);
}
.diff-line.remove {
background: var(--red-bg);
}
.line-no {
display: inline-block;
width: 48px;
text-align: right;
padding-right: 8px;
color: var(--text-subtle);
user-select: none;
flex-shrink: 0;
}
.line-marker {
display: inline-block;
width: 20px;
text-align: center;
flex-shrink: 0;
user-select: none;
}
.diff-line.add .line-marker {
color: var(--green);
}
.diff-line.remove .line-marker {
color: var(--red);
}
.line-content {
flex: 1;
padding-right: 16px;
}
</style>

View file

@ -0,0 +1,712 @@
<script lang="ts">
import {
auth,
toasts,
relativeTime,
absoluteTime,
formatBytes,
inferAction,
isTextFile,
isImageFile,
fileExtension
} from "../lib/stores.svelte";
import type {
DocumentVersionWithoutContent,
DocumentVersion,
ActionType
} from "../lib/types";
import DiffView from "./DiffView.svelte";
import ConfirmDialog from "./ConfirmDialog.svelte";
interface Props {
documentId: string;
onClose: () => void;
onRestore: () => void;
}
let { documentId, onClose, onRestore }: Props = $props();
let versions = $state<DocumentVersionWithoutContent[]>([]);
let loading = $state(true);
let selectedVersion = $state<DocumentVersionWithoutContent | null>(null);
let loadedContent = $state<string | null>(null);
let loadedContentBytes = $state<ArrayBuffer | null>(null);
let loadingContent = $state(false);
let activeTab = $state<"preview" | "diff">("preview");
// Diff state
let diffOldContent = $state<string | null>(null);
let diffNewContent = $state<string | null>(null);
let diffOldLabel = $state("");
let diffNewLabel = $state("");
// Restore state
let showRestoreDialog = $state(false);
let restoreTarget = $state<DocumentVersionWithoutContent | null>(null);
let restoring = $state(false);
let latest = $derived(versions.at(-1) ?? null);
let isDeleted = $derived(latest?.isDeleted ?? false);
let currentPath = $derived(latest?.relativePath ?? "");
// Derive action types
let versionEvents = $derived(
versions.map((v, i) => ({
version: v,
action: inferAction(v, i > 0 ? versions[i - 1] : undefined) as ActionType,
previousPath: i > 0 && versions[i - 1].relativePath !== v.relativePath
? versions[i - 1].relativePath
: undefined
}))
);
async function loadVersions() {
const api = auth.api;
if (!api) return;
loading = true;
try {
versions = await api.fetchDocumentVersions(documentId);
// Auto-select latest
if (versions.length > 0) {
await selectVersion(versions.at(-1)!);
}
} catch {
toasts.add("Failed to load document versions", "error");
} finally {
loading = false;
}
}
async function selectVersion(v: DocumentVersionWithoutContent) {
selectedVersion = v;
activeTab = "preview";
diffOldContent = null;
diffNewContent = null;
loadingContent = true;
loadedContent = null;
loadedContentBytes = null;
const api = auth.api;
if (!api) return;
try {
if (isTextFile(v.relativePath) || fileExtension(v.relativePath) === "") {
const fullVersion = await api.fetchDocumentVersion(
documentId,
v.vaultUpdateId
);
const bytes = Uint8Array.from(atob(fullVersion.contentBase64), c => c.charCodeAt(0));
const decoder = new TextDecoder("utf-8", { fatal: false });
loadedContent = decoder.decode(bytes);
loadedContentBytes = bytes.buffer;
} else if (isImageFile(v.relativePath)) {
loadedContentBytes = await api.fetchDocumentVersionContent(
documentId,
v.vaultUpdateId
);
} else {
loadedContentBytes = await api.fetchDocumentVersionContent(
documentId,
v.vaultUpdateId
);
}
} catch {
toasts.add("Failed to load content", "error");
} finally {
loadingContent = false;
}
}
async function showDiff(v: DocumentVersionWithoutContent, idx: number) {
const api = auth.api;
if (!api || idx === 0) return;
activeTab = "diff";
loadingContent = true;
const prev = versions[idx - 1];
try {
const [oldVer, newVer] = await Promise.all([
api.fetchDocumentVersion(documentId, prev.vaultUpdateId),
api.fetchDocumentVersion(documentId, v.vaultUpdateId)
]);
const decode = (b64: string) => {
const bytes = Uint8Array.from(atob(b64), c => c.charCodeAt(0));
return new TextDecoder("utf-8", { fatal: false }).decode(bytes);
};
diffOldContent = decode(oldVer.contentBase64);
diffNewContent = decode(newVer.contentBase64);
diffOldLabel = `v${prev.vaultUpdateId}`;
diffNewLabel = `v${v.vaultUpdateId}`;
} catch {
toasts.add("Failed to load diff", "error");
} finally {
loadingContent = false;
}
}
function confirmRestore(v: DocumentVersionWithoutContent) {
restoreTarget = v;
showRestoreDialog = true;
}
async function executeRestore() {
const api = auth.api;
if (!api || !restoreTarget) return;
restoring = true;
try {
await api.restoreVersion(
documentId,
restoreTarget.vaultUpdateId
);
toasts.add(
`Restored to version #${restoreTarget.vaultUpdateId}`,
"success"
);
showRestoreDialog = false;
restoreTarget = null;
onRestore();
await loadVersions();
} catch (e) {
toasts.add(`Restore failed: ${e}`, "error");
} finally {
restoring = false;
}
}
function getImageUrl(buffer: ArrayBuffer, path: string): string {
const ext = fileExtension(path);
const mimeMap: Record<string, string> = {
png: "image/png",
jpg: "image/jpeg",
jpeg: "image/jpeg",
gif: "image/gif",
webp: "image/webp",
svg: "image/svg+xml",
ico: "image/x-icon",
bmp: "image/bmp"
};
const mime = mimeMap[ext] ?? "application/octet-stream";
const blob = new Blob([buffer], { type: mime });
return URL.createObjectURL(blob);
}
$effect(() => {
loadVersions();
});
const actionColors: Record<string, string> = {
created: "var(--green)",
updated: "var(--blue)",
renamed: "var(--orange)",
deleted: "var(--red)",
restored: "var(--purple)"
};
const actionBgColors: Record<string, string> = {
created: "var(--green-bg)",
updated: "var(--blue-bg)",
renamed: "var(--orange-bg)",
deleted: "var(--red-bg)",
restored: "var(--purple-bg)"
};
</script>
<div class="detail">
<!-- Header -->
<div class="detail-header">
<button class="back-btn" onclick={onClose} title="Back">
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<path d="M19 12H5M12 19l-7-7 7-7" />
</svg>
</button>
<div class="header-info">
<div class="header-path">
<span class="path-text" class:deleted-path={isDeleted}>
{currentPath}
</span>
{#if isDeleted}
<span class="status-badge deleted-badge">Deleted</span>
{:else}
<span class="status-badge active-badge">Active</span>
{/if}
</div>
<div class="header-meta">
<span class="doc-id" title={documentId}>
{documentId.substring(0, 8)}...
</span>
{#if latest}
<span>&middot;</span>
<span>{versions.length} version{versions.length !== 1 ? "s" : ""}</span>
<span>&middot;</span>
<span>Last by {latest.userId}</span>
{/if}
</div>
</div>
</div>
{#if loading}
<div class="detail-loading">Loading versions...</div>
{:else}
<!-- Content area -->
<div class="detail-body">
<div class="content-panel">
{#if selectedVersion}
<div class="content-tabs">
<button
class="content-tab"
class:active={activeTab === "preview"}
onclick={() => (activeTab = "preview")}
>
Preview
</button>
<button
class="content-tab"
class:active={activeTab === "diff"}
onclick={() => {
if (selectedVersion) {
const idx = versions.indexOf(selectedVersion);
if (idx > 0) showDiff(selectedVersion, idx);
}
}}
disabled={versions.indexOf(selectedVersion) === 0}
>
Diff
</button>
<div class="content-tab-spacer"></div>
<span class="viewing-label">
Viewing v#{selectedVersion.vaultUpdateId}
&middot;
{relativeTime(selectedVersion.updatedDate)}
</span>
</div>
<div class="content-view">
{#if loadingContent}
<div class="content-loading">Loading content...</div>
{:else if activeTab === "diff" && diffOldContent !== null && diffNewContent !== null}
<DiffView
oldContent={diffOldContent}
newContent={diffNewContent}
oldLabel={diffOldLabel}
newLabel={diffNewLabel}
/>
{:else if activeTab === "preview"}
{#if isTextFile(selectedVersion.relativePath) || fileExtension(selectedVersion.relativePath) === ""}
<pre class="text-content">{loadedContent ?? ""}</pre>
{:else if isImageFile(selectedVersion.relativePath) && loadedContentBytes}
<div class="image-preview">
<img
src={getImageUrl(loadedContentBytes, selectedVersion.relativePath)}
alt={selectedVersion.relativePath}
/>
</div>
{:else}
<div class="binary-placeholder">
<div class="binary-icon">📦</div>
<div class="binary-label">Binary file</div>
<div class="binary-size">
{formatBytes(selectedVersion.contentSize)}
</div>
</div>
{/if}
{/if}
</div>
{/if}
</div>
<!-- Version timeline -->
<div class="version-panel">
<div class="version-panel-header">Version History</div>
<div class="version-list">
{#each [...versionEvents].reverse() as event, i}
{@const v = event.version}
{@const isSelected = selectedVersion?.vaultUpdateId === v.vaultUpdateId}
<div class="version-item" class:selected={isSelected}>
<button
class="version-main"
onclick={() => selectVersion(v)}
>
<div class="version-left">
<span class="version-id">#{v.vaultUpdateId}</span>
<span
class="version-action"
style="color: {actionColors[event.action]}; background: {actionBgColors[event.action]}"
>
{event.action}
</span>
</div>
<div class="version-right">
<span class="version-user">{v.userId}</span>
<span
class="version-time"
title={absoluteTime(v.updatedDate)}
>
{relativeTime(v.updatedDate)}
</span>
<span class="version-size">{formatBytes(v.contentSize)}</span>
</div>
</button>
{#if event.previousPath}
<div class="version-rename">
{event.previousPath} &rarr; {v.relativePath}
</div>
{/if}
<div class="version-actions">
{#if i < versionEvents.length - 1}
<button
class="version-btn"
onclick={() => {
const realIdx = versions.indexOf(v);
showDiff(v, realIdx);
}}
>
Diff
</button>
{/if}
{#if v !== latest}
<button
class="version-btn restore-btn"
onclick={() => confirmRestore(v)}
>
Restore
</button>
{/if}
</div>
</div>
{/each}
</div>
</div>
</div>
{/if}
</div>
{#if showRestoreDialog && restoreTarget}
<ConfirmDialog
title="Restore Version"
message={`Restore "${currentPath}" to version #${restoreTarget.vaultUpdateId} from ${absoluteTime(restoreTarget.updatedDate)}? This creates a new version with the old content. Current content is preserved in history.`}
confirmLabel="Restore"
destructive={false}
loading={restoring}
onConfirm={executeRestore}
onCancel={() => {
showRestoreDialog = false;
restoreTarget = null;
}}
/>
{/if}
<style>
.detail {
display: flex;
flex-direction: column;
height: 100%;
overflow: hidden;
}
.detail-header {
display: flex;
align-items: center;
gap: 12px;
padding: 12px 16px;
border-bottom: 1px solid var(--border);
background: var(--bg-secondary);
flex-shrink: 0;
}
.back-btn {
padding: 6px;
border-radius: var(--radius-sm);
color: var(--text-muted);
transition: color 0.15s, background 0.15s;
flex-shrink: 0;
}
.back-btn:hover {
color: var(--text);
background: var(--bg-hover);
}
.header-info {
flex: 1;
min-width: 0;
}
.header-path {
display: flex;
align-items: center;
gap: 8px;
}
.path-text {
font-family: var(--mono);
font-size: 15px;
font-weight: 600;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.deleted-path {
text-decoration: line-through;
opacity: 0.6;
}
.status-badge {
font-size: 10px;
font-weight: 600;
padding: 1px 8px;
border-radius: 10px;
text-transform: uppercase;
flex-shrink: 0;
}
.active-badge {
color: var(--green);
background: var(--green-bg);
}
.deleted-badge {
color: var(--red);
background: var(--red-bg);
}
.header-meta {
font-size: 12px;
color: var(--text-muted);
margin-top: 2px;
display: flex;
gap: 6px;
}
.doc-id {
font-family: var(--mono);
cursor: help;
}
.detail-loading {
padding: 48px;
text-align: center;
color: var(--text-muted);
}
.detail-body {
display: flex;
flex: 1;
overflow: hidden;
}
.content-panel {
flex: 1;
display: flex;
flex-direction: column;
overflow: hidden;
}
.content-tabs {
display: flex;
align-items: center;
padding: 0 16px;
border-bottom: 1px solid var(--border);
background: var(--bg);
flex-shrink: 0;
}
.content-tab {
padding: 8px 12px;
font-size: 12px;
font-weight: 500;
color: var(--text-muted);
border-bottom: 2px solid transparent;
transition: color 0.15s, border-color 0.15s;
}
.content-tab:hover:not(:disabled) {
color: var(--text);
}
.content-tab.active {
color: var(--text);
border-bottom-color: var(--accent);
}
.content-tab:disabled {
opacity: 0.3;
cursor: not-allowed;
}
.content-tab-spacer {
flex: 1;
}
.viewing-label {
font-size: 12px;
color: var(--text-subtle);
font-family: var(--mono);
}
.content-view {
flex: 1;
overflow: auto;
}
.content-loading {
padding: 48px;
text-align: center;
color: var(--text-muted);
}
.text-content {
padding: 16px;
font-family: var(--mono);
font-size: 13px;
line-height: 1.6;
white-space: pre-wrap;
word-break: break-word;
tab-size: 4;
}
.image-preview {
padding: 16px;
display: flex;
align-items: center;
justify-content: center;
}
.image-preview img {
max-width: 100%;
max-height: 60vh;
border-radius: var(--radius);
border: 1px solid var(--border);
}
.binary-placeholder {
padding: 64px;
text-align: center;
color: var(--text-muted);
}
.binary-icon {
font-size: 48px;
margin-bottom: 12px;
}
.binary-label {
font-size: 16px;
font-weight: 500;
}
.binary-size {
font-size: 14px;
margin-top: 4px;
}
/* Version panel */
.version-panel {
width: 320px;
min-width: 320px;
border-left: 1px solid var(--border);
display: flex;
flex-direction: column;
overflow: hidden;
background: var(--bg-secondary);
}
.version-panel-header {
padding: 10px 16px;
font-size: 12px;
font-weight: 600;
color: var(--text-muted);
text-transform: uppercase;
letter-spacing: 0.5px;
border-bottom: 1px solid var(--border);
flex-shrink: 0;
}
.version-list {
flex: 1;
overflow-y: auto;
}
.version-item {
border-bottom: 1px solid var(--border-light);
padding: 8px 12px;
transition: background 0.1s;
}
.version-item:hover {
background: var(--bg-hover);
}
.version-item.selected {
background: var(--blue-bg);
}
.version-main {
width: 100%;
text-align: left;
}
.version-left {
display: flex;
align-items: center;
gap: 8px;
margin-bottom: 4px;
}
.version-id {
font-family: var(--mono);
font-size: 13px;
font-weight: 600;
color: var(--text);
}
.version-action {
font-size: 10px;
font-weight: 600;
padding: 0 6px;
border-radius: 8px;
text-transform: uppercase;
}
.version-right {
display: flex;
align-items: center;
gap: 6px;
font-size: 11px;
color: var(--text-muted);
}
.version-rename {
font-size: 11px;
color: var(--orange);
font-family: var(--mono);
margin: 4px 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.version-actions {
display: flex;
gap: 8px;
margin-top: 6px;
}
.version-btn {
font-size: 11px;
color: var(--accent);
padding: 2px 8px;
border: 1px solid var(--border);
border-radius: var(--radius-sm);
transition: background 0.15s;
}
.version-btn:hover {
background: var(--bg-hover);
}
.restore-btn {
color: var(--orange);
}
</style>

View file

@ -0,0 +1,124 @@
<script lang="ts">
import type { TreeNode } from "../lib/types";
import FileTree from "./FileTree.svelte";
interface Props {
node: TreeNode;
selectedId: string | null;
onSelect: (documentId: string) => void;
depth?: number;
}
let { node, selectedId, onSelect, depth = 0 }: Props = $props();
let expanded = $state<Record<string, boolean>>({});
function toggle(path: string) {
expanded[path] = !expanded[path];
}
function isExpanded(path: string): boolean {
return expanded[path] ?? true;
}
</script>
{#if node.isFolder && depth === 0}
{#each node.children as child}
<FileTree
node={child}
{selectedId}
{onSelect}
depth={depth + 1}
/>
{/each}
{:else if node.isFolder}
<div class="tree-folder">
<button
class="tree-item folder"
style="padding-left: {depth * 16}px"
onclick={() => toggle(node.path)}
>
<span class="expand-icon"
>{isExpanded(node.path) ? "▾" : "▸"}</span
>
<span class="folder-icon">📁</span>
<span class="node-name">{node.name}</span>
</button>
{#if isExpanded(node.path)}
{#each node.children as child}
<FileTree
node={child}
{selectedId}
{onSelect}
depth={depth + 1}
/>
{/each}
{/if}
</div>
{:else}
<button
class="tree-item file"
class:selected={node.document?.documentId === selectedId}
class:deleted={node.isDeleted}
style="padding-left: {depth * 16 + 8}px"
onclick={() =>
node.document && onSelect(node.document.documentId)}
>
<span class="file-icon">{node.isDeleted ? "○" : "●"}</span>
<span class="node-name">{node.name}</span>
</button>
{/if}
<style>
.tree-item {
display: flex;
align-items: center;
gap: 6px;
width: 100%;
padding: 3px 12px;
font-size: 13px;
text-align: left;
transition: background 0.1s;
white-space: nowrap;
overflow: hidden;
}
.tree-item:hover {
background: var(--bg-hover);
}
.tree-item.selected {
background: var(--blue-bg);
}
.tree-item.deleted {
opacity: 0.4;
}
.tree-item.deleted .node-name {
text-decoration: line-through;
}
.expand-icon {
font-size: 10px;
width: 12px;
flex-shrink: 0;
color: var(--text-muted);
}
.folder-icon {
font-size: 14px;
flex-shrink: 0;
}
.file-icon {
font-size: 8px;
flex-shrink: 0;
color: var(--text-subtle);
}
.node-name {
overflow: hidden;
text-overflow: ellipsis;
}
</style>

View file

@ -0,0 +1,144 @@
<script lang="ts">
import { auth } from "../lib/stores.svelte";
interface Props {
vaultId: string;
serverVersion: string;
onRefresh: () => void;
}
let { vaultId, serverVersion, onRefresh }: Props = $props();
</script>
<header class="header">
<div class="header-left">
<svg
width="20"
height="20"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="2"
>
<path d="M12 2L2 7l10 5 10-5-10-5z" />
<path d="M2 17l10 5 10-5" />
<path d="M2 12l10 5 10-5" />
</svg>
<span class="header-title">VaultLink</span>
<span class="header-sep">/</span>
<span class="header-vault">{vaultId}</span>
</div>
<div class="header-right">
<span class="server-version">v{serverVersion}</span>
<button class="header-btn" onclick={onRefresh} title="Refresh">
<svg
width="16"
height="16"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="2"
>
<path
d="M21 2v6h-6M3 12a9 9 0 0 1 15-6.7L21 8M3 22v-6h6M21 12a9 9 0 0 1-15 6.7L3 16"
/>
</svg>
</button>
{#if auth.availableVaults.length > 1}
<button
class="header-btn"
onclick={() => auth.deselectVault()}
title="Switch vault"
>
<svg
width="16"
height="16"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="2"
>
<path d="M22 19a2 2 0 0 1-2 2H4a2 2 0 0 1-2-2V5a2 2 0 0 1 2-2h5l2 3h9a2 2 0 0 1 2 2z"/>
</svg>
</button>
{/if}
<button
class="header-btn"
onclick={() => auth.logout()}
title="Sign out"
>
<svg
width="16"
height="16"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="2"
>
<path d="M9 21H5a2 2 0 0 1-2-2V5a2 2 0 0 1 2-2h4" />
<polyline points="16 17 21 12 16 7" />
<line x1="21" y1="12" x2="9" y2="12" />
</svg>
</button>
</div>
</header>
<style>
.header {
display: flex;
align-items: center;
justify-content: space-between;
height: 48px;
padding: 0 16px;
background: var(--bg-secondary);
border-bottom: 1px solid var(--border);
flex-shrink: 0;
}
.header-left {
display: flex;
align-items: center;
gap: 8px;
color: var(--text);
}
.header-title {
font-weight: 600;
font-size: 15px;
}
.header-sep {
color: var(--text-subtle);
}
.header-vault {
font-family: var(--mono);
font-size: 14px;
color: var(--accent);
}
.header-right {
display: flex;
align-items: center;
gap: 8px;
}
.server-version {
font-size: 12px;
color: var(--text-subtle);
font-family: var(--mono);
}
.header-btn {
padding: 6px;
border-radius: var(--radius-sm);
color: var(--text-muted);
transition: color 0.15s, background 0.15s;
}
.header-btn:hover {
color: var(--text);
background: var(--bg-hover);
}
</style>

View file

@ -0,0 +1,176 @@
<script lang="ts">
import { auth } from "../lib/stores.svelte";
import { listVaults } from "../lib/api";
let token = $state("");
let error = $state("");
let loading = $state(false);
async function handleSubmit(e: Event) {
e.preventDefault();
if (!token.trim()) {
error = "Token is required.";
return;
}
error = "";
loading = true;
try {
const response = await listVaults(token.trim());
auth.authenticate(
token.trim(),
response.userName,
response.vaults
);
} catch {
error = "Authentication failed. Check your token.";
} finally {
loading = false;
}
}
</script>
<div class="login-page">
<div class="login-card">
<div class="logo">
<svg width="32" height="32" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<path d="M12 2L2 7l10 5 10-5-10-5z"/>
<path d="M2 17l10 5 10-5"/>
<path d="M2 12l10 5 10-5"/>
</svg>
<h1>VaultLink</h1>
</div>
<p class="subtitle">Vault History Browser</p>
<form onsubmit={handleSubmit}>
<label>
<span>Token</span>
<input
type="password"
bind:value={token}
placeholder="Enter your access token"
disabled={loading}
/>
</label>
{#if error}
<div class="error">{error}</div>
{/if}
<button type="submit" class="btn-primary" disabled={loading}>
{#if loading}
<span class="btn-spinner"></span>
Connecting...
{:else}
Connect
{/if}
</button>
</form>
</div>
</div>
<style>
.login-page {
display: flex;
align-items: center;
justify-content: center;
height: 100%;
background: var(--bg);
}
.login-card {
width: 100%;
max-width: 400px;
padding: 40px;
background: var(--bg-secondary);
border: 1px solid var(--border);
border-radius: 12px;
box-shadow: var(--shadow);
}
.logo {
display: flex;
align-items: center;
gap: 12px;
margin-bottom: 4px;
color: var(--text);
}
.logo h1 {
font-size: 24px;
font-weight: 600;
}
.subtitle {
color: var(--text-muted);
margin-bottom: 32px;
font-size: 14px;
}
form {
display: flex;
flex-direction: column;
gap: 20px;
}
label {
display: flex;
flex-direction: column;
gap: 6px;
}
label span {
font-size: 13px;
font-weight: 500;
color: var(--text-muted);
}
input {
width: 100%;
}
.error {
color: var(--red);
font-size: 13px;
padding: 8px 12px;
background: var(--red-bg);
border-radius: var(--radius-sm);
}
.btn-primary {
width: 100%;
padding: 10px 16px;
background: var(--accent);
color: #fff;
font-weight: 600;
border-radius: var(--radius);
transition: background 0.15s;
display: flex;
align-items: center;
justify-content: center;
gap: 8px;
}
.btn-primary:hover:not(:disabled) {
background: var(--accent-hover);
}
.btn-primary:disabled {
opacity: 0.6;
cursor: not-allowed;
}
.btn-spinner {
width: 16px;
height: 16px;
border: 2px solid rgba(255, 255, 255, 0.3);
border-top-color: #fff;
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
@keyframes spin {
to {
transform: rotate(360deg);
}
}
</style>

View file

@ -0,0 +1,191 @@
<script lang="ts">
import type { DocumentVersionWithoutContent } from "../lib/types";
import { relativeTime, absoluteTime } from "../lib/stores.svelte";
interface Props {
min: number;
max: number;
value: number | null;
versions: DocumentVersionWithoutContent[];
onchange: (value: number | null) => void;
}
let { min, max, value, versions, onchange }: Props = $props();
let isNow = $derived(value === null || value >= max);
function handleInput(e: Event) {
const target = e.target as HTMLInputElement;
const v = parseInt(target.value, 10);
if (v >= max) {
onchange(null);
} else {
onchange(v);
}
}
function snapToNow() {
onchange(null);
}
let currentVersion = $derived(
value !== null
? versions.find((v) => v.vaultUpdateId === value) ??
versions.reduce(
(closest, v) =>
Math.abs(v.vaultUpdateId - (value ?? max)) <
Math.abs(
closest.vaultUpdateId - (value ?? max)
)
? v
: closest,
versions[0]
)
: null
);
</script>
<div class="time-slider">
<div class="slider-label">
<svg
width="14"
height="14"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="2"
>
<circle cx="12" cy="12" r="10" />
<polyline points="12 6 12 12 16 14" />
</svg>
<span class="label-text">Time Travel</span>
</div>
<div class="slider-track">
<input
type="range"
min={min}
max={max}
value={value ?? max}
oninput={handleInput}
/>
</div>
<div class="slider-info">
{#if isNow}
<span class="now-badge">Now</span>
{:else if currentVersion}
<span
class="time-info"
title={absoluteTime(currentVersion.updatedDate)}
>
#{value}
&middot;
{relativeTime(currentVersion.updatedDate)}
</span>
{:else}
<span class="time-info">#{value}</span>
{/if}
</div>
{#if !isNow}
<button class="snap-btn" onclick={snapToNow} title="Back to now">
<svg
width="14"
height="14"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="2"
>
<path d="M5 12h14M12 5l7 7-7 7" />
</svg>
</button>
{/if}
</div>
<style>
.time-slider {
display: flex;
align-items: center;
gap: 12px;
}
.slider-label {
display: flex;
align-items: center;
gap: 6px;
color: var(--text-muted);
flex-shrink: 0;
}
.label-text {
font-size: 12px;
font-weight: 500;
text-transform: uppercase;
letter-spacing: 0.5px;
}
.slider-track {
flex: 1;
min-width: 120px;
}
.slider-track input[type="range"] {
width: 100%;
height: 4px;
appearance: none;
background: var(--bg-tertiary);
border-radius: 2px;
outline: none;
border: none;
padding: 0;
}
.slider-track input[type="range"]::-webkit-slider-thumb {
appearance: none;
width: 14px;
height: 14px;
background: var(--accent);
border-radius: 50%;
cursor: pointer;
transition: transform 0.1s;
}
.slider-track input[type="range"]::-webkit-slider-thumb:hover {
transform: scale(1.2);
}
.slider-info {
flex-shrink: 0;
min-width: 100px;
}
.now-badge {
font-size: 11px;
font-weight: 600;
color: var(--green);
background: var(--green-bg);
padding: 2px 10px;
border-radius: 10px;
text-transform: uppercase;
}
.time-info {
font-size: 12px;
color: var(--text-muted);
font-family: var(--mono);
}
.snap-btn {
padding: 4px;
color: var(--accent);
border-radius: var(--radius-sm);
transition: background 0.15s;
flex-shrink: 0;
}
.snap-btn:hover {
background: var(--bg-hover);
}
</style>

View file

@ -0,0 +1,80 @@
<script lang="ts">
import { toasts } from "../lib/stores.svelte";
const typeColors: Record<string, string> = {
success: "var(--green)",
error: "var(--red)",
info: "var(--accent)"
};
</script>
{#if toasts.items.length > 0}
<div class="toast-container">
{#each toasts.items as toast (toast.id)}
<div
class="toast"
style="border-left-color: {typeColors[toast.type]}"
>
<span class="toast-message">{toast.message}</span>
<button
class="toast-dismiss"
onclick={() => toasts.dismiss(toast.id)}
>
&times;
</button>
</div>
{/each}
</div>
{/if}
<style>
.toast-container {
position: fixed;
bottom: 16px;
right: 16px;
z-index: 9999;
display: flex;
flex-direction: column;
gap: 8px;
max-width: 400px;
}
.toast {
display: flex;
align-items: center;
gap: 12px;
padding: 12px 16px;
background: var(--bg-secondary);
border: 1px solid var(--border);
border-left-width: 3px;
border-radius: var(--radius);
box-shadow: var(--shadow);
animation: slide-in 0.2s ease-out;
}
.toast-message {
flex: 1;
font-size: 13px;
}
.toast-dismiss {
font-size: 18px;
color: var(--text-muted);
padding: 0 4px;
}
.toast-dismiss:hover {
color: var(--text);
}
@keyframes slide-in {
from {
transform: translateX(100%);
opacity: 0;
}
to {
transform: translateX(0);
opacity: 1;
}
}
</style>

View file

@ -0,0 +1,198 @@
<script lang="ts">
import { auth } from "../lib/stores.svelte";
import { relativeTime } from "../lib/stores.svelte";
import type { VaultInfo } from "../lib/types";
function select(vault: VaultInfo) {
auth.selectVault(vault.name);
}
function formatStats(vault: VaultInfo): string {
const docs = vault.documentCount === 1
? "1 document"
: `${vault.documentCount} documents`;
if (!vault.createdAt) return docs;
return `${docs} · created ${relativeTime(vault.createdAt)}`;
}
</script>
<div class="picker-page">
<div class="picker-card">
<div class="picker-header">
<div class="logo">
<svg width="28" height="28" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<path d="M12 2L2 7l10 5 10-5-10-5z"/>
<path d="M2 17l10 5 10-5"/>
<path d="M2 12l10 5 10-5"/>
</svg>
<div>
<h1>Select a vault</h1>
<p class="user-info">
Signed in as <strong>{auth.userName}</strong>
<button class="logout-link" onclick={() => auth.logout()}>Sign out</button>
</p>
</div>
</div>
</div>
{#if auth.availableVaults.length === 0}
<div class="empty">
<p>No vaults found</p>
<p class="empty-hint">
Vaults are created when a sync client first connects.
</p>
</div>
{:else}
<ul class="vault-list">
{#each auth.availableVaults as vault}
<li>
<button class="vault-item" onclick={() => select(vault)}>
<svg class="vault-icon" width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<path d="M22 19a2 2 0 0 1-2 2H4a2 2 0 0 1-2-2V5a2 2 0 0 1 2-2h5l2 3h9a2 2 0 0 1 2 2z"/>
</svg>
<div class="vault-details">
<span class="vault-name">{vault.name}</span>
<span class="vault-stats">{formatStats(vault)}</span>
</div>
<svg class="vault-arrow" width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<polyline points="9 18 15 12 9 6"/>
</svg>
</button>
</li>
{/each}
</ul>
{/if}
</div>
</div>
<style>
.picker-page {
display: flex;
align-items: center;
justify-content: center;
height: 100%;
background: var(--bg);
}
.picker-card {
width: 100%;
max-width: 480px;
background: var(--bg-secondary);
border: 1px solid var(--border);
border-radius: 12px;
box-shadow: var(--shadow);
overflow: hidden;
}
.picker-header {
padding: 32px 32px 24px;
}
.logo {
display: flex;
align-items: flex-start;
gap: 12px;
color: var(--text);
}
.logo svg {
margin-top: 2px;
flex-shrink: 0;
}
.logo h1 {
font-size: 20px;
font-weight: 600;
}
.user-info {
font-size: 13px;
color: var(--text-muted);
margin-top: 4px;
}
.user-info strong {
color: var(--text);
font-weight: 500;
}
.logout-link {
color: var(--text-subtle);
font-size: 13px;
text-decoration: underline;
margin-left: 8px;
padding: 0;
}
.logout-link:hover {
color: var(--text-muted);
}
.vault-list {
list-style: none;
border-top: 1px solid var(--border);
}
.vault-list li + li {
border-top: 1px solid var(--border-light);
}
.vault-item {
display: flex;
align-items: center;
gap: 12px;
width: 100%;
padding: 14px 24px;
text-align: left;
color: var(--text);
transition: background 0.12s;
}
.vault-item:hover {
background: var(--bg-hover);
}
.vault-icon {
color: var(--text-muted);
flex-shrink: 0;
}
.vault-details {
flex: 1;
display: flex;
flex-direction: column;
gap: 2px;
min-width: 0;
}
.vault-name {
font-family: var(--mono);
font-size: 14px;
}
.vault-stats {
font-size: 12px;
color: var(--text-subtle);
}
.vault-arrow {
color: var(--text-subtle);
flex-shrink: 0;
}
.empty {
padding: 32px;
text-align: center;
border-top: 1px solid var(--border);
}
.empty p {
color: var(--text-muted);
}
.empty-hint {
font-size: 13px;
color: var(--text-subtle);
margin-top: 8px;
}
</style>

View file

@ -0,0 +1,121 @@
import type {
DocumentVersion,
DocumentVersionWithoutContent,
FetchLatestDocumentsResponse,
ListVaultsResponse,
PingResponse,
VaultHistoryResponse
} from "./types";
async function fetchJsonWithToken<T>(
path: string,
token: string,
init?: RequestInit
): Promise<T> {
const response = await fetch(path, {
...init,
headers: {
Authorization: `Bearer ${token}`,
"device-id": "history-ui",
...init?.headers
}
});
if (!response.ok) {
const body = await response.text();
throw new Error(`HTTP ${response.status}: ${body}`);
}
return response.json() as Promise<T>;
}
export async function listVaults(
token: string
): Promise<ListVaultsResponse> {
return fetchJsonWithToken("/vaults", token);
}
export class ApiClient {
constructor(
private vaultId: string,
private token: string
) {}
private get baseUrl(): string {
return `/vaults/${encodeURIComponent(this.vaultId)}`;
}
private async fetchJson<T>(
path: string,
init?: RequestInit
): Promise<T> {
return fetchJsonWithToken(path, this.token, init);
}
async ping(): Promise<PingResponse> {
return this.fetchJson(`${this.baseUrl}/ping`);
}
async fetchLatestDocuments(): Promise<FetchLatestDocumentsResponse> {
return this.fetchJson(`${this.baseUrl}/documents`);
}
async fetchDocumentVersions(
documentId: string
): Promise<DocumentVersionWithoutContent[]> {
return this.fetchJson(
`${this.baseUrl}/documents/${documentId}/versions`
);
}
async fetchDocumentVersion(
documentId: string,
vaultUpdateId: number
): Promise<DocumentVersion> {
return this.fetchJson(
`${this.baseUrl}/documents/${documentId}/versions/${vaultUpdateId}`
);
}
async fetchDocumentVersionContent(
documentId: string,
vaultUpdateId: number
): Promise<ArrayBuffer> {
const response = await fetch(
`${this.baseUrl}/documents/${documentId}/versions/${vaultUpdateId}/content`,
{ headers: this.headers() }
);
if (!response.ok) {
throw new Error(`HTTP ${response.status}`);
}
return response.arrayBuffer();
}
async fetchVaultHistory(
limit?: number,
beforeUpdateId?: number
): Promise<VaultHistoryResponse> {
const params = new URLSearchParams();
if (limit !== undefined) params.set("limit", String(limit));
if (beforeUpdateId !== undefined)
params.set("before_update_id", String(beforeUpdateId));
const qs = params.toString();
return this.fetchJson(
`${this.baseUrl}/history${qs ? `?${qs}` : ""}`
);
}
async restoreVersion(
documentId: string,
vaultUpdateId: number
): Promise<DocumentVersionWithoutContent> {
return this.fetchJson(
`${this.baseUrl}/documents/${documentId}/restore`,
{
method: "POST",
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify({ vaultUpdateId })
}
);
}
}

View file

@ -0,0 +1,305 @@
import { ApiClient } from "./api";
import type {
DocumentVersionWithoutContent,
VaultInfo,
VersionEvent,
ActionType,
TreeNode
} from "./types";
class AuthStore {
token = $state("");
userName = $state("");
vaultId = $state("");
serverVersion = $state("");
availableVaults = $state<VaultInfo[]>([]);
isAuthenticated = $state(false);
api = $state<ApiClient | null>(null);
authenticate(
token: string,
userName: string,
vaults: VaultInfo[]
) {
this.token = token;
this.userName = userName;
this.availableVaults = vaults;
sessionStorage.setItem("vaultlink_token", token);
}
selectVault(vaultId: string) {
this.vaultId = vaultId;
this.isAuthenticated = true;
this.api = new ApiClient(vaultId, this.token);
sessionStorage.setItem("vaultlink_vault", vaultId);
}
deselectVault() {
this.vaultId = "";
this.isAuthenticated = false;
this.api = null;
sessionStorage.removeItem("vaultlink_vault");
}
logout() {
this.token = "";
this.userName = "";
this.vaultId = "";
this.serverVersion = "";
this.availableVaults = [];
this.isAuthenticated = false;
this.api = null;
sessionStorage.removeItem("vaultlink_token");
sessionStorage.removeItem("vaultlink_vault");
}
tryRestore(): { token: string; vaultId?: string } | null {
const token = sessionStorage.getItem("vaultlink_token");
if (!token) return null;
const vaultId =
sessionStorage.getItem("vaultlink_vault") ?? undefined;
return { token, vaultId };
}
}
export const auth = new AuthStore();
// Navigation
export type View =
| { kind: "dashboard" }
| { kind: "document"; documentId: string };
class NavStore {
current = $state<View>({ kind: "dashboard" });
goto(view: View) {
this.current = view;
}
goHome() {
this.current = { kind: "dashboard" };
}
}
export const nav = new NavStore();
// Toasts
export interface Toast {
id: number;
message: string;
type: "success" | "error" | "info";
}
class ToastStore {
items = $state<Toast[]>([]);
private nextId = 0;
add(message: string, type: Toast["type"] = "info") {
const id = this.nextId++;
this.items.push({ id, message, type });
setTimeout(() => this.dismiss(id), 5000);
}
dismiss(id: number) {
this.items = this.items.filter((t) => t.id !== id);
}
}
export const toasts = new ToastStore();
// Utilities
export function inferAction(
version: DocumentVersionWithoutContent,
previousVersion?: DocumentVersionWithoutContent
): ActionType {
if (version.isDeleted) return "deleted";
if (!previousVersion) return "created";
if (
previousVersion.isDeleted &&
!version.isDeleted
)
return "restored";
if (previousVersion.relativePath !== version.relativePath)
return "renamed";
return "updated";
}
export function enrichVersions(
versions: DocumentVersionWithoutContent[]
): VersionEvent[] {
// versions should be sorted by vaultUpdateId ascending
const sorted = [...versions].sort(
(a, b) => a.vaultUpdateId - b.vaultUpdateId
);
const byDoc = new Map<string, DocumentVersionWithoutContent[]>();
for (const v of sorted) {
let arr = byDoc.get(v.documentId);
if (!arr) {
arr = [];
byDoc.set(v.documentId, arr);
}
arr.push(v);
}
return sorted.map((v) => {
const docVersions = byDoc.get(v.documentId)!;
const idx = docVersions.indexOf(v);
const prev = idx > 0 ? docVersions[idx - 1] : undefined;
const action = inferAction(v, prev);
return {
...v,
action,
previousPath:
action === "renamed" ? prev?.relativePath : undefined
};
});
}
export function buildTree(
documents: DocumentVersionWithoutContent[],
showDeleted: boolean
): TreeNode {
const root: TreeNode = {
name: "",
path: "",
isFolder: true,
children: []
};
const filtered = showDeleted
? documents
: documents.filter((d) => !d.isDeleted);
for (const doc of filtered) {
const parts = doc.relativePath.split("/");
let current = root;
for (let i = 0; i < parts.length; i++) {
const part = parts[i];
const isFile = i === parts.length - 1;
const path = parts.slice(0, i + 1).join("/");
if (isFile) {
current.children.push({
name: part,
path,
isFolder: false,
children: [],
document: doc,
isDeleted: doc.isDeleted
});
} else {
let folder = current.children.find(
(c) => c.isFolder && c.name === part
);
if (!folder) {
folder = {
name: part,
path,
isFolder: true,
children: []
};
current.children.push(folder);
}
current = folder;
}
}
}
sortTree(root);
return root;
}
function sortTree(node: TreeNode) {
node.children.sort((a, b) => {
if (a.isFolder !== b.isFolder) return a.isFolder ? -1 : 1;
return a.name.localeCompare(b.name);
});
for (const child of node.children) {
if (child.isFolder) sortTree(child);
}
}
export function relativeTime(dateStr: string): string {
const date = new Date(dateStr);
const now = Date.now();
const diff = now - date.getTime();
const seconds = Math.floor(diff / 1000);
const minutes = Math.floor(seconds / 60);
const hours = Math.floor(minutes / 60);
const days = Math.floor(hours / 24);
if (seconds < 60) return "just now";
if (minutes < 60) return `${minutes}m ago`;
if (hours < 24) return `${hours}h ago`;
if (days < 7) return `${days}d ago`;
return date.toLocaleDateString("en-US", {
month: "short",
day: "numeric",
year: days > 365 ? "numeric" : undefined
});
}
export function absoluteTime(dateStr: string): string {
return new Date(dateStr).toLocaleString();
}
export function formatBytes(bytes: number): string {
if (bytes === 0) return "0 B";
const k = 1024;
const sizes = ["B", "KB", "MB", "GB"];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return `${parseFloat((bytes / Math.pow(k, i)).toFixed(1))} ${sizes[i]}`;
}
export function fileExtension(path: string): string {
const dot = path.lastIndexOf(".");
return dot > -1 ? path.substring(dot + 1).toLowerCase() : "";
}
export function isTextFile(path: string): boolean {
const textExts = new Set([
"md",
"txt",
"json",
"yaml",
"yml",
"toml",
"xml",
"html",
"css",
"js",
"ts",
"svelte",
"rs",
"py",
"sh",
"bash",
"zsh",
"csv",
"svg",
"log",
"conf",
"cfg",
"ini",
"env",
"gitignore",
"editorconfig"
]);
return textExts.has(fileExtension(path));
}
export function isImageFile(path: string): boolean {
const imageExts = new Set([
"png",
"jpg",
"jpeg",
"gif",
"webp",
"svg",
"ico",
"bmp"
]);
return imageExts.has(fileExtension(path));
}

View file

@ -0,0 +1,7 @@
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.
import type { VaultInfo } from "./VaultInfo";
/**
* Response to listing vaults accessible to the authenticated user.
*/
export type ListVaultsResponse = { vaults: Array<VaultInfo>, hasMore: boolean, userName: string, };

View file

@ -0,0 +1,6 @@
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.
/**
* Summary of a single vault returned by the list-vaults endpoint.
*/
export type VaultInfo = { name: string, documentCount: number, createdAt: string | null, };

View file

@ -0,0 +1,7 @@
import { mount } from "svelte";
import App from "./App.svelte";
import "./app.css";
const app = mount(App, { target: document.getElementById("app")! });
export default app;

View file

@ -0,0 +1,5 @@
import { vitePreprocess } from "@sveltejs/vite-plugin-svelte";
export default {
preprocess: vitePreprocess()
};

View file

@ -0,0 +1,16 @@
{
"compilerOptions": {
"target": "ESNext",
"module": "ESNext",
"moduleResolution": "bundler",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"isolatedModules": true,
"verbatimModuleSyntax": true,
"types": ["svelte"]
},
"include": ["src/**/*", "src/**/*.svelte"]
}

View file

@ -0,0 +1,15 @@
import { defineConfig } from "vite";
import { svelte } from "@sveltejs/vite-plugin-svelte";
export default defineConfig({
plugins: [svelte()],
build: {
outDir: "dist",
emptyOutDir: true
},
server: {
proxy: {
"/vaults": "http://localhost:3010"
}
}
});

View file

@ -1423,13 +1423,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/@types/murmurhash3js-revisited": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/@types/murmurhash3js-revisited/-/murmurhash3js-revisited-3.0.3.tgz",
"integrity": "sha512-QvlqvYtGBYIDeO8dFdY4djkRubcrc+yTJtBc7n8VZPlJDUS/00A+PssbvERM8f9bYRmcaSEHPZgZojeQj7kzAA==",
"dev": true,
"license": "MIT"
},
"node_modules/@types/node": {
"version": "25.0.2",
"resolved": "https://registry.npmjs.org/@types/node/-/node-25.0.2.tgz",
@ -3965,15 +3958,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/murmurhash3js-revisited": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/murmurhash3js-revisited/-/murmurhash3js-revisited-3.0.0.tgz",
"integrity": "sha512-/sF3ee6zvScXMb1XFJ8gDsSnY+X8PbOyjIuBhtgis10W2Jx4ZjIhikUCIF9c4gpJxVnQIsPAFrSwTCuAjicP6g==",
"license": "MIT",
"engines": {
"node": ">=8.0.0"
}
},
"node_modules/nanoid": {
"version": "3.3.11",
"dev": true,
@ -5911,17 +5895,13 @@
},
"sync-client": {
"version": "0.14.0",
"dependencies": {
"murmurhash3js-revisited": "^3.0.0"
},
"devDependencies": {
"@sentry/browser": "^10.30.0",
"@types/murmurhash3js-revisited": "^3.0.3",
"@types/node": "^25.0.2",
"byte-base64": "^1.1.0",
"minimatch": "^10.1.1",
"p-queue": "^9.0.1",
"reconcile-text": "^0.11.0",
"reconcile-text": "^0.8.0",
"ts-loader": "^9.5.4",
"tslib": "2.8.1",
"tsx": "^4.21.0",
@ -5946,6 +5926,13 @@
"url": "https://github.com/sponsors/isaacs"
}
},
"sync-client/node_modules/reconcile-text": {
"version": "0.8.0",
"resolved": "https://registry.npmjs.org/reconcile-text/-/reconcile-text-0.8.0.tgz",
"integrity": "sha512-evskVha3YgpP2ZelsFxP9t7CuKnwE7TrsH3FdrH2mfKbzjUWiNF7scHXsFbFS921lmFlAOB94DHNAWPvL34Mqg==",
"dev": true,
"license": "MIT"
},
"test-client": {
"version": "0.14.0",
"bin": {

View file

@ -1,9 +1,6 @@
import { describe, it } from "node:test";
import type {
Database,
DocumentRecord,
RelativePath
} from "../persistence/database";
import type { DocumentId, DocumentRecord, RelativePath } from "../sync-operations/types";
import type { SyncEventQueue } from "../sync-operations/sync-event-queue";
import { FileOperations } from "./file-operations";
import { Logger } from "../tracing/logger";
import { assertSetContainsExactly } from "../utils/assert-set-contains-exactly";
@ -21,19 +18,18 @@ class MockServerConfig implements Pick<ServerConfig, "getConfig"> {
}
}
class MockDatabase implements Partial<Database> {
public getLatestDocumentByRelativePath(
_target: RelativePath
class MockQueue implements Pick<SyncEventQueue, "getDocument" | "moveDocument"> {
public getDocument(
_path: RelativePath
): DocumentRecord | undefined {
// no-op
return undefined;
}
public move(
_oldRelativePath: RelativePath,
_newRelativePath: RelativePath
): void {
// no-op
public moveDocument(
_oldPath: RelativePath,
_newPath: RelativePath
): DocumentId | undefined {
return undefined;
}
}
@ -89,7 +85,7 @@ describe("File operations", () => {
const fileSystemOperations = new FakeFileSystemOperations();
const fileOperations = new FileOperations(
new Logger(),
new MockDatabase() as Database, // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
new MockQueue() as SyncEventQueue, // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
fileSystemOperations,
new MockServerConfig() as ServerConfig // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
);
@ -119,7 +115,7 @@ describe("File operations", () => {
const fileSystemOperations = new FakeFileSystemOperations();
const fileOperations = new FileOperations(
new Logger(),
new MockDatabase() as Database, // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
new MockQueue() as SyncEventQueue, // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
fileSystemOperations,
new MockServerConfig() as ServerConfig // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
);
@ -159,7 +155,7 @@ describe("File operations", () => {
const fileSystemOperations = new FakeFileSystemOperations();
const fileOperations = new FileOperations(
new Logger(),
new MockDatabase() as Database, // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
new MockQueue() as SyncEventQueue, // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
fileSystemOperations,
new MockServerConfig() as ServerConfig // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
);
@ -178,7 +174,7 @@ describe("File operations", () => {
const fileSystemOperations = new FakeFileSystemOperations();
const fileOperations = new FileOperations(
new Logger(),
new MockDatabase() as Database, // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
new MockQueue() as SyncEventQueue, // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
fileSystemOperations,
new MockServerConfig() as ServerConfig // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
);
@ -207,7 +203,7 @@ describe("File operations", () => {
const fileSystemOperations = new FakeFileSystemOperations();
const fileOperations = new FileOperations(
new Logger(),
new MockDatabase() as Database, // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
new MockQueue() as SyncEventQueue, // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
fileSystemOperations,
new MockServerConfig() as ServerConfig // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
);

View file

@ -1,6 +1,7 @@
import type { Logger } from "../tracing/logger";
import type { FileSystemOperations } from "./filesystem-operations";
import type { Database, RelativePath } from "../persistence/database";
import type { RelativePath } from "../sync-operations/types";
import type { SyncEventQueue } from "../sync-operations/sync-event-queue";
import { SafeFileSystemOperations } from "./safe-filesystem-operations";
import type { TextWithCursors } from "reconcile-text";
import { reconcile } from "reconcile-text";
@ -14,7 +15,7 @@ export class FileOperations {
public constructor(
private readonly logger: Logger,
private readonly database: Database,
private readonly queue: SyncEventQueue,
fs: FileSystemOperations,
private readonly serverConfig: ServerConfig,
private readonly nativeLineEndings = "\n"
@ -58,7 +59,10 @@ export class FileOperations {
return this.fs.write(path, this.toNativeLineEndings(newContent));
}
public async ensureClearPath(path: RelativePath): Promise<void> {
// Returns the deconflicted path if a file was moved, undefined otherwise
public async ensureClearPath(
path: RelativePath
): Promise<RelativePath | undefined> {
if (await this.fs.exists(path)) {
const deconflictedPath = await this.deconflictPath(path);
try {
@ -66,14 +70,16 @@ export class FileOperations {
`Didn't expect ${path} to exist, deconflicting by moving it to '${deconflictedPath}'`
);
this.database.move(path, deconflictedPath);
this.queue.moveDocument(path, deconflictedPath);
await this.fs.rename(path, deconflictedPath, true);
return deconflictedPath;
} finally {
this.fs.unlock(deconflictedPath);
}
} else {
await this.createParentDirectories(path);
}
return undefined;
}
/**
@ -160,21 +166,24 @@ export class FileOperations {
return this.fs.exists(path);
}
// Returns the deconflicted path if a file at the target was displaced
public async move(
oldPath: RelativePath,
newPath: RelativePath
): Promise<void> {
): Promise<RelativePath | undefined> {
if (oldPath === newPath) {
return;
return undefined;
}
await this.ensureClearPath(newPath);
this.database.move(oldPath, newPath);
const deconflictedPath = await this.ensureClearPath(newPath);
this.queue.moveDocument(oldPath, newPath);
await this.fs.rename(oldPath, newPath);
await this.deletingEmptyParentDirectoriesOfDeletedFile(oldPath);
return deconflictedPath;
}
public reset(): void {
this.fs.reset();
}
@ -274,17 +283,15 @@ export class FileOperations {
newName = `${directory}${stem} (${currentCount})${extension}`;
// Avoid multiple deconflictPath calls returning the same path
if (this.fs.tryLock(newName)) {
const newDocument =
this.database.getLatestDocumentByRelativePath(newName);
if (
newDocument?.isDeleted === false || // the document might have been confirmed by the server at a new path but haven't yet moved there locally
(await this.fs.exists(newName, true))
) {
this.fs.unlock(newName);
} else {
return newName;
}
await this.fs.waitForLock(newName);
const existingRecord = this.queue.getDocument(newName);
if (
existingRecord !== undefined || // the document might have been confirmed by the server at a new path but haven't yet moved there locally
(await this.fs.exists(newName, true))
) {
this.fs.unlock(newName);
} else {
return newName;
}
}
}

View file

@ -1,301 +1,2 @@
import type { Logger } from "../tracing/logger";
import { EMPTY_HASH } from "../utils/hash";
import { CoveredValues } from "../utils/data-structures/min-covered";
import { awaitAll } from "../utils/await-all";
import { removeFromArray } from "../utils/remove-from-array";
export type VaultUpdateId = number;
export type DocumentId = string;
export type RelativePath = string;
export interface DocumentMetadata {
documentId: DocumentId;
parentVersionId: VaultUpdateId;
hash: string;
remoteRelativePath?: RelativePath;
}
export interface StoredDocumentMetadata {
relativePath: RelativePath;
documentId: DocumentId;
parentVersionId: VaultUpdateId;
remoteRelativePath?: RelativePath;
hash: string;
}
export interface StoredDatabase {
documents: StoredDocumentMetadata[];
lastSeenUpdateId: VaultUpdateId | undefined;
}
/**
* Represents a document in the database.
*
* It is mutable and its content should always represent the latest
* state of the document on disk based on the update events we have seen.
*/
export interface DocumentRecord {
relativePath: RelativePath;
metadata: DocumentMetadata | undefined;
isDeleted: boolean;
parallelVersion: number;
}
export class Database {
private documents: DocumentRecord[];
private lastSeenUpdateIds: CoveredValues;
public constructor(
private readonly logger: Logger,
initialState: Partial<StoredDatabase> | undefined,
private readonly saveData: (data: StoredDatabase) => Promise<void>
) {
initialState ??= {};
this.documents =
initialState.documents?.map(({ relativePath, ...metadata }) => ({
relativePath,
metadata,
isDeleted: false,
parallelVersion: 0
})) ?? [];
this.ensureConsistency();
this.logger.debug(`Loaded ${this.documents.length} documents`);
const { lastSeenUpdateId } = initialState;
this.logger.debug(`Loaded last seen update id: ${lastSeenUpdateId}`);
this.lastSeenUpdateIds = new CoveredValues(
Math.max(0, lastSeenUpdateId ?? 0) // the first updateId will be 1 which is the first integer after -1
);
this.documents.forEach((doc) => {
this.lastSeenUpdateIds.add(doc.metadata?.parentVersionId);
});
}
public get length(): number {
return this.documents.length;
}
public get resolvedDocuments(): DocumentRecord[] {
const paths = new Map<string, DocumentRecord[]>();
this.documents
// eslint-disable-next-line no-restricted-syntax -- Type narrowing, not removing a specific item
.filter(({ metadata }) => metadata !== undefined)
.forEach((record) =>
paths.set(record.relativePath, [
record,
...(paths.get(record.relativePath) ?? [])
])
);
return Array.from(paths.values()).map((records) => {
records.sort(
(a, b) => b.parallelVersion - a.parallelVersion // descending
);
if (
records.length > 1 &&
records.some((current, i) =>
i === 0
? false
: records[i - 1].parallelVersion ===
current.parallelVersion
)
) {
throw new Error(
`Multiple documents with the same parallel version and path at ${records[0].relativePath}`
);
}
return records[0];
});
}
public updateDocumentMetadata(
metadata: {
documentId: DocumentId;
parentVersionId: VaultUpdateId;
hash: string;
remoteRelativePath: RelativePath;
},
target: DocumentRecord
): void {
if (!this.documents.includes(target)) {
throw new Error("Document not found in database");
}
this.logger.debug(
`Updating document metadata for ${target.relativePath} from ${JSON.stringify(
target.metadata,
null,
2
)} to ${JSON.stringify(metadata, null, 2)}`
);
target.metadata = metadata;
this.saveInTheBackground();
}
public getLatestDocumentByRelativePath(
target: RelativePath
): DocumentRecord | undefined {
const candidates = this.documents.filter(
({ relativePath }) => relativePath === target
);
candidates.sort((a, b) => b.parallelVersion - a.parallelVersion); // descending
return candidates[0];
}
public createNewPendingDocument(
relativePath: RelativePath
): DocumentRecord {
this.logger.debug(`Creating new pending document: ${relativePath}`);
const previousEntry =
this.getLatestDocumentByRelativePath(relativePath);
const entry = {
relativePath,
metadata: undefined,
isDeleted: false,
parallelVersion:
previousEntry?.parallelVersion === undefined
? 0
: previousEntry.parallelVersion + 1
};
this.documents.push(entry);
// no need to save as we only save documents which have metadata
return entry;
}
public getDocumentByDocumentId(
target: DocumentId
): DocumentRecord | undefined {
return this.documents.find(
({ metadata }) => metadata?.documentId === target
);
}
public move(
oldRelativePath: RelativePath,
newRelativePath: RelativePath
): void {
const oldDocument =
this.getLatestDocumentByRelativePath(oldRelativePath);
if (oldDocument === undefined) {
return;
}
const newDocument =
this.getLatestDocumentByRelativePath(newRelativePath);
if (newDocument?.isDeleted === false) {
throw new Error(
`Document already exists at new location: ${newRelativePath}`
);
}
oldDocument.relativePath = newRelativePath;
// We might be in a strange state where the target of the move has just got deleted,
// however, its metadata might already have a bunch of updates queued up for
// the document at the new location. We need to keep these updates.
oldDocument.parallelVersion =
newDocument !== undefined ? newDocument.parallelVersion + 1 : 0;
this.saveInTheBackground();
}
public delete(relativePath: RelativePath): void {
const candidate = this.getLatestDocumentByRelativePath(relativePath);
if (candidate === undefined) {
return;
}
candidate.isDeleted = true;
}
public removeDocument(target: DocumentRecord): void {
removeFromArray(this.documents, target);
this.saveInTheBackground();
}
public getLastSeenUpdateId(): VaultUpdateId {
return this.lastSeenUpdateIds.min;
}
public addSeenUpdateId(value: number): void {
const previousMin = this.lastSeenUpdateIds.min;
this.lastSeenUpdateIds.add(value);
if (previousMin !== this.lastSeenUpdateIds.min) {
this.saveInTheBackground();
}
}
public setLastSeenUpdateId(value: number): void {
this.lastSeenUpdateIds.min = value;
this.saveInTheBackground();
}
public reset(): void {
this.documents = [];
this.lastSeenUpdateIds = new CoveredValues(
0 // the first updateId will be 1 which is the first integer after -1
);
this.saveInTheBackground();
}
public async save(): Promise<void> {
return this.saveData({
documents: this.resolvedDocuments.map(
({ relativePath, metadata }) => ({
relativePath,
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
...metadata! // `resolvedDocuments` only returns docs with metadata set
})
),
lastSeenUpdateId: this.lastSeenUpdateIds.min
});
}
private ensureConsistency(): void {
const idToPath = new Map<string, string[]>();
this.resolvedDocuments.forEach(({ relativePath, metadata }) => {
if (metadata === undefined) {
return;
}
idToPath.set(metadata.documentId, [
...(idToPath.get(metadata.documentId) ?? []),
relativePath
]);
});
const duplicates = Array.from(idToPath.entries())
.filter(([_, paths]) => paths.length > 1)
.map(([id, paths]) => {
let details = "";
for (const path of paths) {
const doc = this.getLatestDocumentByRelativePath(path);
details += `\n- ${JSON.stringify(doc, null, 2)}`;
}
return `${id} (${paths.join(", ")}): ${details}`;
});
if (duplicates.length > 0) {
throw new Error(
"Document IDs are not unique, found duplicates: " +
duplicates.join("; ")
);
}
}
private saveInTheBackground(): void {
this.ensureConsistency();
void this.save().catch((error: unknown) => {
this.logger.error(`Error saving data: ${error}`);
});
}
}
// This file is intentionally empty
// All document tracking has been moved to sync-event-queue.ts

View file

@ -2,13 +2,14 @@ import type {
DocumentId,
RelativePath,
VaultUpdateId
} from "../persistence/database";
} from "../sync-operations/types";
import type { Logger } from "../tracing/logger";
import type { Settings } from "../persistence/settings";
import type { FetchController } from "./fetch-controller";
import { sleep } from "../utils/sleep";
import { SyncResetError } from "../errors/sync-reset-error";
import { HttpClientError } from "../errors/http-client-error";
import type { SerializedError } from "./types/SerializedError";
import type { DocumentVersionWithoutContent } from "./types/DocumentVersionWithoutContent";
import type { DocumentUpdateResponse } from "./types/DocumentUpdateResponse";
@ -139,13 +140,7 @@ export class SyncService {
}
);
if (!response.ok) {
throw new Error(
`Failed to update document: ${await SyncService.errorFromResponse(
response
)}`
);
}
await SyncService.throwIfNotOk(response, "update document");
const result: DocumentUpdateResponse =
(await response.json()) as DocumentUpdateResponse; // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
@ -192,13 +187,7 @@ export class SyncService {
}
);
if (!response.ok) {
throw new Error(
`Failed to update document: ${await SyncService.errorFromResponse(
response
)}`
);
}
await SyncService.throwIfNotOk(response, "update document");
const result: DocumentUpdateResponse =
(await response.json()) as DocumentUpdateResponse; // eslint-disable-line @typescript-eslint/no-unsafe-type-assertion
@ -413,8 +402,10 @@ export class SyncService {
try {
return await fn();
} catch (e) {
// We must not retry errors coming from reset
if (e instanceof SyncResetError) {
if (
e instanceof SyncResetError ||
e instanceof HttpClientError
) {
throw e;
}
@ -427,4 +418,16 @@ export class SyncService {
}
}
}
private static async throwIfNotOk(
response: Response,
operation: string
): Promise<void> {
if (response.ok) return;
const message = `Failed to ${operation}: ${await SyncService.errorFromResponse(response)}`;
if (response.status >= 400 && response.status < 500) {
throw new HttpClientError(response.status, message);
}
throw new Error(message);
}
}

View file

@ -0,0 +1,7 @@
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.
import type { VaultInfo } from "./VaultInfo";
/**
* Response to listing vaults accessible to the authenticated user.
*/
export interface ListVaultsResponse { vaults: VaultInfo[], hasMore: boolean, userName: string, }

View file

@ -0,0 +1,6 @@
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.
/**
* Summary of a single vault returned by the list-vaults endpoint.
*/
export interface VaultInfo { name: string, documentCount: number, createdAt: string | null, }

View file

@ -4,7 +4,6 @@ import type { WebSocketServerMessage } from "./types/WebSocketServerMessage";
import type { WebSocketClientMessage } from "./types/WebSocketClientMessage";
import type { CursorPositionFromClient } from "./types/CursorPositionFromClient";
import type { ClientCursors } from "./types/ClientCursors";
import { createPromise } from "../utils/create-promise";
import type { WebSocketVaultUpdate } from "./types/WebSocketVaultUpdate";
import {
WEBSOCKET_DISCONNECT_TIMEOUT_IN_SECONDS,
@ -42,6 +41,10 @@ export class WebSocketManager {
private readonly webSocketFactoryImplementation: typeof globalThis.WebSocket = WebSocket
) {}
public get hasOutstandingWork(): boolean {
return this.outstandingPromises.length > 0;
}
public get isWebSocketConnected(): boolean {
return (
this.webSocket?.readyState ===
@ -55,7 +58,7 @@ export class WebSocketManager {
}
public async stop(): Promise<void> {
const [promise, resolve] = createPromise();
const { promise, resolve } = Promise.withResolvers<void>();
this.resolveDisconnectingPromise = resolve;
this.isStopped = true;

View file

@ -2,8 +2,8 @@ import type { PersistenceProvider } from "./persistence/persistence";
import type { HistoryEntry, HistoryStats } from "./tracing/sync-history";
import { SyncHistory } from "./tracing/sync-history";
import { Logger, LogLevel, LogLine } from "./tracing/logger";
import type { RelativePath, StoredDatabase } from "./persistence/database";
import { Database } from "./persistence/database";
import type { RelativePath, StoredSyncState } from "./sync-operations/types";
import { SyncEventQueue } from "./sync-operations/sync-event-queue";
import * as Sentry from "@sentry/browser";
import type { SyncSettings } from "./persistence/settings";
import { DEFAULT_SETTINGS, Settings } from "./persistence/settings";
@ -12,7 +12,6 @@ import { Syncer } from "./sync-operations/syncer";
import type { FileSystemOperations } from "./file-operations/filesystem-operations";
import { FileOperations } from "./file-operations/file-operations";
import { FetchController } from "./services/fetch-controller";
import { UnrestrictedSyncer } from "./sync-operations/unrestricted-syncer";
import { rateLimit } from "./utils/rate-limit";
import type { NetworkConnectionStatus } from "./types/network-connection-status";
import { DocumentSyncStatus } from "./types/document-sync-status";
@ -40,7 +39,7 @@ export class SyncClient {
public readonly logger: Logger,
private readonly history: SyncHistory,
private readonly settings: Settings,
private readonly database: Database,
private readonly syncEventQueue: SyncEventQueue,
private readonly syncer: Syncer,
private readonly webSocketManager: WebSocketManager,
private readonly fetchController: FetchController,
@ -52,13 +51,13 @@ export class SyncClient {
private readonly persistence: PersistenceProvider<
Partial<{
settings: Partial<SyncSettings>;
database: Partial<StoredDatabase>;
database: Partial<StoredSyncState>;
}>
>
) { }
public get documentCount(): number {
return this.database.length;
return this.syncEventQueue.documentCount;
}
public get isWebSocketConnected(): boolean {
@ -111,7 +110,7 @@ export class SyncClient {
persistence: PersistenceProvider<
Partial<{
settings: Partial<SyncSettings>;
database: Partial<StoredDatabase>;
database: Partial<StoredSyncState>;
}>
>;
fetch?: typeof globalThis.fetch;
@ -136,8 +135,6 @@ export class SyncClient {
state.settings,
async (data): Promise<void> => {
state = { ...state, settings: data };
// we're not rate-limiting settings saves as (1) we need to initialise the settings to know the rate limit
// and (2) settings changes are infrequent enough that rate-limiting is not necessary
await persistence.save(state);
}
);
@ -147,7 +144,8 @@ export class SyncClient {
() => settings.getSettings().minimumSaveIntervalMs
);
const database = new Database(
const syncEventQueue = new SyncEventQueue(
settings,
logger,
state.database,
async (data): Promise<void> => {
@ -173,7 +171,7 @@ export class SyncClient {
const fileOperations = new FileOperations(
logger,
database,
syncEventQueue,
fs,
serverConfig,
nativeLineEndings
@ -182,16 +180,6 @@ export class SyncClient {
const contentCache = new FixedSizeDocumentCache(
1024 * 1024 * DIFF_CACHE_SIZE_MB
);
const unrestrictedSyncer = new UnrestrictedSyncer(
logger,
database,
settings,
syncService,
fileOperations,
history,
contentCache,
serverConfig
);
const webSocketManager = new WebSocketManager(
logger,
@ -202,17 +190,20 @@ export class SyncClient {
const syncer = new Syncer(
deviceId,
logger,
database,
settings,
webSocketManager,
fileOperations,
unrestrictedSyncer
syncService,
history,
contentCache,
serverConfig,
syncEventQueue
);
const fileChangeNotifier = new FileChangeNotifier();
const cursorTracker = new CursorTracker(
logger,
database,
syncEventQueue,
webSocketManager,
fileOperations,
fileChangeNotifier
@ -221,7 +212,7 @@ export class SyncClient {
logger,
history,
settings,
database,
syncEventQueue,
syncer,
webSocketManager,
fetchController,
@ -319,7 +310,7 @@ export class SyncClient {
/**
* Wait for the in-flight operations to finish, reset all tracking,
* and the local database but retain the settings.
* and the local state but retain the settings.
* The SyncClient can be used again after calling this method.
*/
public async reset(): Promise<void> {
@ -330,10 +321,9 @@ export class SyncClient {
);
await this.pause();
// clear all local state
this.logger.info("Resetting SyncClient's local state");
this.database.reset();
await this.database.save(); // ensure the new database reads as empty
this.syncEventQueue.resetState();
await this.syncEventQueue.save();
this.resetInMemoryState();
this.hasFinishedOfflineSync = false;
this.serverConfig.reset();
@ -362,40 +352,47 @@ export class SyncClient {
await this.settings.setSettings(value);
}
public async syncLocallyCreatedFile(
public syncLocallyCreatedFile(
relativePath: RelativePath
): Promise<void> {
): void {
this.checkIfDestroyed("syncLocallyCreatedFile");
this.fileChangeNotifier.notifyOfFileChange(relativePath);
return this.syncer.syncLocallyCreatedFile(relativePath);
this.syncer.syncLocallyCreatedFile(relativePath);
}
public async syncLocallyDeletedFile(
public syncLocallyDeletedFile(
relativePath: RelativePath
): Promise<void> {
): void {
this.checkIfDestroyed("syncLocallyDeletedFile");
this.fileChangeNotifier.notifyOfFileChange(relativePath);
return this.syncer.syncLocallyDeletedFile(relativePath);
this.syncer.syncLocallyDeletedFile(relativePath);
}
public async syncLocallyUpdatedFile({
public syncLocallyUpdatedFile({
oldPath,
relativePath
}: {
oldPath?: RelativePath;
relativePath: RelativePath;
}): Promise<void> {
}): void {
this.checkIfDestroyed("syncLocallyUpdatedFile");
this.fileChangeNotifier.notifyOfFileChange(relativePath);
return this.syncer.syncLocallyUpdatedFile({
this.syncer.syncLocallyUpdatedFile({
oldPath,
relativePath
});
}
public get hasPendingWork(): boolean {
return (
this.syncEventQueue.size > 0 ||
this.webSocketManager.hasOutstandingWork
);
}
public getDocumentSyncingStatus(
relativePath: RelativePath
): DocumentSyncStatus {
@ -426,7 +423,7 @@ export class SyncClient {
this.checkIfDestroyed("waitUntilIdle");
await this.syncer.waitUntilFinished();
await this.webSocketManager.waitUntilFinished();
await this.database.save(); // flush all changes to disk
await this.syncEventQueue.save();
}
/**
@ -436,7 +433,6 @@ export class SyncClient {
public async destroy(): Promise<void> {
this.checkIfDestroyed("destroy");
// Prevent concurrent destroy calls
if (this.isDestroying) {
this.logger.warn(
"destroy() called while already destroying, ignoring"
@ -445,14 +441,12 @@ export class SyncClient {
}
this.isDestroying = true;
// cancel everything that's in progress
await this.pause();
this.hasBeenDestroyed = true;
this.resetInMemoryState();
// Clean up event listeners to prevent memory leaks
this.eventUnsubscribers.forEach((unsubscribe) => {
unsubscribe();
});
@ -467,7 +461,6 @@ export class SyncClient {
this.checkIfDestroyed("startSyncing");
this.fetchController.finishReset();
// warm the cache
await this.serverConfig.getConfig();
await this.syncer.scheduleSyncForOfflineChanges();
@ -486,7 +479,6 @@ export class SyncClient {
private resetInMemoryState(): void {
this.history.reset();
this.contentCache.reset();
// don't reset the logger
this.cursorTracker.reset();
this.syncer.reset();
this.fileOperations.reset();

View file

@ -1,5 +1,6 @@
import type { FileOperations } from "../file-operations/file-operations";
import type { Database, RelativePath } from "../persistence/database";
import type { RelativePath } from "./types";
import type { SyncEventQueue } from "./sync-event-queue";
import type { ClientCursors } from "../services/types/ClientCursors";
import type { CursorSpan } from "../services/types/CursorSpan";
import type { DocumentWithCursors } from "../services/types/DocumentWithCursors";
@ -35,7 +36,7 @@ export class CursorTracker {
public constructor(
private readonly logger: Logger,
private readonly database: Database,
private readonly queue: SyncEventQueue,
private readonly webSocketManager: WebSocketManager,
private readonly fileOperations: FileOperations,
private readonly fileChangeNotifier: FileChangeNotifier
@ -104,21 +105,16 @@ export class CursorTracker {
for (const [relativePath, cursors] of Object.entries(
documentToCursors
)) {
const record =
this.database.getLatestDocumentByRelativePath(relativePath);
const record = this.queue.getDocument(relativePath);
if (!record) {
continue; // Let's wait for the file to be created before sending cursors
}
if (!record.metadata) {
continue; // this is a new document, no need to sync the cursors
}
documentsWithCursors.push({
relative_path: relativePath,
document_id: record.metadata.documentId,
vault_update_id: record.metadata.parentVersionId,
document_id: record.documentId,
vault_update_id: record.parentVersionId,
cursors: cursors.map(({ start, end }) => ({
start: Math.min(start, end),
end: Math.max(start, end)
@ -139,10 +135,8 @@ export class CursorTracker {
const readContent = await this.fileOperations.read(
doc.relative_path
);
const record = this.database.getLatestDocumentByRelativePath(
doc.relative_path
);
if (record?.metadata?.hash !== (await hash(readContent))) {
const record = this.queue.getDocument(doc.relative_path);
if (record?.hash !== (await hash(readContent))) {
doc.vault_update_id = null;
}
}
@ -227,9 +221,7 @@ export class CursorTracker {
private async getDocumentUpToDateness(
document: DocumentWithCursors
): Promise<DocumentUpToDateness> {
const record = this.database.getLatestDocumentByRelativePath(
document.relative_path
);
const record = this.queue.getDocument(document.relative_path);
if (!record) {
// the document of the cursor must be from the future
@ -237,13 +229,11 @@ export class CursorTracker {
}
if (
(record.metadata?.parentVersionId ?? 0) <
(document.vault_update_id ?? 0)
record.parentVersionId < (document.vault_update_id ?? 0)
) {
return DocumentUpToDateness.Later;
} else if (
(document.vault_update_id ?? 0) <
(record.metadata?.parentVersionId ?? 0)
(document.vault_update_id ?? 0) < record.parentVersionId
) {
// the document of the cursor must be from the past
return DocumentUpToDateness.Prior;
@ -253,9 +243,8 @@ export class CursorTracker {
document.relative_path
);
return this.database.getLatestDocumentByRelativePath(
document.relative_path
)?.metadata?.hash === (await hash(currentContent))
const currentRecord = this.queue.getDocument(document.relative_path);
return currentRecord?.hash === (await hash(currentContent))
? DocumentUpToDateness.UpToDate
: DocumentUpToDateness.Prior;
}

View file

@ -1,46 +1,443 @@
import { describe, it } from "node:test";
import assert from "node:assert";
import { SyncEventQueue, type SyncEvent } from "./sync-event-queue";
import { SyncEventQueue } from "./sync-event-queue";
import { Settings } from "../persistence/settings";
import { Logger } from "../tracing/logger";
import type { DocumentVersionWithoutContent } from "../services/types/DocumentVersionWithoutContent";
import { SyncEventType } from "./types";
function createQueue(ignorePatterns: string[] = []): SyncEventQueue {
const logger = new Logger();
const settings = new Settings(logger, { ignorePatterns }, async () => {});
return new SyncEventQueue(settings, logger, undefined, async () => {});
}
function fakeRemoteVersion(
documentId: string,
overrides: Partial<DocumentVersionWithoutContent> = {}
): DocumentVersionWithoutContent {
return {
vaultUpdateId: 1,
documentId,
relativePath: `${documentId}.md`,
updatedDate: "2026-01-01",
isDeleted: false,
userId: "user",
deviceId: "device",
contentSize: 100,
...overrides
};
}
describe("SyncEventQueue", () => {
it("delete collapses interleaved events for one document while leaving the other intact", () => {
const queue = new SyncEventQueue();
queue.enqueue({ type: "local-content-update", documentId: "A" });
queue.enqueue({ type: "remote-content-update", documentId: "B" });
queue.enqueue({ type: "local-content-update", documentId: "A" });
queue.enqueue({ type: "move", documentId: "A" });
queue.enqueue({ type: "remote-content-update", documentId: "A" });
queue.enqueue({ type: "delete", documentId: "A" });
queue.enqueue({ type: "local-content-update", documentId: "B" });
assert.deepStrictEqual(queue.next(), { type: "delete", documentId: "A" });
assert.deepStrictEqual(queue.next(), {
type: "local-content-update",
documentId: "B"
it("sync-local followed by delete for the same document returns only the delete", () => {
const queue = createQueue();
queue.setDocument("a.md", {
documentId: "A",
parentVersionId: 1,
hash: "hash-a"
});
queue.enqueue({ type: SyncEventType.SyncLocal, documentId: "A" });
queue.enqueue({ type: SyncEventType.SyncLocal, documentId: "A" });
queue.enqueue({
type: SyncEventType.Delete,
documentId: "A",
path: "a.md",
});
const event = queue.next();
assert.strictEqual(event?.type, SyncEventType.Delete);
if (event?.type === SyncEventType.Delete) {
assert.strictEqual(event.documentId, "A");
}
assert.strictEqual(queue.next(), undefined);
});
it("updates coalesce up to a move boundary then post-move events are processed separately", () => {
const queue = new SyncEventQueue();
queue.enqueue({ type: "local-content-update", documentId: "X" });
queue.enqueue({ type: "remote-content-update", documentId: "X" });
queue.enqueue({ type: "file-create", path: "new.md" });
queue.enqueue({ type: "local-content-update", documentId: "X" });
queue.enqueue({ type: "move", documentId: "X" });
queue.enqueue({ type: "remote-content-update", documentId: "X" });
queue.enqueue({ type: "local-content-update", documentId: "X" });
it("sync-local events for the same document coalesce to one", () => {
const queue = createQueue();
queue.setDocument("a.md", {
documentId: "A",
parentVersionId: 1,
hash: "hash-a"
});
assert.deepStrictEqual(queue.next(), {
type: "local-content-update",
documentId: "X"
});
assert.deepStrictEqual(queue.next(), { type: "file-create", path: "new.md" });
assert.deepStrictEqual(queue.next(), { type: "move", documentId: "X" });
assert.deepStrictEqual(queue.next(), {
type: "local-content-update",
documentId: "X"
});
queue.enqueue({ type: SyncEventType.SyncLocal, documentId: "A" });
queue.enqueue({ type: SyncEventType.SyncLocal, documentId: "A" });
queue.enqueue({ type: SyncEventType.SyncLocal, documentId: "A" });
const event = queue.next();
assert.strictEqual(event?.type, SyncEventType.SyncLocal);
assert.strictEqual(queue.next(), undefined);
});
it("sync-remote events for the same documentId coalesce to the last one", () => {
const queue = createQueue();
queue.enqueue({
type: SyncEventType.SyncRemote,
remoteVersion: fakeRemoteVersion("A", { vaultUpdateId: 1 })
});
queue.enqueue({
type: SyncEventType.SyncRemote,
remoteVersion: fakeRemoteVersion("A", { vaultUpdateId: 2 })
});
queue.enqueue({
type: SyncEventType.SyncRemote,
remoteVersion: fakeRemoteVersion("A", { vaultUpdateId: 3 })
});
const event = queue.next();
assert.strictEqual(event?.type, SyncEventType.SyncRemote);
if (event?.type === SyncEventType.SyncRemote) {
assert.strictEqual(event.remoteVersion.vaultUpdateId, 3);
}
assert.strictEqual(queue.next(), undefined);
});
it("create events are returned FIFO", () => {
const queue = createQueue();
queue.enqueue({ type: SyncEventType.Create, path: "a.md" });
queue.enqueue({ type: SyncEventType.Create, path: "b.md" });
const first = queue.next();
assert.strictEqual(first?.type, SyncEventType.Create);
if (first?.type === SyncEventType.Create) {
assert.strictEqual(first.path, "a.md");
}
const second = queue.next();
assert.strictEqual(second?.type, SyncEventType.Create);
if (second?.type === SyncEventType.Create) {
assert.strictEqual(second.path, "b.md");
}
});
it("duplicate creates for the same path are skipped", () => {
const queue = createQueue();
queue.enqueue({ type: SyncEventType.Create, path: "a.md" });
queue.enqueue({ type: SyncEventType.Create, path: "a.md" });
assert.strictEqual(queue.size, 1);
});
it("create is skipped if the path already has a tracked document", () => {
const queue = createQueue();
queue.setDocument("a.md", {
documentId: "A",
parentVersionId: 1,
hash: "hash-a"
});
queue.enqueue({ type: SyncEventType.Create, path: "a.md" });
assert.strictEqual(queue.size, 0);
});
it("delete uses the provided documentId", () => {
const queue = createQueue();
queue.enqueue({
type: SyncEventType.Delete,
documentId: "A",
path: "a.md",
});
const event = queue.next();
assert.strictEqual(event?.type, SyncEventType.Delete);
if (event?.type === SyncEventType.Delete) {
assert.strictEqual(event.documentId, "A");
}
});
it("updateCreatePath updates the path of a create event in the queue", () => {
const queue = createQueue();
queue.enqueue({ type: SyncEventType.Create, path: "old.md" });
const updated = queue.updateCreatePath("old.md", "new.md");
assert.strictEqual(updated, true);
assert.strictEqual(queue.hasCreateEvent("old.md"), false);
assert.strictEqual(queue.hasCreateEvent("new.md"), true);
const event = queue.next();
assert.strictEqual(event?.type, SyncEventType.Create);
if (event?.type === SyncEventType.Create) {
assert.strictEqual(event.path, "new.md");
}
});
it("updateCreatePath returns false when no create event exists", () => {
const queue = createQueue();
const updated = queue.updateCreatePath("old.md", "new.md");
assert.strictEqual(updated, false);
});
it("hasCreateEvent detects pending creates", () => {
const queue = createQueue();
assert.strictEqual(queue.hasCreateEvent("a.md"), false);
queue.enqueue({ type: SyncEventType.Create, path: "a.md" });
assert.strictEqual(queue.hasCreateEvent("a.md"), true);
queue.next();
assert.strictEqual(queue.hasCreateEvent("a.md"), false);
});
it("document store CRUD operations work correctly", () => {
const queue = createQueue();
assert.strictEqual(queue.getDocument("a.md"), undefined);
assert.strictEqual(queue.documentCount, 0);
queue.setDocument("a.md", {
documentId: "A",
parentVersionId: 1,
hash: "hash-a"
});
assert.strictEqual(queue.documentCount, 1);
assert.deepStrictEqual(queue.getDocument("a.md"), {
documentId: "A",
parentVersionId: 1,
hash: "hash-a"
});
const found = queue.getDocumentByDocumentId("A");
assert.strictEqual(found?.path, "a.md");
assert.strictEqual(found?.record.documentId, "A");
queue.removeDocument("a.md");
assert.strictEqual(queue.documentCount, 0);
assert.strictEqual(queue.getDocument("a.md"), undefined);
});
it("moveDocument moves a document and returns displaced documentId", () => {
const queue = createQueue();
queue.setDocument("a.md", {
documentId: "A",
parentVersionId: 1,
hash: "hash-a"
});
queue.setDocument("b.md", {
documentId: "B",
parentVersionId: 2,
hash: "hash-b"
});
const displacedId = queue.moveDocument("a.md", "b.md");
assert.strictEqual(displacedId, "B");
assert.strictEqual(queue.getDocument("a.md"), undefined);
assert.strictEqual(queue.getDocument("b.md")?.documentId, "A");
assert.strictEqual(queue.documentCount, 1);
});
it("moveDocument returns undefined when target is unoccupied", () => {
const queue = createQueue();
queue.setDocument("a.md", {
documentId: "A",
parentVersionId: 1,
hash: "hash-a"
});
const displacedId = queue.moveDocument("a.md", "b.md");
assert.strictEqual(displacedId, undefined);
assert.strictEqual(queue.getDocument("b.md")?.documentId, "A");
});
it("interleaved events for different documents are not confused", () => {
const queue = createQueue();
queue.setDocument("a.md", {
documentId: "A",
parentVersionId: 1,
hash: "hash-a"
});
queue.setDocument("b.md", {
documentId: "B",
parentVersionId: 2,
hash: "hash-b"
});
queue.enqueue({ type: SyncEventType.SyncLocal, documentId: "A" });
queue.enqueue({ type: SyncEventType.SyncLocal, documentId: "B" });
queue.enqueue({
type: SyncEventType.Delete,
documentId: "A",
path: "a.md",
});
queue.enqueue({ type: SyncEventType.SyncLocal, documentId: "B" });
// First next() should see the delete for A (coalescing sync-local + delete)
const first = queue.next();
assert.strictEqual(first?.type, SyncEventType.Delete);
if (first?.type === SyncEventType.Delete) {
assert.strictEqual(first.documentId, "A");
}
// Remaining should be the coalesced sync-local for B
const second = queue.next();
assert.strictEqual(second?.type, SyncEventType.SyncLocal);
if (second?.type === SyncEventType.SyncLocal) {
assert.strictEqual(second.documentId, "B");
}
assert.strictEqual(queue.next(), undefined);
});
it("delete discards subsequent sync-remote events for the same document", () => {
const queue = createQueue();
queue.enqueue({
type: SyncEventType.Delete,
documentId: "A",
path: "a.md",
});
queue.enqueue({
type: SyncEventType.SyncRemote,
remoteVersion: fakeRemoteVersion("A", { vaultUpdateId: 5 })
});
const event = queue.next();
assert.strictEqual(event?.type, SyncEventType.Delete);
assert.strictEqual(queue.next(), undefined);
});
it("delete discards subsequent sync-local and sync-remote for the same document", () => {
const queue = createQueue();
queue.setDocument("a.md", {
documentId: "A",
parentVersionId: 1,
hash: "hash-a"
});
queue.enqueue({
type: SyncEventType.Delete,
documentId: "A",
path: "a.md",
});
queue.enqueue({ type: SyncEventType.SyncLocal, documentId: "A" });
queue.enqueue({ type: SyncEventType.Create, path: "b.md" });
queue.enqueue({
type: SyncEventType.SyncRemote,
remoteVersion: fakeRemoteVersion("A", { vaultUpdateId: 5 })
});
const first = queue.next();
assert.strictEqual(first?.type, SyncEventType.Delete);
// Only the unrelated create should remain
const second = queue.next();
assert.strictEqual(second?.type, SyncEventType.Create);
assert.strictEqual(queue.next(), undefined);
});
it("delete with empty documentId does not discard other events", () => {
const queue = createQueue();
queue.setDocument("a.md", {
documentId: "A",
parentVersionId: 1,
hash: "hash-a"
});
queue.enqueue({
type: SyncEventType.Delete,
documentId: "",
path: "unknown.md",
});
queue.enqueue({ type: SyncEventType.SyncLocal, documentId: "A" });
queue.next();
const second = queue.next();
assert.strictEqual(second?.type, SyncEventType.SyncLocal);
});
it("create can be re-enqueued after being dequeued", () => {
const queue = createQueue();
queue.enqueue({ type: SyncEventType.Create, path: "a.md" });
queue.next();
queue.enqueue({ type: SyncEventType.Create, path: "a.md" });
assert.strictEqual(queue.size, 1);
});
it("silently ignores create events matching ignore patterns", () => {
const queue = createQueue(["*.tmp", ".hidden/**"]);
queue.enqueue({ type: SyncEventType.Create, path: "scratch.tmp" });
queue.enqueue({
type: SyncEventType.Create,
path: ".hidden/secret.md",
});
assert.strictEqual(queue.size, 0);
queue.enqueue({ type: SyncEventType.Create, path: "notes-new.md" });
assert.strictEqual(queue.size, 1);
queue.enqueue({
type: SyncEventType.SyncRemote,
remoteVersion: fakeRemoteVersion("N")
});
assert.strictEqual(queue.size, 2);
});
it("clear removes events but keeps documents", () => {
const queue = createQueue();
queue.setDocument("a.md", {
documentId: "A",
parentVersionId: 1,
hash: "hash-a"
});
queue.enqueue({ type: SyncEventType.Create, path: "b.md" });
queue.enqueue({ type: SyncEventType.SyncLocal, documentId: "A" });
assert.strictEqual(queue.size, 2);
queue.clear();
assert.strictEqual(queue.size, 0);
assert.strictEqual(queue.documentCount, 1);
assert.strictEqual(queue.getDocument("a.md")?.documentId, "A");
});
it("allDocuments returns all tracked documents", () => {
const queue = createQueue();
queue.setDocument("a.md", {
documentId: "A",
parentVersionId: 1,
hash: "hash-a"
});
queue.setDocument("b.md", {
documentId: "B",
parentVersionId: 2,
hash: "hash-b"
});
const docs = queue.allDocuments();
assert.strictEqual(docs.length, 2);
const paths = docs.map(([p]) => p).sort();
assert.deepStrictEqual(paths, ["a.md", "b.md"]);
});
it("loads initial state from persistence", () => {
const logger = new Logger();
const settings = new Settings(logger, {}, async () => {});
const queue = new SyncEventQueue(settings, logger, {
documents: [
{
relativePath: "a.md",
documentId: "A",
parentVersionId: 5,
hash: "hash-a"
},
{
relativePath: "b.md",
documentId: "B",
parentVersionId: 3,
hash: "hash-b"
}
],
lastSeenUpdateId: 4
}, async () => {});
assert.strictEqual(queue.documentCount, 2);
assert.strictEqual(queue.getDocument("a.md")?.documentId, "A");
assert.strictEqual(queue.getDocument("b.md")?.documentId, "B");
assert.strictEqual(queue.getLastSeenUpdateId(), 5);
});
});

View file

@ -1,85 +1,336 @@
import type { DocumentId, RelativePath } from "../persistence/database";
export type SyncEvent =
| { type: "file-create"; path: RelativePath }
| { type: "local-content-update"; documentId: DocumentId }
| { type: "remote-content-update"; documentId: DocumentId }
| { type: "move"; documentId: DocumentId }
| { type: "delete"; documentId: DocumentId };
import type { Settings } from "../persistence/settings";
import type { Logger } from "../tracing/logger";
import { globsToRegexes } from "../utils/globs-to-regexes";
import { CoveredValues } from "../utils/data-structures/min-covered";
import { removeFromArray } from "../utils/remove-from-array";
import {
SyncEventType,
type DocumentId,
type DocumentRecord,
type RelativePath,
type StoredSyncState,
type SyncEvent,
type VaultUpdateId,
} from "./types";
export class SyncEventQueue {
private readonly events: SyncEvent[] = [];
private readonly documents = new Map<RelativePath, DocumentRecord>();
private readonly recentlyDeletedDocumentIds = new Set<DocumentId>();
private lastSeenUpdateIds: CoveredValues;
private ignorePatterns: RegExp[];
public constructor(
private readonly settings: Settings,
private readonly logger: Logger,
initialState: Partial<StoredSyncState> | undefined,
private readonly saveData: (data: StoredSyncState) => Promise<void>
) {
this.ignorePatterns = globsToRegexes(
this.settings.getSettings().ignorePatterns,
this.logger
);
this.settings.onSettingsChanged.add((newSettings) => {
this.ignorePatterns = globsToRegexes(
newSettings.ignorePatterns,
this.logger
);
});
initialState ??= {};
if (initialState.documents !== undefined) {
for (const { relativePath, ...record } of initialState.documents) {
this.documents.set(relativePath, record);
}
}
const { lastSeenUpdateId } = initialState;
this.lastSeenUpdateIds = new CoveredValues(
Math.max(0, lastSeenUpdateId ?? 0)
);
for (const [, record] of this.documents) {
this.lastSeenUpdateIds.add(record.parentVersionId);
}
this.logger.debug(`Loaded ${this.documents.size} documents`);
}
public get size(): number {
return this.events.length;
}
public get documentCount(): number {
return this.documents.size;
}
public getLastSeenUpdateId(): VaultUpdateId {
return this.lastSeenUpdateIds.min;
}
public addSeenUpdateId(value: number): void {
const previousMin = this.lastSeenUpdateIds.min;
this.lastSeenUpdateIds.add(value);
if (previousMin !== this.lastSeenUpdateIds.min) {
this.saveInTheBackground();
}
}
public setLastSeenUpdateId(value: number): void {
this.lastSeenUpdateIds.min = value;
this.saveInTheBackground();
}
public getDocument(path: RelativePath): DocumentRecord | undefined {
return this.documents.get(path);
}
public getDocumentByDocumentId(
target: DocumentId
): { path: RelativePath; record: DocumentRecord } | undefined {
for (const [path, record] of this.documents) {
if (record.documentId === target) {
return { path, record };
}
}
return undefined;
}
public setDocument(path: RelativePath, record: DocumentRecord): void {
this.documents.set(path, record);
this.saveInTheBackground();
}
public removeDocument(path: RelativePath): void {
const record = this.documents.get(path);
if (record !== undefined) {
this.recentlyDeletedDocumentIds.add(record.documentId);
}
this.documents.delete(path);
this.saveInTheBackground();
}
/**
* Move a document from oldPath to newPath.
* If the target path is occupied by a different document, it is removed
* and its documentId is returned so the caller can handle the displacement.
*/
public moveDocument(
oldPath: RelativePath,
newPath: RelativePath
): DocumentId | undefined {
const record = this.documents.get(oldPath);
if (record === undefined) return undefined;
let displacedDocumentId: DocumentId | undefined = undefined;
const existingAtTarget = this.documents.get(newPath);
if (
existingAtTarget !== undefined &&
existingAtTarget.documentId !== record.documentId
) {
displacedDocumentId = existingAtTarget.documentId;
this.recentlyDeletedDocumentIds.add(displacedDocumentId);
this.documents.delete(newPath);
}
this.documents.delete(oldPath);
this.documents.set(newPath, record);
this.saveInTheBackground();
return displacedDocumentId;
}
public wasRecentlyDeleted(documentId: DocumentId): boolean {
return this.recentlyDeletedDocumentIds.has(documentId);
}
public unmarkRecentlyDeleted(documentId: DocumentId): void {
this.recentlyDeletedDocumentIds.delete(documentId);
}
public allDocuments(): [RelativePath, DocumentRecord][] {
return Array.from(this.documents.entries());
}
public hasCreateEvent(path: RelativePath): boolean {
return this.events.some(
(e) => e.type === SyncEventType.Create && e.path === path
);
}
public updateCreatePath(
oldPath: RelativePath,
newPath: RelativePath
): boolean {
for (const event of this.events) {
if (event.type === SyncEventType.Create && event.path === oldPath) {
event.path = newPath;
return true;
}
}
return false;
}
public hasPendingEventsForPath(path: RelativePath): boolean {
const record = this.documents.get(path);
const docId = record?.documentId;
return this.events.some(
(e) =>
(e.type === SyncEventType.Create && e.path === path) ||
(e.type === SyncEventType.SyncLocal &&
docId !== undefined &&
e.documentId === docId) ||
(e.type === SyncEventType.Delete &&
docId !== undefined &&
e.documentId === docId) ||
(e.type === SyncEventType.SyncRemote &&
e.remoteVersion.relativePath === path)
);
}
public async save(): Promise<void> {
return this.saveData({
documents: Array.from(this.documents.entries()).map(
([relativePath, record]) => ({
relativePath,
...record
})
),
lastSeenUpdateId: this.lastSeenUpdateIds.min
});
}
public resetState(): void {
this.documents.clear();
this.recentlyDeletedDocumentIds.clear();
this.lastSeenUpdateIds = new CoveredValues(0);
this.saveInTheBackground();
}
public clear(): void {
this.events.length = 0;
this.recentlyDeletedDocumentIds.clear();
}
public enqueue(event: SyncEvent): void {
if (this.isIgnored(event)) return;
if (event.type === SyncEventType.Create) {
if (this.documents.has(event.path)) return;
if (this.hasCreateEvent(event.path)) return;
}
this.events.push(event);
}
public next(): SyncEvent | undefined {
if (this.events.length === 0) return undefined;
const first = this.events[0];
if (first.type === "file-create") {
const [first] = this.events;
// Creates are always returned immediately (FIFO)
if (first.type === SyncEventType.Create) {
this.events.shift();
return first;
}
const { documentId } = first;
// If there's an eventual delete, discard everything for this document
const deleteEvent = this.events.find(
(e) => e.type === "delete" && e.documentId === documentId
);
if (deleteEvent) {
this.removeAllForDocument(documentId);
return deleteEvent;
}
// Coalesce updates: return the last update before the next move for this document.
// Moves act as barriers since they depend on each other
const moveIndex = this.events.findIndex(
(e) => e.type === "move" && e.documentId === documentId
);
const boundary = moveIndex === -1 ? this.events.length : moveIndex;
const updateIndices: number[] = [];
for (let i = 0; i < boundary; i++) {
const e = this.events[i];
if (
(e.type === "local-content-update" ||
e.type === "remote-content-update") &&
e.documentId === documentId
) {
updateIndices.push(i);
// Deletes are returned immediately; also discard any subsequent
// events for the same documentId so stale broadcasts don't
// resurrect the document
if (first.type === SyncEventType.Delete) {
this.events.shift();
const { documentId } = first;
if (documentId !== "") {
this.removeAllEventsForDocumentId(documentId);
}
return first;
}
if (updateIndices.length > 0) {
const result = this.events[updateIndices[updateIndices.length - 1]];
for (let i = updateIndices.length - 1; i >= 0; i--) {
this.events.splice(updateIndices[i], 1);
if (first.type === SyncEventType.SyncLocal) {
const { documentId } = first;
// If there's a later delete for the same documentId, discard
// all sync-locals for that document and return the delete
const deleteEvent = this.events.find(
(e) =>
e.type === SyncEventType.Delete &&
e.documentId === documentId
);
if (deleteEvent !== undefined) {
this.removeAllSyncLocalsForDocumentId(documentId);
removeFromArray(this.events, deleteEvent);
return deleteEvent;
}
// Coalesce multiple sync-locals for the same documentId to the last one
const matching = this.events.filter(
(e) =>
e.type === SyncEventType.SyncLocal &&
e.documentId === documentId
);
const result = matching[matching.length - 1];
for (const item of matching) {
removeFromArray(this.events, item);
}
return result;
}
// First event is a move with no preceding updates
this.events.shift();
return first;
// SyncRemote: coalesce multiple events for the same documentId to the last one
const { documentId } = first.remoteVersion;
const matching = this.events.filter(
(e) =>
e.type === SyncEventType.SyncRemote &&
e.remoteVersion.documentId === documentId
);
const result = matching[matching.length - 1];
for (const item of matching) {
removeFromArray(this.events, item);
}
return result;
}
private removeAllForDocument(documentId: DocumentId): void {
private isIgnored(event: SyncEvent): boolean {
if (event.type !== SyncEventType.Create) return false;
return this.ignorePatterns.some((pattern) => pattern.test(event.path));
}
private removeAllEventsForDocumentId(documentId: DocumentId): void {
for (let i = this.events.length - 1; i >= 0; i--) {
const e = this.events[i];
if (e.type !== "file-create" && e.documentId === documentId) {
if (
(e.type === SyncEventType.SyncLocal &&
e.documentId === documentId) ||
(e.type === SyncEventType.SyncRemote &&
e.remoteVersion.documentId === documentId) ||
(e.type === SyncEventType.Delete &&
e.documentId === documentId)
) {
// eslint-disable-next-line no-restricted-syntax -- Bulk removal by predicate, not single-item removal
this.events.splice(i, 1);
}
}
}
private removeAllSyncLocalsForDocumentId(documentId: DocumentId): void {
for (let i = this.events.length - 1; i >= 0; i--) {
const e = this.events[i];
if (
e.type === SyncEventType.SyncLocal &&
e.documentId === documentId
) {
// eslint-disable-next-line no-restricted-syntax -- Bulk removal by predicate, not single-item removal
this.events.splice(i, 1);
}
}
}
private saveInTheBackground(): void {
void this.save().catch((error: unknown) => {
this.logger.error(`Error saving sync state: ${error}`);
});
}
}

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,42 @@
import type { DocumentVersionWithoutContent } from "../services/types/DocumentVersionWithoutContent";
export type VaultUpdateId = number;
export type DocumentId = string;
export type RelativePath = string;
export interface DocumentRecord {
documentId: DocumentId;
parentVersionId: VaultUpdateId;
hash: string;
remoteRelativePath?: RelativePath;
}
export interface StoredDocument extends DocumentRecord {
relativePath: RelativePath;
}
export interface StoredSyncState {
documents: StoredDocument[];
lastSeenUpdateId: VaultUpdateId | undefined;
}
export enum SyncEventType {
Create = "create",
SyncLocal = "sync-local",
SyncRemote = "sync-remote",
Delete = "delete",
}
export type SyncEvent =
| { type: SyncEventType.Create; path: RelativePath }
| { type: SyncEventType.SyncLocal; documentId: DocumentId }
| {
type: SyncEventType.Delete;
documentId: DocumentId;
path: RelativePath;
displacedAtVersion?: VaultUpdateId;
}
| {
type: SyncEventType.SyncRemote;
remoteVersion: DocumentVersionWithoutContent;
};

View file

@ -1,612 +0,0 @@
import type {
Database,
DocumentRecord,
RelativePath
} from "../persistence/database";
import { diff } from "reconcile-text";
import type { SyncService } from "../services/sync-service";
import type { Logger } from "../tracing/logger";
import type {
CommonHistoryEntry,
SyncCreateDetails,
SyncDeleteDetails,
SyncDetails,
SyncHistory,
SyncMovedDetails,
SyncUpdateDetails
} from "../tracing/sync-history";
import { SyncStatus, SyncType } from "../tracing/sync-history";
import { EMPTY_HASH, hash } from "../utils/hash";
import { base64ToBytes } from "byte-base64";
import type { Settings } from "../persistence/settings";
import type { FileOperations } from "../file-operations/file-operations";
import { FileNotFoundError } from "../errors/file-not-found-error";
import { SyncResetError } from "../errors/sync-reset-error";
import { globsToRegexes } from "../utils/globs-to-regexes";
import type { DocumentVersion } from "../services/types/DocumentVersion";
import type { DocumentUpdateResponse } from "../services/types/DocumentUpdateResponse";
import type { DocumentVersionWithoutContent } from "../services/types/DocumentVersionWithoutContent";
import type { FixedSizeDocumentCache } from "../utils/data-structures/fix-sized-cache";
import { isFileTypeMergable } from "../utils/is-file-type-mergable";
import { isBinary } from "../utils/is-binary";
import type { ServerConfig } from "../services/server-config";
export class UnrestrictedSyncer {
private ignorePatterns: RegExp[];
public constructor(
private readonly logger: Logger,
private readonly database: Database,
private readonly settings: Settings,
private readonly syncService: SyncService,
private readonly operations: FileOperations,
private readonly history: SyncHistory,
private readonly contentCache: FixedSizeDocumentCache,
private readonly serverConfig: ServerConfig
) {
this.ignorePatterns = globsToRegexes(
this.settings.getSettings().ignorePatterns,
this.logger
);
this.settings.onSettingsChanged.add((newSettings) => {
this.ignorePatterns = globsToRegexes(
newSettings.ignorePatterns,
this.logger
);
});
}
public async unrestrictedSyncLocallyCreatedOrUpdatedFile({
oldPath,
// We use the same code path for both local and remote updates. We need to force the update
// if there are no local changes but we know that the remote version is newer.
force = false,
document
}: {
oldPath?: RelativePath;
force?: boolean;
document: DocumentRecord;
}): Promise<void> {
const updateDetails:
| SyncCreateDetails
| SyncUpdateDetails
| SyncMovedDetails =
document.metadata === undefined
? {
type: SyncType.CREATE,
relativePath: document.relativePath
}
: oldPath !== undefined
? {
type: SyncType.MOVE,
relativePath: document.relativePath,
movedFrom: oldPath
}
: {
type: SyncType.UPDATE,
relativePath: document.relativePath
};
await this.executeSync(updateDetails, async () => {
const originalRelativePath = document.relativePath;
if (document.isDeleted) {
this.logger.debug(
`Document ${document.relativePath} has been already deleted, no need to update it`
);
return;
}
const contentBytes = await this.operations.read(
document.relativePath
); // this can throw FileNotFoundError
const contentHash = await hash(contentBytes);
let response: DocumentVersion | DocumentUpdateResponse | undefined =
undefined;
if (document.metadata === undefined) {
response = await this.syncService.create({
relativePath: originalRelativePath,
contentBytes
});
await this.handleMaybeMergingResponse({
document,
response,
contentHash,
originalRelativePath,
originalContentBytes: contentBytes,
isCreate: true
});
} else {
const areThereLocalChanges =
document.metadata.hash !== contentHash ||
oldPath !== undefined;
if (areThereLocalChanges) {
const isText =
!isBinary(contentBytes) &&
isFileTypeMergable(
document.relativePath,
(await this.serverConfig.getConfig())
.mergeableFileExtensions
);
const cachedVersion = this.contentCache.get(
document.metadata.parentVersionId
);
response =
isText && cachedVersion !== undefined
? await this.syncService.putText({
documentId: document.metadata.documentId,
parentVersionId:
document.metadata.parentVersionId,
relativePath: document.relativePath,
content: diff(
new TextDecoder().decode(cachedVersion),
new TextDecoder().decode(contentBytes)
)
})
: await this.syncService.putBinary({
documentId: document.metadata.documentId,
parentVersionId:
document.metadata.parentVersionId,
relativePath: document.relativePath,
contentBytes
});
} else {
if (!force) {
this.logger.debug(
`File hash of ${document.relativePath} matches with last synced version and the path hasn't changed; no need to sync`
);
return;
}
// we use this code path (force == true) to sync remotely updated files which have no local changes
response = await this.syncService.get({
documentId: document.metadata.documentId
});
}
await this.handleMaybeMergingResponse({
document,
response,
contentHash,
originalRelativePath,
originalContentBytes: contentBytes
});
}
if (!("type" in response) || response.type === "MergingUpdate") {
if (!force) {
this.history.addHistoryEntry({
status: SyncStatus.SUCCESS,
details: updateDetails,
message: `The file we updated had been updated remotely, so we downloaded the merged version`
});
return;
}
}
const actualUpdateDetails: SyncUpdateDetails | SyncMovedDetails =
oldPath !== undefined ||
response.relativePath != originalRelativePath
? {
type: SyncType.MOVE,
relativePath: response.relativePath,
movedFrom: originalRelativePath
}
: {
type: SyncType.UPDATE,
relativePath: response.relativePath
};
if (!response.isDeleted) {
this.history.addHistoryEntry({
status: SyncStatus.SUCCESS,
details: actualUpdateDetails,
message: `Successfully downloaded remotely updated file from the server`,
author: response.userId,
timestamp: new Date(response.updatedDate)
});
} else {
this.history.addHistoryEntry({
status: SyncStatus.SUCCESS,
details: {
type: SyncType.DELETE,
relativePath: document.relativePath
},
message:
"Successfully deleted file which had been deleted remotely",
author: response.userId,
timestamp: new Date(response.updatedDate)
});
}
});
}
public async unrestrictedSyncLocallyDeletedFile(
document: DocumentRecord
): Promise<void> {
const updateDetails: SyncDeleteDetails = {
type: SyncType.DELETE,
relativePath: document.relativePath
};
await this.executeSync(updateDetails, async () => {
if (document.metadata === undefined) {
this.logger.debug(
`Document ${document.relativePath} has never been synced, no need to delete it remotely`
);
return;
}
const response = await this.syncService.delete({
documentId: document.metadata.documentId,
relativePath: document.relativePath
});
this.database.updateDocumentMetadata(
{
documentId: response.documentId,
parentVersionId: response.vaultUpdateId,
hash: EMPTY_HASH,
remoteRelativePath: document.relativePath
},
document
);
this.database.addSeenUpdateId(response.vaultUpdateId);
this.history.addHistoryEntry({
status: SyncStatus.SUCCESS,
details: updateDetails,
message: `Successfully deleted locally deleted file on the server`,
author: response.userId
});
});
}
public async unrestrictedSyncRemotelyUpdatedFile(
remoteVersion: DocumentVersionWithoutContent,
document?: DocumentRecord
): Promise<void> {
const updateDetails: SyncCreateDetails = {
type: SyncType.CREATE,
relativePath: remoteVersion.relativePath
};
await this.executeSync(updateDetails, async () => {
if (document?.metadata !== undefined) {
// If the file exists locally, let's pretend the user has updated it
// and deal with remote update/deletion within `unrestrictedSyncLocallyUpdatedFile`
if (
document.metadata.parentVersionId >=
remoteVersion.vaultUpdateId
) {
this.logger.debug(
`Document ${document.relativePath} is already at least as up-to-date as the fetched version`
);
return;
}
return this.unrestrictedSyncLocallyCreatedOrUpdatedFile({
document,
force: true
});
} else if (remoteVersion.isDeleted) {
// Either the document hasn't made it to us before and therefore we don't need to delete it,
// or we already have it, in which case the preceeding if would've dealt with it
this.logger.debug(
`Document ${remoteVersion.relativePath} has been deleted remotely, no need to sync`
);
return;
}
// Don't download oversized files
const historyEntryForSkippedOversizedFile =
this.getHistoryEntryForSkippedOversizedFile(
remoteVersion.contentSize,
remoteVersion.relativePath
);
if (historyEntryForSkippedOversizedFile !== undefined) {
this.history.addHistoryEntry(
historyEntryForSkippedOversizedFile
);
return;
}
const contentBytes =
await this.syncService.getDocumentVersionContent({
documentId: remoteVersion.documentId,
vaultUpdateId: remoteVersion.vaultUpdateId
});
// We're trying to create an entirely new document that didn't exist locally
document = this.database.getDocumentByDocumentId(
remoteVersion.documentId
);
// It can happen that a concurrent sync operation has already created the document, so we can bail here
if (document !== undefined) {
this.logger.debug(
`Document ${remoteVersion.relativePath} has already been created locally, no need to create it again`
);
return;
}
await this.operations.ensureClearPath(remoteVersion.relativePath);
this.database.updateDocumentMetadata(
{
documentId: remoteVersion.documentId,
parentVersionId: remoteVersion.vaultUpdateId,
hash: await hash(contentBytes),
remoteRelativePath: remoteVersion.relativePath
},
this.database.createNewPendingDocument(
remoteVersion.relativePath
)
);
await this.operations.create(
remoteVersion.relativePath,
contentBytes
);
await this.updateCache(
remoteVersion.vaultUpdateId,
contentBytes,
remoteVersion.relativePath
);
this.history.addHistoryEntry({
status: SyncStatus.SUCCESS,
details: updateDetails,
message: `Successfully downloaded remote file which hadn't existed locally`,
author: remoteVersion.userId,
timestamp: new Date(remoteVersion.updatedDate)
});
});
}
private async executeSync<T>(
details: SyncDetails,
fn: () => Promise<T>
): Promise<T | undefined> {
if (!this.settings.getSettings().isSyncEnabled) {
this.logger.info(
`Skipping sync operation for file '${details.relativePath}' because sync is disabled`
);
return;
}
for (const pattern of this.ignorePatterns) {
if (pattern.test(details.relativePath)) {
this.logger.debug(
`File '${details.relativePath}' is ignored by the ignore pattern: ${pattern}`
);
return; // bail without SKIPPED status because we were told to ignore this file and we shouldn't clutter up the history
}
}
try {
// Only check the size of files which already exist locally.
if (await this.operations.exists(details.relativePath)) {
const sizeInBytes = await this.operations.getFileSize(
details.relativePath
);
const historyEntryForSkippedOversizedFile =
this.getHistoryEntryForSkippedOversizedFile(
sizeInBytes,
details.relativePath
);
if (historyEntryForSkippedOversizedFile !== undefined) {
this.history.addHistoryEntry(
historyEntryForSkippedOversizedFile
);
return;
}
}
return await fn();
} catch (e) {
if (e instanceof FileNotFoundError) {
// A subsequent sync operation must have been creating to deal with this
this.logger.info(
`Skiping file '${details.relativePath}' because it no longer exists when trying to ${details.type.toLocaleLowerCase()} it`
);
return;
}
if (e instanceof SyncResetError) {
this.logger.info(
`Interrupting sync operation because of a reset`
);
return;
} else {
this.history.addHistoryEntry({
status: SyncStatus.ERROR,
details,
message: `Failed to sync file '${details.relativePath}' because of ${e} when trying to ${details.type.toLocaleLowerCase()} it`
});
throw e;
}
}
}
private async handleMaybeMergingResponse({
document,
response,
contentHash,
originalRelativePath,
originalContentBytes,
isCreate
}: {
document: DocumentRecord;
response: DocumentVersion | DocumentUpdateResponse;
contentHash: string;
originalRelativePath: string;
originalContentBytes: Uint8Array;
isCreate?: boolean;
}): Promise<void> {
// `document` is mutable and reflects the latest state in the local database
if (document.isDeleted) {
this.logger.info(
`Document ${document.relativePath} has been deleted before we could finish updating it`
);
this.database.addSeenUpdateId(response.vaultUpdateId);
return;
}
if (
(document.metadata?.parentVersionId ?? 0) > response.vaultUpdateId
) {
this.logger.debug(
`Document ${document.relativePath} is already more up to date than the fetched version`
);
this.database.addSeenUpdateId(response.vaultUpdateId); // in case the previous `vaultUpdateId` update hasn't made it through
return;
}
if (response.isDeleted) {
return this.applyRemoteDeleteLocally(document, response);
}
let actualPath = document.relativePath;
if (isCreate) {
// We have a file locally that got moved by another client to the same path as the one we're trying to create.
// The server returns a merging update for the document ID that already exists locally (but at another path).
// We have to merge these two documents by extending the provenance of the existing document and deleting
// the old document that the new document already contains the content for.
const existingDocument = this.database.getDocumentByDocumentId(
response.documentId
);
if (existingDocument !== undefined) {
this.logger.info(
`Merging existing document ${existingDocument.relativePath} into ${document.relativePath
} after concurrent move & creation`
);
if (!existingDocument.isDeleted) {
this.database.delete(existingDocument.relativePath); // make sure syncLocallyDeletedFile doesn't actually schedule deleting the new file
this.database.removeDocument(existingDocument);
await this.operations.move(existingDocument.relativePath, document.relativePath);
} else {
this.database.removeDocument(existingDocument);
}
}
}
// this can't happen on the creation path as we can only get a merging response if a document already exists remotely on the same path
if (response.relativePath != originalRelativePath) {
actualPath = response.relativePath;
// Make sure to update the remote relative path to avoid uploading
// the file as a result of this filesystem event.
if (document.metadata !== undefined) {
document.metadata.remoteRelativePath = response.relativePath;
}
await this.operations.move(
document.relativePath,
response.relativePath
); // this can throw FileNotFoundError
}
if (!("type" in response) || response.type === "MergingUpdate") {
const responseBytes = base64ToBytes(response.contentBase64);
contentHash = await hash(responseBytes);
this.database.updateDocumentMetadata(
{
documentId: response.documentId,
parentVersionId: response.vaultUpdateId,
hash: contentHash,
remoteRelativePath: response.relativePath
},
document
);
await this.operations.write(
actualPath,
originalContentBytes,
responseBytes
);
await this.updateCache(
response.vaultUpdateId,
responseBytes,
actualPath
);
} else {
this.database.updateDocumentMetadata(
{
documentId: response.documentId,
parentVersionId: response.vaultUpdateId,
hash: contentHash,
remoteRelativePath: response.relativePath
},
document
);
await this.updateCache(
response.vaultUpdateId,
originalContentBytes,
actualPath
);
}
this.database.addSeenUpdateId(response.vaultUpdateId);
}
private getHistoryEntryForSkippedOversizedFile(
sizeInBytes: number,
relativePath: RelativePath
): CommonHistoryEntry | undefined {
const sizeInMB = Math.round(sizeInBytes / 1024 / 1024);
const { maxFileSizeMB } = this.settings.getSettings();
if (sizeInMB > maxFileSizeMB) {
return {
status: SyncStatus.SKIPPED,
details: {
type: SyncType.SKIPPED,
relativePath
},
message: `File size of ${sizeInMB} MB exceeds the maximum file size limit of ${maxFileSizeMB
} MB`
};
}
}
private async updateCache(
updateId: number,
contentBytes: Uint8Array,
filePath: RelativePath
): Promise<void> {
if (
isFileTypeMergable(
filePath,
(await this.serverConfig.getConfig()).mergeableFileExtensions
) &&
!isBinary(contentBytes)
) {
this.contentCache.put(updateId, contentBytes);
}
}
private async applyRemoteDeleteLocally(
document: DocumentRecord,
response: DocumentVersion | DocumentUpdateResponse
): Promise<void> {
this.database.delete(document.relativePath);
this.database.updateDocumentMetadata(
{
documentId: response.documentId,
parentVersionId: response.vaultUpdateId,
hash: EMPTY_HASH,
remoteRelativePath: response.relativePath
},
document
);
await this.operations.delete(document.relativePath);
this.database.addSeenUpdateId(response.vaultUpdateId);
}
}

6
package-lock.json generated Normal file
View file

@ -0,0 +1,6 @@
{
"name": "vault-link",
"lockfileVersion": 3,
"requires": true,
"packages": {}
}

View file

@ -29,9 +29,9 @@ echo "Stopping existing server..."
pkill -f "sync_server" 2>/dev/null || true
sleep 1
# Clean databases
# Clean databases (uses tmpfs via /dev/shm for zero disk I/O)
echo "Cleaning databases..."
rm -rf databases
rm -rf /host/tmp/vaultlink-e2e-databases
# Start the server in the background
echo "Starting server..."

View file

@ -1,5 +1,16 @@
// generated by `sqlx migrate build-script`
fn main() {
// trigger recompilation when a new migration is added
println!("cargo:rerun-if-changed=migrations");
// Ensure the history-ui dist directory exists so rust-embed can compile
// even when the frontend hasn't been built yet.
let dist_path = std::path::Path::new("../frontend/history-ui/dist");
if !dist_path.exists() {
std::fs::create_dir_all(dist_path).expect("Failed to create history-ui dist directory");
std::fs::write(
dist_path.join("index.html"),
"<!DOCTYPE html><html><body><p>Run <code>npm run build -w history-ui</code> first.</p></body></html>",
)
.expect("Failed to write placeholder index.html");
}
}

View file

@ -0,0 +1,2 @@
CREATE INDEX IF NOT EXISTS idx_documents_document_id
ON documents (document_id, vault_update_id);

View file

@ -33,7 +33,7 @@ pub fn get_authenticated_handshake(
let user = auth(state, handshake.token.trim(), vault_id)?;
Ok(AuthenticatedWebSocketHandshake { handshake, user })
}
WebSocketClientMessage::CursorPositions(_) | WebSocketClientMessage::Ping {} => Err(
WebSocketClientMessage::CursorPositions(_) => Err(
unauthenticated_error(anyhow::anyhow!("Expected a handshake message")),
),
}

View file

@ -4,24 +4,27 @@ mod delete_document;
mod device_id_header;
mod fetch_document_version;
mod fetch_document_version_content;
mod fetch_document_versions;
mod fetch_latest_document_version;
mod fetch_latest_documents;
mod fetch_vault_history;
mod index;
mod list_vaults;
mod ping;
mod rate_limit;
mod requests;
mod responses;
mod restore_document_version;
mod update_document;
mod websocket;
use anyhow::{Context as _, Result, anyhow};
use anyhow::{Context as _, Result};
use auth::auth_middleware;
use axum::{
Router,
extract::{DefaultBodyLimit, Request},
http::{self, HeaderValue, Method},
middleware,
response::IntoResponse,
routing::{IntoMakeService, delete, get, post, put},
};
use device_id_header::DEVICE_ID_HEADER_NAME;
@ -52,7 +55,7 @@ pub async fn create_server(config: Config) -> Result<()> {
let server_config = app_state.config.server.clone();
let app = Router::new()
let mut app = Router::new()
.nest("/", get_authed_routes(app_state.clone()))
.route("/", get(index::index))
.route("/assets/*path", get(index::spa_assets))
@ -155,6 +158,10 @@ fn get_authed_routes(app_state: AppState) -> Router<AppState> {
"/vaults/:vault_id/documents/:document_id/text",
put(update_document::update_text),
)
.route(
"/vaults/:vault_id/documents/:document_id/versions",
get(fetch_document_versions::fetch_document_versions),
)
.route(
"/vaults/:vault_id/documents/:document_id/versions/:vault_update_id",
get(fetch_document_version::fetch_document_version),
@ -167,6 +174,14 @@ fn get_authed_routes(app_state: AppState) -> Router<AppState> {
"/vaults/:vault_id/documents/:document_id",
delete(delete_document::delete_document),
)
.route(
"/vaults/:vault_id/documents/:document_id/restore",
post(restore_document_version::restore_document_version),
)
.route(
"/vaults/:vault_id/history",
get(fetch_vault_history::fetch_vault_history),
)
.layer(middleware::from_fn_with_state(app_state, auth_middleware))
}

View file

@ -0,0 +1,42 @@
use axum::{
Json,
extract::{Path, State},
};
use log::debug;
use serde::Deserialize;
use crate::{
app_state::{
AppState,
database::models::{DocumentId, DocumentVersionWithoutContent, VaultId},
},
errors::{SyncServerError, server_error},
utils::normalize::normalize,
};
#[derive(Deserialize)]
pub struct FetchDocumentVersionsPathParams {
#[serde(deserialize_with = "normalize")]
vault_id: VaultId,
document_id: DocumentId,
}
#[axum::debug_handler]
pub async fn fetch_document_versions(
Path(FetchDocumentVersionsPathParams {
vault_id,
document_id,
}): Path<FetchDocumentVersionsPathParams>,
State(state): State<AppState>,
) -> Result<Json<Vec<DocumentVersionWithoutContent>>, SyncServerError> {
debug!("Fetching all versions for document `{document_id}` in vault `{vault_id}`");
let versions = state
.database
.get_document_versions(&vault_id, &document_id, None)
.await
.map_err(server_error)?;
Ok(Json(versions))
}

View file

@ -0,0 +1,70 @@
use axum::{
Json,
extract::{Path, Query, State},
};
use log::debug;
use serde::Deserialize;
use super::responses::VaultHistoryResponse;
use crate::{
app_state::{
AppState,
database::models::{VaultId, VaultUpdateId},
},
errors::{SyncServerError, client_error, server_error},
utils::normalize::normalize,
};
const DEFAULT_LIMIT: i64 = 50;
const MAX_LIMIT: i64 = 500;
#[derive(Deserialize)]
pub struct FetchVaultHistoryPathParams {
#[serde(deserialize_with = "normalize")]
vault_id: VaultId,
}
#[derive(Deserialize)]
pub struct QueryParams {
limit: Option<i64>,
before_update_id: Option<VaultUpdateId>,
}
#[axum::debug_handler]
pub async fn fetch_vault_history(
Path(FetchVaultHistoryPathParams { vault_id }): Path<FetchVaultHistoryPathParams>,
Query(QueryParams {
limit,
before_update_id,
}): Query<QueryParams>,
State(state): State<AppState>,
) -> Result<Json<VaultHistoryResponse>, SyncServerError> {
if let Some(id) = before_update_id
&& id <= 0
{
return Err(client_error(anyhow::anyhow!(
"before_update_id must be a positive integer"
)));
}
let limit = limit.unwrap_or(DEFAULT_LIMIT).clamp(1, MAX_LIMIT);
debug!(
"Fetching vault history for vault `{vault_id}` (limit={limit}, before={before_update_id:?})"
);
// Fetch one extra row to determine if there are more results
let mut versions = state
.database
.get_vault_history(&vault_id, limit + 1, before_update_id, None)
.await
.map_err(server_error)?;
#[allow(clippy::cast_sign_loss)] // limit is clamped to [1, 500] above
let has_more = versions.len() > limit as usize;
if has_more {
versions.pop();
}
Ok(Json(VaultHistoryResponse { versions, has_more }))
}

View file

@ -1,7 +1,77 @@
use axum::response::{Html, IntoResponse};
use axum::{
body::Body,
extract::{Path, State},
http::{StatusCode, header},
response::{Html, IntoResponse, Response},
};
use log::warn;
use rust_embed::Embed;
pub async fn index() -> impl IntoResponse {
const HTML_CONTENT: &str = include_str!("./assets/index.html");
let html_content = HTML_CONTENT;
Html(html_content)
use crate::app_state::AppState;
#[derive(Embed)]
#[folder = "../frontend/history-ui/dist/"]
struct HistoryUiAssets;
pub async fn index(State(_state): State<AppState>) -> impl IntoResponse {
if let Some(content) = HistoryUiAssets::get("index.html") {
Html(
std::str::from_utf8(content.data.as_ref())
.inspect_err(|e| warn!("Embedded index.html is not valid UTF-8: {e}"))
.unwrap_or("<h1>VaultLink</h1>")
.to_owned(),
)
.into_response()
} else {
warn!("No embedded index.html found — history UI may not have been built");
Html("<h1>VaultLink server</h1>".to_owned()).into_response()
}
}
pub async fn spa_assets(Path(path): Path<String>) -> impl IntoResponse {
// The route is /assets/*path so path is relative to assets/.
// The embedded files include the assets/ prefix from the dist directory.
let full_path = format!("assets/{path}");
if let Some(content) = HistoryUiAssets::get(&full_path) {
let mime = mime_guess::from_path(&full_path).first_or_octet_stream();
return Response::builder()
.status(StatusCode::OK)
.header(header::CONTENT_TYPE, mime.as_ref())
.body(Body::from(content.data.to_vec()))
.unwrap_or_else(|_| {
Response::builder()
.status(StatusCode::INTERNAL_SERVER_ERROR)
.body(Body::empty())
.unwrap_or_else(|_| Response::new(Body::empty()))
});
}
// Asset paths must match an embedded file — no SPA fallback.
// Serving index.html here would return 200 with text/html for missing
// .css/.js files, causing the browser to silently ignore the content.
Response::builder()
.status(StatusCode::NOT_FOUND)
.body(Body::from("Not found"))
.unwrap_or_else(|_| Response::new(Body::from("Not found")))
}
/// SPA fallback for production: serves index.html for client-side routes
/// (e.g. `/documents/123`).
pub async fn spa_fallback() -> impl IntoResponse {
match HistoryUiAssets::get("index.html") {
Some(content) => Response::builder()
.status(StatusCode::OK)
.header(header::CONTENT_TYPE, "text/html")
.body(Body::from(content.data.to_vec()))
.unwrap_or_else(|_| {
Response::builder()
.status(StatusCode::INTERNAL_SERVER_ERROR)
.body(Body::empty())
.unwrap_or_else(|_| Response::new(Body::empty()))
}),
None => Response::builder()
.status(StatusCode::NOT_FOUND)
.body(Body::from("Not found"))
.unwrap_or_else(|_| Response::new(Body::from("Not found"))),
}
}

View file

@ -0,0 +1,147 @@
use anyhow::anyhow;
use axum::{
Extension, Json,
extract::{Path, State},
};
use axum_extra::TypedHeader;
use log::{debug, info};
use serde::Deserialize;
use super::device_id_header::DeviceIdHeader;
use crate::{
app_state::{
AppState,
database::models::{
DocumentId, DocumentVersionWithoutContent, StoredDocumentVersion, VaultId,
VaultUpdateId,
},
},
config::user_config::User,
errors::{SyncServerError, client_error, not_found_error, server_error, write_transaction_error},
utils::{find_first_available_path::find_first_available_path, normalize::normalize},
};
#[derive(Deserialize)]
pub struct RestorePathParams {
#[serde(deserialize_with = "normalize")]
vault_id: VaultId,
document_id: DocumentId,
}
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct RestoreDocumentVersionRequest {
pub vault_update_id: VaultUpdateId,
}
#[axum::debug_handler]
pub async fn restore_document_version(
Path(RestorePathParams {
vault_id,
document_id,
}): Path<RestorePathParams>,
Extension(user): Extension<User>,
TypedHeader(device_id): TypedHeader<DeviceIdHeader>,
State(state): State<AppState>,
Json(request): Json<RestoreDocumentVersionRequest>,
) -> Result<Json<DocumentVersionWithoutContent>, SyncServerError> {
debug!(
"Restoring document `{document_id}` in vault `{vault_id}` to version `{}`",
request.vault_update_id
);
if request.vault_update_id <= 0 {
return Err(client_error(anyhow!(
"Invalid vault_update_id: `{}`",
request.vault_update_id
)));
}
let mut transaction = state
.database
.create_write_transaction(&vault_id)
.await
.map_err(write_transaction_error)?;
let target_version = state
.database
.get_document_version(&vault_id, request.vault_update_id, Some(&mut *transaction))
.await
.map_err(server_error)?
.ok_or_else(|| {
not_found_error(anyhow!("Version `{}` not found", request.vault_update_id))
})?;
if target_version.document_id != document_id {
transaction.rollback().await.map_err(server_error)?;
return Err(not_found_error(anyhow!(
"Version `{}` does not belong to document `{document_id}`",
request.vault_update_id,
)));
}
if target_version.is_deleted {
transaction.rollback().await.map_err(server_error)?;
return Err(client_error(anyhow!(
"Cannot restore to a deleted version `{}`",
request.vault_update_id,
)));
}
let existing = state
.database
.get_latest_non_deleted_document_by_path(
&vault_id,
&target_version.relative_path,
Some(&mut *transaction),
)
.await
.map_err(server_error)?;
let restore_path = if let Some(existing_doc) = &existing
&& existing_doc.document_id != document_id
{
find_first_available_path(
&vault_id,
&target_version.relative_path,
&state.database,
&mut transaction,
)
.await
.map_err(server_error)?
} else {
target_version.relative_path.clone()
};
let last_update_id = state
.database
.get_max_update_id_in_vault(&vault_id, Some(&mut *transaction))
.await
.map_err(server_error)?;
let new_version = StoredDocumentVersion {
vault_update_id: last_update_id + 1,
document_id,
relative_path: restore_path,
content: target_version.content,
updated_date: chrono::Utc::now(),
is_deleted: false,
user_id: user.name.clone(),
device_id: device_id.0.clone(),
has_been_merged: false,
};
state
.database
.insert_document_version(&vault_id, &new_version, Some(transaction))
.await
.map_err(server_error)?;
info!(
"Restored document `{document_id}` to version `{}` as new version `{}`",
request.vault_update_id, new_version.vault_update_id
);
Ok(Json(new_version.into()))
}