vault-link/docs/architecture/index.md

Architecture Overview

Central sync server with multiple clients. High-level architecture and design decisions.

System Components

┌─────────────────────────────────────────────────────────────┐
│                        Clients                              │
├─────────────────────┬───────────────────┬───────────────────┤
│  Obsidian Plugin    │  Obsidian Plugin  │   CLI Client      │
│  (User A - Device1) │  (User A - Device2│   (Server/Backup) │
└──────────┬──────────┴─────────┬─────────┴──────────┬────────┘
          │                    │                    │
          │    WebSocket       │   WebSocket        │   WebSocket
          │                    │                    │
          └────────────────────┼────────────────────┘
                                │
                    ┌───────────▼───────────┐
                    │   Sync Server         │
                    │   (Rust + Axum)       │
                    │                       │
                    │  ┌─────────────────┐  │
                    │  │  WebSocket Hub  │  │
                    │  └────────┬────────┘  │
                    │           │           │
                    │  ┌────────▼────────┐  │
                    │  │ Sync Engine     │  │
                    │  │ (OT Algorithm)  │  │
                    │  └────────┬────────┘  │
                    │           │           │
                    │  ┌────────▼────────┐  │
                    │  │ SQLite Database │  │
                    │  │ (Per Vault)     │  │
                    │  └─────────────────┘  │
                    └───────────────────────┘

Core Components

Sync Server

Central authority for synchronisation. Rust + Axum framework.

Responsibilities:

• Accept WebSocket connections from clients
• Authenticate users via token-based auth
• Store document versions in SQLite
• Coordinate real-time updates between clients
• Apply operational transformation for conflict resolution
• Manage vault access control

Technology:

• Language: Rust 1.92+
• Framework: Axum (async web framework)
• Database: SQLite with SQLx
• Protocol: WebSockets for real-time communication
• Sync Algorithm: reconcile-text (operational transformation)

Sync Client Library

TypeScript library with core sync logic. Used by Obsidian plugin and CLI client.

Responsibilities:

• Manage WebSocket connection to server
• Watch local filesystem for changes
• Upload and download files
• Apply remote changes locally
• Handle conflict resolution
• Maintain sync metadata

Technology:

• Language: TypeScript
• Build: Webpack
• Protocol: WebSocket client
• File System: Node.js fs API / Obsidian API

Obsidian Plugin

Integration layer between sync client and Obsidian.

Responsibilities:

• Provide UI for configuration
• Bridge sync client with Obsidian's file system API
• Handle Obsidian lifecycle events
• Display sync status to users

Technology:

• Platform: Obsidian Plugin API
• Core: sync-client library
• UI: Obsidian settings UI

CLI Client

Standalone executable for syncing vaults without Obsidian.

Responsibilities:

• Command-line interface
• File system access via Node.js
• Daemon mode for continuous sync
• Health check endpoint for monitoring

Technology:

• Language: TypeScript
• Runtime: Node.js
• CLI: Commander.js
• Core: sync-client library

Data Flow

Initial Connection

1. Client connects via WebSocket to server
2. Server authenticates using provided token
3. Server verifies user has access to requested vault
4. Connection established, sync begins

File Upload Flow

Client                          Server
  │                               │
  │  1. File changed locally      │
  │                               │
  │  2. Read file content         │
  │                               │
  │  3. WebSocket: Upload file    │
  ├──────────────────────────────►│
  │                               │ 4. Store in SQLite
  │                               │
  │                               │ 5. Broadcast to other clients
  │                               ├───────────────────────►
  │  6. Ack upload                │
  │◄──────────────────────────────┤

File Download Flow

Client A                  Server                  Client B
  │                         │                         │
  │                         │  1. File uploaded       │
  │                         │◄────────────────────────┤
  │                         │                         │
  │                         │ 2. Store in DB          │
  │                         │                         │
  │  3. Push notification   │                         │
  │◄────────────────────────┤                         │
  │                         │                         │
  │  4. Download file       │                         │
  ├────────────────────────►│                         │
  │                         │                         │
  │  5. Write locally       │                         │
  │                         │                         │

Conflict Resolution

When two clients edit the same file simultaneously:

Client A                  Server                  Client B
  │                         │                         │
  │ 1. Edit file            │                         │ 1. Edit same file
  │                         │                         │
  │ 2. Upload changes       │                         │ 2. Upload changes
  ├────────────────────────►│◄────────────────────────┤
  │                         │                         │
  │                         │ 3. Apply OT algorithm   │
  │                         │    - Merge both edits   │
  │                         │    - Preserve all changes│
  │                         │                         │
  │ 4. Receive merged ver.  │ 5. Receive merged ver.  │
  │◄────────────────────────┤────────────────────────►│
  │                         │                         │
  │ 6. Apply locally        │                         │ 6. Apply locally

Storage Architecture

Server Storage

Each vault has its own SQLite database:

databases/
├── vault-1.db
├── vault-2.db
└── shared-team.db

Database Schema (simplified):

• documents: File metadata (path, size, modified time)
• versions: Document content with version history
• cursors: Client sync state

Client Storage

Clients maintain sync metadata:

.vaultlink/
├── metadata.json       # Sync state
└── cache/              # Optional local cache

The .vaultlink directory tracks which files have been synced and their versions to enable efficient synchronisation.

Communication Protocol

WebSocket Messages

Client-server communication uses JSON messages over WebSocket.

Message Types:

• upload_file: Client → Server (file upload)
• download_file: Client → Server (request file)
• file_updated: Server → Client (file changed notification)
• file_deleted: Server → Client (file deleted notification)
• sync_complete: Server → Client (initial sync finished)

Authentication

Token-based authentication on connection:

// Client sends token on connect
{
  type: "auth",
  token: "user-auth-token",
  vault: "vault-name"
}

// Server responds
{
  type: "auth_success"
}
// or
{
  type: "auth_error",
  message: "Invalid token"
}

Scalability Considerations

Current Architecture

• SQLite per vault: Simple, performant, limited to single server
• WebSocket connections: Stateful, requires sticky sessions for load balancing
• Operational transformation: Centralized on server

Scaling Approaches

Vertical Scaling:

• Increase server resources (CPU, RAM, storage)
• Optimize database queries and indexing
• Tune connection limits

Horizontal Scaling (future):

• Separate vault servers (vault sharding)
• Load balancer with sticky sessions
• Shared storage layer for SQLite databases
• Consider alternative databases (PostgreSQL) for multi-server setups

Performance Characteristics

• Small vaults (< 1000 files): Excellent performance
• Medium vaults (1000-10000 files): Good performance with tuning
• Large vaults (> 10000 files): May require optimisation
• Concurrent users: Tested with dozens of simultaneous clients per vault

Security Model

Authentication

• Token-based authentication
• Tokens configured in server config.yml
• No password hashing (tokens are secrets)

Authorization

• Per-user vault access control
• Allow-list or deny-list patterns
• Global access or vault-specific access

Network Security

• WebSocket over TLS (WSS) for encrypted transport
• No built-in SSL (use reverse proxy)
• CORS configured for web clients

Data Security

• No encryption at rest (use encrypted filesystems if needed)
• No end-to-end encryption (server sees all content)
• Self-hosted model: you control the data

Technology Choices

Rust: Low latency, memory safe, excellent async with Tokio, compile-time SQL verification

SQLite: No separate database server, fast for reads, single file per vault, backups are file copies

WebSocket: Bidirectional push, no polling overhead, built-in browser/Node.js support

Operational Transformation: Automatic conflict resolution, preserves all edits, real-time collaboration

Design Principles

1. Self-hosted first: Users control their data and infrastructure
2. Simplicity: Easy to deploy and operate
3. Real-time: Changes appear immediately
4. Reliability: Handle network failures gracefully
5. Performance: Fast sync for typical vault sizes
6. Privacy: No third-party services or telemetry

Next Steps

• Learn about the sync algorithm →
• Understand data flow in detail →
• Deploy the server →