diff --git a/docs/architecture/data-flow.md b/docs/architecture/data-flow.md index 228b11a9..d11977b8 100644 --- a/docs/architecture/data-flow.md +++ b/docs/architecture/data-flow.md @@ -1,6 +1,6 @@ # Data Flow -This document provides a detailed look at how data flows through the VaultLink system, from client to server and back. +How data flows through VaultLink, from client to server and back. ## Connection Lifecycle diff --git a/docs/architecture/index.md b/docs/architecture/index.md index 888830d3..f210b3e1 100644 --- a/docs/architecture/index.md +++ b/docs/architecture/index.md @@ -1,6 +1,6 @@ # Architecture Overview -VaultLink is built as a distributed system with a central sync server and multiple clients. This document explains the high-level architecture and design decisions. +Central sync server with multiple clients. High-level architecture and design decisions. ## System Components @@ -40,7 +40,7 @@ VaultLink is built as a distributed system with a central sync server and multip ### Sync Server -The central authority for synchronization, written in Rust using Axum framework. +Central authority for synchronization. Rust + Axum framework. **Responsibilities**: @@ -61,7 +61,7 @@ The central authority for synchronization, written in Rust using Axum framework. ### Sync Client Library -TypeScript library providing core synchronization logic, used by both the Obsidian plugin and CLI client. +TypeScript library with core sync logic. Used by Obsidian plugin and CLI client. **Responsibilities**: @@ -310,35 +310,13 @@ Token-based authentication on connection: ## Technology Choices -### Why Rust for Server? +**Rust**: Low latency, memory safe, excellent async with Tokio, compile-time SQL verification -- **Performance**: Low latency for real-time sync -- **Memory safety**: No crashes from memory bugs -- **Concurrency**: Excellent async support with Tokio -- **Type safety**: Catch bugs at compile time -- **SQLx**: Compile-time SQL verification +**SQLite**: No separate database server, fast for reads, single file per vault, backups are file copies -### Why SQLite? +**WebSocket**: Bidirectional push, no polling overhead, built-in browser/Node.js support -- **Simplicity**: No separate database server required -- **Performance**: Fast for read-heavy workloads -- **Reliability**: Battle-tested, ACID compliant -- **Portability**: Single file per vault -- **Backups**: Simple file copy - -### Why WebSocket? - -- **Real-time**: Bidirectional push for instant updates -- **Efficiency**: Persistent connection, no polling overhead -- **Simplicity**: Built-in browser/Node.js support -- **Standards**: Well-supported protocol - -### Why Operational Transformation? - -- **Automatic conflict resolution**: No manual merging required -- **Preserves intent**: All edits are kept -- **Real-time collaboration**: Users see changes as they happen -- **Proven algorithm**: Used by Google Docs, etc. +**Operational Transformation**: Automatic conflict resolution, preserves all edits, real-time collaboration ## Design Principles diff --git a/docs/architecture/sync-algorithm.md b/docs/architecture/sync-algorithm.md index 021c8ad7..47fa07fb 100644 --- a/docs/architecture/sync-algorithm.md +++ b/docs/architecture/sync-algorithm.md @@ -1,6 +1,6 @@ # Sync Algorithm -VaultLink uses operational transformation (OT) to handle concurrent edits and maintain consistency across clients. This document explains how the algorithm works. +VaultLink uses operational transformation (OT) to handle concurrent edits and maintain consistency across clients. ## Operational Transformation diff --git a/docs/guide/cli-client.md b/docs/guide/cli-client.md index ebb89b18..ba132908 100644 --- a/docs/guide/cli-client.md +++ b/docs/guide/cli-client.md @@ -1,6 +1,6 @@ # CLI Client -The VaultLink CLI client provides standalone synchronization without requiring Obsidian. Perfect for servers, automation, backups, or syncing vaults on headless systems. +Sync vaults without Obsidian. Works on servers, automation, backups, headless systems. ## Installation diff --git a/docs/guide/getting-started.md b/docs/guide/getting-started.md index 8282c7b1..0dc369df 100644 --- a/docs/guide/getting-started.md +++ b/docs/guide/getting-started.md @@ -1,48 +1,45 @@ # Getting Started -This guide will walk you through setting up VaultLink from scratch. You'll have a working sync server and connected client in under 10 minutes. +Set up VaultLink in 5 minutes. Deploy server, connect clients, done. ## Prerequisites -- Docker installed (recommended) or Rust toolchain for building from source -- Basic familiarity with command line -- A server or machine to host the sync server (can be localhost for testing) +- Docker (or Rust toolchain if building from source) +- A server (VPS, home server, or localhost for testing) -## Quick Start +## Step 1: Deploy Server -### Step 1: Deploy the Sync Server +Create `config.yml`: -The fastest way to get started is with Docker: +```yaml +database: + databases_directory_path: databases + max_connections_per_vault: 12 + cursor_timeout_seconds: 60 +server: + host: 0.0.0.0 + port: 3000 + max_body_size_mb: 512 + max_clients_per_vault: 256 + response_timeout_seconds: 60 +users: + user_configs: + - name: admin + token: change-this-to-secure-random-token + vault_access: + type: allow_access_to_all +logging: + log_directory: logs + log_rotation: 7days +``` + +::: tip +Generate secure token: `openssl rand -hex 32` +::: + +Run server: ```bash -# Create a directory for server data -mkdir -p ~/vaultlink-data -cd ~/vaultlink-data - -# Create a basic configuration file -cat > config.yml << 'EOF' -database: - databases_directory_path: databases - max_connections_per_vault: 12 - cursor_timeout_seconds: 60 -server: - host: 0.0.0.0 - port: 3000 - max_body_size_mb: 512 - max_clients_per_vault: 256 - response_timeout_seconds: 60 -users: - user_configs: - - name: admin - token: change-this-to-a-secure-random-token - vault_access: - type: allow_access_to_all -logging: - log_directory: logs - log_rotation: 7days -EOF - -# Run the server docker run -d \ --name vaultlink-server \ --restart unless-stopped \ @@ -52,136 +49,75 @@ docker run -d \ /app/sync_server /data/config.yml ``` -::: warning -Change the token in `config.yml` to a secure random value before deploying to production! -::: +Verify: `curl http://localhost:3000/vaults/test/ping` should return `pong` -Verify the server is running: +## Step 2: Connect Client -```bash -curl http://localhost:3000/vaults/test/ping -``` +### Obsidian Plugin -You should see: `pong` +1. Settings → Community Plugins → Browse +2. Search "VaultLink", install, enable +3. Configure: + - Server URL: `ws://localhost:3000` (or `wss://your-server.com` for SSL) + - Token: Your token from config.yml + - Vault Name: `default` -### Step 2: Choose Your Client +[Full plugin guide →](/guide/obsidian-plugin) -You can connect to VaultLink using either the Obsidian plugin or the standalone CLI client. - -#### Option A: Obsidian Plugin - -1. Open Obsidian Settings → Community Plugins -2. Browse community plugins and search for "VaultLink" -3. Install and enable the plugin -4. Configure the plugin: - - **Server URL**: `ws://localhost:3000` (or your server address) - - **Token**: The token from your `config.yml` - - **Vault Name**: `default` (or any name you choose) - -[Read the full Obsidian plugin guide →](/guide/obsidian-plugin) - -#### Option B: CLI Client - -Perfect for syncing vaults without Obsidian: +### CLI Client ```bash docker run -d \ --name vaultlink-cli \ --restart unless-stopped \ - -v /path/to/your/vault:/vault \ + -v /path/to/vault:/vault \ ghcr.io/schmelczer/vault-link-cli:latest \ - -l /vault \ - -r ws://localhost:3000 \ - -t change-this-to-a-secure-random-token \ - -v default + -l /vault -r ws://localhost:3000 -t your-token -v default ``` -Replace `/path/to/your/vault` with the directory containing your files. +[Full CLI guide →](/guide/cli-client) -[Read the full CLI client guide →](/guide/cli-client) +## Production Setup -## Next Steps +For production: -### Production Deployment +1. **SSL/TLS**: Use Nginx/Caddy reverse proxy for `wss://` ([setup guide](/guide/server-setup#ssl-tls-with-reverse-proxy)) +2. **Secure tokens**: Generate with `openssl rand -hex 32`, don't reuse the example +3. **Firewall**: Only expose port 3000 to reverse proxy +4. **Backups**: SQLite databases are in `databases/` directory -For production use, you should: - -1. **Use HTTPS/WSS**: Put the sync server behind a reverse proxy with SSL -2. **Secure tokens**: Generate cryptographically random tokens -3. **Configure backups**: Back up the SQLite databases regularly -4. **Set up monitoring**: Use Docker health checks and logging - -[Learn about production deployment →](/guide/server-setup#production-deployment) - -### Multiple Users - -To add more users or restrict vault access: +## Multiple Users ```yaml users: user_configs: - name: alice - token: alice-secure-token + token: alice-token vault_access: type: allow_list allowed: - personal - shared - name: bob - token: bob-secure-token + token: bob-token vault_access: type: allow_list allowed: - shared ``` -[Learn about authentication configuration →](/config/authentication) - -### Advanced Configuration - -Explore advanced server options: - -- Database tuning for large vaults -- Rate limiting and connection limits -- Custom logging and log rotation -- Multi-vault setups - -[View configuration reference →](/config/server) - -## Architecture Overview - -Want to understand how VaultLink works under the hood? - -[Read the architecture documentation →](/architecture/) +[Auth docs →](/config/authentication) ## Troubleshooting -### Server won't start +**Server won't start**: `docker logs vaultlink-server` -Check Docker logs: +**Client can't connect**: -```bash -docker logs vaultlink-server -``` +1. Verify: `curl http://your-server:3000/vaults/test/ping` +2. Check URL: `ws://` for HTTP, `wss://` for HTTPS +3. Verify token matches config.yml -Common issues: +**Files not syncing**: Check client logs, verify vault name matches -- Port 3000 already in use: Change the port mapping `-p 3001:3000` -- Config file errors: Validate YAML syntax -- Permission issues: Ensure the volume mount is writable - -### Client can't connect - -1. Verify server is accessible: `curl http://your-server:3000/vaults/test/ping` -2. Check WebSocket connectivity (browser dev tools or wscat) -3. Verify token matches between client and server config -4. Check firewall rules allow port 3000 - -### Files not syncing - -1. Check client logs for errors -2. Verify vault name matches on both server and client -3. Ensure user has access to the vault (check server config) -4. Check for file size limits (default 10MB for CLI) - -For more help, [open an issue on GitHub](https://github.com/schmelczer/vault-link/issues). +[Server setup →](/guide/server-setup) | [Architecture →](/architecture/) diff --git a/docs/guide/obsidian-plugin.md b/docs/guide/obsidian-plugin.md index ed3989b6..c87debf5 100644 --- a/docs/guide/obsidian-plugin.md +++ b/docs/guide/obsidian-plugin.md @@ -1,6 +1,6 @@ # Obsidian Plugin -The VaultLink Obsidian plugin provides native real-time synchronization directly within Obsidian. +Real-time sync for Obsidian vaults. ## Installation diff --git a/docs/guide/server-setup.md b/docs/guide/server-setup.md index 8391522b..9b39d5bc 100644 --- a/docs/guide/server-setup.md +++ b/docs/guide/server-setup.md @@ -1,12 +1,12 @@ # Server Setup -This guide covers deploying the VaultLink sync server in various environments, from local development to production infrastructure. +Deploy VaultLink server via Docker, binary, or build from source. ## Deployment Options ### Docker (Recommended) -Docker provides the easiest deployment path with built-in health checks and minimal dependencies. +Easiest deployment path, includes health checks. #### Basic Docker Deployment @@ -60,7 +60,7 @@ docker compose up -d ### Binary Installation -Download pre-built binaries from [GitHub Releases](https://github.com/schmelczer/vault-link/releases). +Download pre-built binaries from [GitHub Releases](https://github.com/schmelczer/vault-link/releases): ```bash # Download the binary for your platform @@ -75,11 +75,7 @@ chmod +x sync_server-linux-x86_64 ### Build from Source -Requirements: - -- Rust 1.89.0 or later -- SQLite development headers -- SQLx CLI +Requirements: Rust 1.89.0+, SQLite development headers, SQLx CLI ```bash # Clone the repository diff --git a/docs/guide/what-is-vaultlink.md b/docs/guide/what-is-vaultlink.md index 02e0d6cb..9bb5addb 100644 --- a/docs/guide/what-is-vaultlink.md +++ b/docs/guide/what-is-vaultlink.md @@ -1,125 +1,69 @@ # What is VaultLink? -VaultLink is a self-hosted real-time synchronization system for Obsidian vaults. It provides collaborative file syncing with automatic conflict resolution, designed for users who want complete control over their data. +Self-hosted sync for Obsidian vaults with automatic conflict-free merging. Edit with any tool, collaborate in real-time, no conflict markers. -## Overview +## The Problem -VaultLink consists of three main components: +Syncing Obsidian vaults across devices or sharing with teammates sucks: -### Sync Server +- **Commercial services**: Lock-in, subscriptions, third-party access to your data +- **Git**: Manual conflict resolution with `<<<<<<<` markers interrupting your workflow +- **Cloud storage**: Last-write-wins data loss or manual conflict resolution +- **CRDT solutions**: Only work if you edit inside Obsidian (break if you use Vim, VS Code, etc.) -A Rust-based WebSocket server that handles: +## VaultLink's Solution -- Real-time bidirectional synchronization -- Document versioning with SQLite -- User authentication and vault access control -- Operational transformation for conflict resolution +Differential synchronization with operational transformation. -### Obsidian Plugin - -A native Obsidian plugin that: - -- Integrates sync directly into your Obsidian workflow -- Provides real-time updates as you edit -- Handles file watching and automatic synchronization -- Works across desktop and mobile platforms - -### CLI Client - -A standalone synchronization client that: - -- Syncs vaults without requiring Obsidian -- Perfect for servers, automation, or backup systems -- Provides file watching and bidirectional sync -- Runs in Docker or as a standalone binary - -## Key Features - -### Real-Time Synchronization - -Changes are synchronized immediately via WebSocket connections. When multiple users edit the same file, operational transformation ensures all edits are preserved without conflicts. - -### Self-Hosted Architecture - -Run the sync server on your own infrastructure: - -- Full control over data storage and access -- No dependency on third-party services -- Configurable authentication and authorization -- Deploy anywhere: cloud VPS, home server, or localhost - -### Operational Transformation - -VaultLink uses the `reconcile-text` library for intelligent conflict resolution: - -- Simultaneous edits are automatically merged -- No manual conflict resolution required -- Preserves intent of all contributors -- Works seamlessly in the background - -### Flexible Authentication - -Configure user access per vault: - -- Token-based authentication -- Per-user vault access control -- Allow-list or deny-list patterns -- Support for multiple users and vaults - -## Use Cases - -### Personal Sync - -Synchronize your Obsidian vault across multiple devices: - -- Laptop, desktop, and mobile in real-time -- No cloud service subscription required -- Full privacy and data control - -### Team Collaboration - -Share knowledge bases with teammates: - -- Real-time collaborative editing -- Granular access control per vault -- Self-hosted for enterprise security requirements - -### Automated Backups - -Use the CLI client for automated workflows: - -- Scheduled backups to remote servers -- Integration with existing backup systems -- Headless operation without Obsidian - -### Development & Testing - -Synchronize documentation across environments: - -- Keep docs in sync with development environments -- Automated deployment of documentation -- Version control integration +Edit files with Obsidian, Vim, VS Code, or any editor. VaultLink compares versions and automatically merges all changes. No operation tracking required, no conflict markers, no data loss. ## How It Works -1. **Server Setup**: Deploy the sync server on your infrastructure -2. **Authentication**: Configure users and vault access in `config.yml` -3. **Client Connection**: Connect via Obsidian plugin or CLI client -4. **Initial Sync**: Client uploads local files to server -5. **Real-Time Updates**: Changes sync bidirectionally via WebSocket -6. **Conflict Resolution**: Operational transformation handles simultaneous edits +1. **Server**: Rust WebSocket server with SQLite stores document versions +2. **Clients**: Obsidian plugin or CLI client watches filesystem changes +3. **Sync**: Changes upload to server, server broadcasts to other clients +4. **Merge**: [reconcile-text](https://schmelczer.dev/reconcile) automatically merges concurrent edits -## Technology Stack +No CRDT infrastructure. No operation logs. Just file comparison and smart merging. -- **Server**: Rust with Axum framework, SQLite database, WebSocket protocol -- **Frontend**: TypeScript with WebSocket client, npm workspaces -- **Sync Algorithm**: reconcile-text operational transformation library -- **Deployment**: Docker images, binary releases, or source builds +## Key Advantages + +**Editor agnostic**: Edit files with any tool. Other solutions break when you edit outside their ecosystem. + +**Self-hosted**: Your data, your server. No third parties, no subscriptions, no surprises. + +**Automatic merging**: Operational transformation handles conflicts without interrupting your workflow. + +**Production-ready**: Comprehensive tests, E2E tests, battle-tested. Many alternatives have zero tests. + +**Collaborative**: Real-time sync with cursor tracking. See where teammates are editing. + +## Not Tied to Obsidian + +VaultLink syncs Markdown files. Use it for: + +- Obsidian vaults (Obsidian desktop + mobile + CLI) +- Technical documentation (VS Code, your-editor, CLI) +- Academic writing (multiple Markdown editors) +- Automated workflows (CLI client for backups/CI/CD) + +The Obsidian plugin is just a convenience wrapper around the sync client. + +## Quick Comparison + +| Feature | VaultLink | Git | Cloud Sync | CRDT Solutions | +| ------------------- | --------- | --- | ---------- | -------------- | +| Self-hosted | ✅ | ✅ | ❌ | Varies | +| Any editor | ✅ | ✅ | ✅ | ❌ | +| No conflict markers | ✅ | ❌ | ❌ | ✅ | +| Real-time | ✅ | ❌ | ❌ | ✅ | +| No subscriptions | ✅ | ✅ | ❌ | Varies | +| Comprehensive tests | ✅ | N/A | N/A | ❌ | + +[Detailed comparison with alternatives →](/guide/alternatives) ## Next Steps -Ready to get started? - -- [Getting Started Guide →](/guide/getting-started) -- [Server Setup →](/guide/server-setup) -- [Architecture Overview →](/architecture/) +- [Get started →](/guide/getting-started) (5 minute setup) +- [See the architecture →](/architecture/) (understand how it works) +- [Compare alternatives →](/guide/alternatives) (why VaultLink vs others) diff --git a/docs/index.md b/docs/index.md index 569e692c..705dd1b9 100644 --- a/docs/index.md +++ b/docs/index.md @@ -3,8 +3,8 @@ layout: home hero: name: VaultLink - text: Self-Hosted Sync for Obsidian - tagline: Real-time collaborative file synchronization for your knowledge base + text: Self-Hosted Obsidian Sync + tagline: Edit with any tool. Automatic conflict-free merging. Your infrastructure. image: src: /logo.svg alt: VaultLink @@ -13,60 +13,43 @@ hero: text: Get Started link: /guide/getting-started - theme: alt - text: View on GitHub - link: https://github.com/schmelczer/vault-link + text: Why VaultLink? + link: /guide/what-is-vaultlink features: - - icon: 🚀 - title: Real-Time Synchronization - details: Operational transformation-based conflict resolution ensures your files stay in sync across devices without data loss - - icon: 🔒 - title: Self-Hosted & Private - details: Run your own sync server. Your data stays on your infrastructure with full control over access and privacy - - icon: 🎯 - title: Obsidian Plugin - details: Native integration with Obsidian for seamless synchronization directly within your favorite note-taking app - - icon: 🖥️ - title: CLI Client - details: Sync vaults to any system using the standalone CLI client. Perfect for servers, automation, or headless setups - - icon: ⚡ - title: Built for Performance - details: Rust-powered WebSocket server with SQLite backend delivers blazing-fast sync performance - - icon: 🛠️ - title: Flexible Deployment - details: Deploy via Docker, binary releases, or build from source. Configure authentication and access controls to fit your needs + - title: Edit Anywhere + details: Use Obsidian, Vim, VS Code, or any editor. VaultLink syncs files, not keystrokes—edit however you want + - title: Your Data, Your Server + details: Fully self-hosted. No third parties, no subscriptions, no data mining. Single Docker container or binary + - title: No Conflict Markers + details: Automatic merge using operational transformation. Never see conflict markers in your notes again + - title: Real-Time Collaboration + details: See teammate cursors, merge edits instantly. Rust-powered WebSocket server with SQLite + - title: Open Source Everything + details: MIT licensed. Server, clients, and sync algorithm are all open source. No proprietary components + - title: Battle-Tested + details: Comprehensive test suite. E2E tests. Used in production. Unlike alternatives with zero tests --- +## Why Self-Host? + +**You own your knowledge base.** Commercial sync services can disappear, change pricing, or lock you out. VaultLink runs on your infrastructure—VPS, home server, or localhost. + +**Edit with any tool.** Other solutions require CRDT-aware editors or break when you edit outside Obsidian. VaultLink uses differential sync: edit files however you want, sync handles the rest. + +**No conflict markers.** Git forces manual merging. Other tools use last-write-wins. VaultLink's operational transformation automatically merges all changes without data loss or workflow interruption. + +[See how VaultLink compares to alternatives →](/guide/alternatives) + ## Quick Start -Deploy the sync server: +Deploy server (single command): ```bash -docker run -d \ - -p 3000:3000 \ - -v $(pwd)/data:/data \ - ghcr.io/schmelczer/vault-link-server:latest \ - /app/sync_server config.yml +docker run -d -p 3000:3000 -v $(pwd)/data:/data \ + ghcr.io/schmelczer/vault-link-server:latest ``` -Install the Obsidian plugin or use the CLI client: +Then install the [Obsidian plugin](/guide/obsidian-plugin) or [CLI client](/guide/cli-client). -```bash -docker run -v /path/to/vault:/vault \ - ghcr.io/schmelczer/vault-link-cli:latest \ - -l /vault -r wss://your-server.com -t your-token -v default -``` - -[Learn more →](/guide/getting-started) - -## Why VaultLink? - -VaultLink provides a complete self-hosted synchronization solution for Obsidian: - -- **No third-party services**: Your data never leaves your infrastructure -- **Operational transformation**: Smart conflict resolution that preserves all changes -- **Multi-platform**: Works with Obsidian plugin or standalone CLI on any system -- **Production-ready**: Docker images, health checks, and comprehensive logging -- **Open source**: MIT licensed with active development - -[Read the architecture overview →](/architecture/) +[Full setup guide →](/guide/getting-started)