andras/vault-link
1
0
Fork 0
Code Issues Pull requests 18 Projects Releases 55 Packages Wiki Activity Actions 1

wip

Browse source
This commit is contained in:
András Schmelczer 2025-12-23 23:14:12 +01:00
parent 9a75569e83
commit 2b568fd20e
7 changed files with 173 additions and 328 deletions
Download patch file Download diff file Expand all files Collapse all files

5
docs/README.md
View file

@ -4,11 +4,6 @@ This directory contains the VaultLink documentation site built with [VitePress](
## Development
### Prerequisites
- Node.js 18+
- npm
### Setup
```bash

10
docs/architecture/index.md
View file

@ -9,12 +9,12 @@ Central sync server with multiple clients. High-level architecture and design de
│ Clients │
├─────────────────────┬───────────────────┬───────────────────┤
│ Obsidian Plugin │ Obsidian Plugin │ CLI Client │
│ (User A - Device1) │ (User A - Device2│ (Server/Backup) │
│ (User A - Device1) │ (User A - Device2)│ (Server/Backup) │
└──────────┬──────────┴─────────┬─────────┴──────────┬────────┘
│ │ │
│ WebSocket │ WebSocket │ WebSocket
│ │ │
└────────────────────┼────────────────────┘
│ │ │
│ WebSocket │ WebSocket │ WebSocket
│ │ │
└────────────────────┼────────────────────┘
│
┌───────────▼───────────┐
│ Sync Server │

80
docs/guide/alternatives.md
View file

@ -2,6 +2,86 @@
VaultLink is one of several solutions for synchronising Obsidian vaults. This page compares VaultLink with popular alternatives to help you choose the right tool.
## Universal sync alternatives
syncthing
## VaultLink Obsidian plugin alternatives
There are already numerious ways to sync an Obsidian vault.
- LiveSync - 300k - https://github.com/vrtmrz/obsidian-livesync
- Relay - 24k downloads https://github.com/No-Instructions/Relay
- Sync server isn't open source
- Has to maintain separate CRDT per file, no out-of-band updates
- Remotely Save 1.1M downloads https://github.com/remotely-save/remotely-save
- conflict resolution is paid pro feature
- no tests & trash code
- Remotely Sync - 38k downloads https://github.com/sboesen/remotely-sync
- Same as remotely save
- Self-hosted LiveSync - 300k downloads
- SyncFTP 5k - downloads https://github.com/alex-donnan/SyncFTP - ridiculously simple
- Syncthing Integration - 22.6k downloads https://github.com/LBF38/obsidian-syncthing-integration
- Unfinished
- Manual conflict resolution
- Git - 1.4M downloads
- https://github.com/conql/obsidian-seafile - cool file change indicators
## Obsidian alternatives
- https://anytype.io/
- https://github.com/gamosoft/NoteDiscovery?ref=selfh.st
- https://github.com/timothepoznanski/poznote?ref=selfh.st
- https://appflowy.com/ - could steal editor
- https://demo.flatnotes.io/
- https://etherpad.org/
- https://kitemaker.co/ - discontinued after sale of company
- https://b3log.org/siyuan/en/
- https://apps.apple.com/us/app/standard-notes/id1285392450
- https://affine.pro/
- Notion plugin https://developers.notion.com/reference/intro
- https://www.xda-developers.com/self-hosted-markdown-editors-that-sync-without-the-cloud/
- https://simplenote.com/
- https://github.com/fccview/rwMarkable?ref=selfh.st
- https://notesnook.com/
- https://apps.apple.com/us/app/logseq/id1601013908
- blinko
- Zen notes https://www.sheshbabu.com/zen/
- https://www.getdnote.com/ - CLI only
- https://juretriglav.si/open-source-collaborative-text-editors/
- https://logseq.com/
- https://github.com/suitenumerique/docs - French goverment project
- https://silverbullet.md/ - pretty simplistic but nice editor (can't change font though)
- https://joplinapp.org/
- https://github.com/outline/outline
- https://github.com/TriliumNext/Trilium obsidian but worse
- https://github.com/suitenumerique/docs
- https://livebook.dev/ - collaborative jupyter
- https://www.blocknotejs.org/docs/editor-basics for online editor
- https://github.com/colanode/colanode - notion like
- Blinko
- https://www.reddit.com/r/selfhosted/comments/1kswy2n/many_notes_v090_markdown_notetaking_app_designed/#lightbox online editor https://github.com/brufdev/many-notes
- https://www.sheshbabu.com/zen/ text editor and database
- https://github.com/ekzhang/rustpad
- VS Code-eque vibes, single markdown editor based on OT, self-hostable, long
- rwmarkable
Bad ones:
- https://cryptpad.org/instances/
- https://hedgedoc.org/ - only has split editor
- https://www.usememos.com/ - twitter notes
- https://alextselegidis.com/get/plainpad/ - too simple
- https://standardnotes.com/ - have to pay for folders
- https://turtlapp.com/
## Key Differentiator: Editor Agnostic
**VaultLink is not tied to Obsidian.** While it includes an Obsidian plugin for convenience, VaultLink synchronises plain text files and works with any editor:

102
docs/guide/getting-started.md
View file

@ -1,13 +1,25 @@
# Getting Started
Set up VaultLink in 5 minutes. Deploy server, connect clients, done.
Set up VaultLink in 5 minutes.
## Prerequisites
## Self-hosting the VaultLink server with docker-compose
- Docker (or Rust toolchain if building from source)
- A server (VPS, home server, or localhost for testing)
todo: add compose file
## Step 1: Deploy Server
Mount the configuration, databases, and (optionally) logs folder to your host:
- Configuration: the default path to it is. Mount the parent folder with something like:
```
- volumes:
```
and the configuration.yaml will be created on first startup. When a new config option is added to a future server version, the config.yml will be automatically updated. This also means that comments are not preserved inside the file but previously set values are.
On first startup, an admin account is generated if no configuration.yaml exists or if the existing one contains no users. Changes made to the users section take into effect without requiring a server restart. This is controlled through the `hot-reload` flag within the `users` section.
- Databases: each vault is stored in its dedicated sqlite database located in the databases folder
- Logs: Logs are located at ... and are rotated every 7 days by default.
Add an expandable examplle <summary/>
Create `config.yml`:
@ -34,7 +46,7 @@ logging:
```
::: tip
Generate secure token: `openssl rand -hex 32`
You can generate secure token using: `openssl rand -hex 32`
:::
Run server:
@ -46,25 +58,42 @@ docker run -d \
-p 3000:3000 \
-v $(pwd):/data \
ghcr.io/schmelczer/vault-link-server:latest \
/app/sync_server /data/config.yml
/app/sync_server
```
Verify: `curl http://localhost:3000/vaults/test/ping` should return server version and auth status
## Step 2: Connect Client
Verify: `curl http://localhost:3000/vaults/test/ping` should return server version and auth status and http://localhost:3000 should show a version of this documentation bundled with the server's version.
### Obsidian Plugin
::: warning
The server doesn't terminate HTTPS connections and is thus recommended to deploy behind a reverse proxy such as [NGINX] or [Caddy].
:::
::: tip
The server uses a local sqlite database to track documents within a Vault as opposed to plain text files. This is necessary so that the full provenance (history) of every document can be retrieved and it also prevents corruption of the server's state coming from outside changes to these files. In case you'd like to keep the latest version of a Vault's files on the server, you can deploy the [CLI client][#cli].
:::
In case of failures, check the logs on stdout or historical logs in the logs folder.
[Full server guide →](/guide/obsidian-plugin)
## Connecting to a VaultLink server using the VaultLink Obsidian plugin
1. Settings → Community Plugins → Browse
2. Search "VaultLink", install, enable
3. Configure:
- Server URL: `ws://localhost:3000` (or `wss://your-server.com` for SSL)
3. Then, update the sync settings within the VaultLink options:
- Server URL: `http://localhost:3000` (or `https://your-server.com` for SSL)
- Token: Your token from config.yml
- Vault Name: `default`
- Vault Name: Pick any name the token has access to, such as `default`. Users may be allowed to access a predefined list of vaults or every vault managed by the server. See [authorization](#auth) for more on this.
In case of failures, check the state summary (todo screenshot) or the logs tab available through the follow button from the settings tab (todo screenshot).
[Full plugin guide →](/guide/obsidian-plugin)
### CLI Client
## CLI Client
```bash
docker run -d \
@ -77,49 +106,4 @@ docker run -d \
[Full CLI guide →](/guide/cli-client)
## Production Setup
For production:
1. **SSL/TLS**: Use Nginx/Caddy reverse proxy for `wss://` ([setup guide](/guide/server-setup#ssl-tls-with-reverse-proxy))
2. **Secure tokens**: Generate with `openssl rand -hex 32`, don't reuse the example
3. **Firewall**: Only expose port 3000 to reverse proxy
4. **Backups**: SQLite databases are in `databases/` directory
## Multiple Users
```yaml
users:
user_configs:
- name: alice
token: alice-token
vault_access:
type: allow_list
allowed:
- personal
- shared
- name: bob
token: bob-token
vault_access:
type: allow_list
allowed:
- shared
```
[Auth docs →](/config/authentication)
## Troubleshooting
**Server won't start**: `docker logs vaultlink-server`
**Client can't connect**:
1. Verify server: `curl http://your-server:3000/vaults/test/ping`
2. Check URL: `ws://` for HTTP, `wss://` for HTTPS
3. Verify token matches config.yml
**Understanding limitations**: [See what VaultLink can and can't do →](/guide/limitations)
**Files not syncing**: Check client logs, verify vault name matches
[Server setup →](/guide/server-setup) | [Architecture →](/architecture/)

183
docs/guide/limitations.md
View file

@ -2,191 +2,28 @@
VaultLink works well for most Obsidian vaults, but has some constraints you should know about.
## No end-to-end encryption (E2EE)
VaultLink is meant to be self-hosted, so trust towards the server is expected. This is why E2EE isn't a must-have feature. Additionally, the server needs to be able to read the user's file's content in order to merge it. The [sync algorithm](./sync) doesn't support homomorphic operations and thus must operate on clear text.
## File Type Limitations
### Mergeable Files
Only **`.md`** and **`.txt`** files get automatic conflict-free merging.
By default, only **`.md`** and **`.txt`** files get automatic conflict-free merging. This can be extended thorugh the `server.mergeable_file_extensions` config to text files with arbitrary file extensions. However, changes to binary files can't be merged automatically. To be fair, they shouldn't be, as without knowing the structure of the binary, the merging algorithm would just corrupt the file.
Other file types (images, PDFs, etc.) use last-write-wins:
So in case of bianry file types (images, PDFs, etc.) use last-write-wins:
```
User A updates diagram.png → Server stores version 1
User B updates diagram.png → Server stores version 2 (overwrites A's changes)
```
**Workaround**: Avoid editing the same non-text file simultaneously.
### Binary Detection
Files are treated as binary if they:
Files are treated as binary if any of the following is true:
- Their file extension isn't incldued in the `server.mergeable_file_extensions` list (matching is case-insensitive)
- Contain NUL bytes (`0x00`)
- Fail UTF-8 validation
Binary files within `.md` or `.txt` extensions still get last-write-wins (no merge).
## Performance Constraints
### Server Limits (Configurable)
| Resource | Default | Maximum Tested |
| ------------------------ | ------- | -------------- |
| Clients per vault | 256 | ~256 |
| Database connections | 12 | 20 |
| Max file size | 512 MB | 4096 MB |
| Request timeout | 60s | 180s |
| WebSocket cursor timeout | 60s | 300s |
| Database busy timeout | 3600s | - |
### Vault Size
- **Small vaults** (< 1000 files): Excellent performance
- **Medium vaults** (1000-10000 files): Good performance
- **Large vaults** (> 10000 files): Works, but initial sync slower
No hard file count limit—constrained by disk space and sync time.
### Resource Usage
Rough estimates (varies by vault size and activity):
- **RAM**: ~50-200 MB base + ~1-5 MB per active client
- **CPU**: Low (< 5%) for typical usage, spikes during merges
- **Disk**: Vault size + version history (grows over time)
## Version History
### Storage
- All versions stored indefinitely (no automatic cleanup)
- Each vault is a separate SQLite database
- Deleted files marked as deleted (not purged)
**Growth**: Version history grows with every change. A 10 MB vault with frequent edits might grow to 100+ MB over months.
**Cleanup**: Manual only (see [Advanced Configuration](/config/advanced#version-history-cleanup)).
### Implications
- Disk usage grows over time
- Database size affects backup time
- No built-in retention policy
## Merge Quality
### Text Merging
VaultLink uses word-level tokenisation for merging:
```markdown
Parent: "The quick brown fox"
User A: "The quick red fox"
User B: "The very quick brown fox"
Result: "The very quick red fox" ← Both changes preserved
```
**Imperfect scenarios**:
- Complex nested Markdown (tables, code blocks)
- Simultaneous edits to the same sentence
- Large structural changes (moving sections around)
**Result**: Merged file might need manual cleanup in ~1-5% of concurrent edits.
## Scalability
### SQLite Limitations
- One SQLite database per vault
- Single-server architecture (no built-in clustering)
- Write serialisation through database
**For high concurrency**: Consider multiple vaults instead of one massive shared vault.
### Horizontal Scaling
Not currently supported. Running multiple servers requires manual vault partitioning.
## Network Requirements
### Latency
- Real-time sync typically < 500ms on good connections
- Mobile/slow networks: 1-5s latency possible
- Timeout failures on very slow connections (> 60s)
### Offline Behaviour
- Clients queue changes locally
- On reconnect, sync all changes since last connection
- Conflicts resolved automatically (for mergeable files)
**Limitation**: No offline conflict preview—merged result appears after reconnect.
## Security
### No End-to-End Encryption
- Server sees all file contents
- Transport encryption only (WSS/TLS)
- Trust your server
**Workaround**: Self-host on infrastructure you control.
### Authentication
- Token-based only (no OAuth, SAML, etc.)
- Tokens configured in server config file
- No runtime user management
## Known Edge Cases
### Simultaneous Deletes and Edits
```
User A deletes note.md
User B edits note.md
Result: Edit wins (file recreated with B's content)
```
Operational transformation prioritises content preservation.
### Large File Uploads
Files > 100 MB may time out on slow connections. Increase `response_timeout_seconds` or split large files.
### Mobile Sync
- Mobile networks may drop WebSocket connections frequently
- Client auto-reconnects, but causes sync delays
- Battery impact from constant reconnections
## What VaultLink is NOT
- **Not a backup solution**: Version history helps but isn't a backup (make backups!)
- **Not Git**: No branching, no commit messages, no diffs to review before merge
- **Not encrypted storage**: Server sees everything
- **Not multi-master**: One server, multiple clients (not peer-to-peer)
## Recommendations
### Good Use Cases
- Personal multi-device sync (< 10 devices)
- Small team collaboration (< 20 people)
- Primarily text/Markdown content
- Trusted server environment
### Poor Use Cases
- Large teams (> 50 concurrent users per vault)
- Primarily binary files (images, videos, large PDFs)
- Untrusted server (need E2E encryption)
- Highly regulated environments (HIPAA, etc.)
## Next Steps
- [Server configuration limits →](/config/server)
- [Advanced tuning →](/config/advanced)
- [Architecture details →](/architecture/)

71
docs/guide/what-is-vaultlink.md
View file

@ -1,71 +0,0 @@
# What is VaultLink?
Self-hosted sync for Obsidian vaults with automatic conflict-free merging. Edit with any tool, collaborate in real-time, no conflict markers.
## The Problem
Syncing Obsidian vaults across devices or sharing with teammates sucks:
- **Commercial services**: Lock-in, subscriptions, third-party access to your data
- **Git**: Manual conflict resolution with `<<<<<<<` markers interrupting your workflow
- **Cloud storage**: Last-write-wins data loss or manual conflict resolution
- **CRDT solutions**: Only work if you edit inside Obsidian (break if you use Vim, VS Code, etc.)
## VaultLink's Solution
Differential synchronisation with operational transformation for Markdown and text files.
Edit `.md` and `.txt` files with Obsidian, Vim, VS Code, or any editor. VaultLink compares versions and automatically merges all changes. No operation tracking required, no conflict markers.
**Note**: Binary files (images, PDFs, etc.) use last-write-wins. [See limitations →](/guide/limitations)
## How It Works
1. **Server**: Rust WebSocket server with SQLite stores document versions
2. **Clients**: Obsidian plugin or CLI client watches filesystem changes
3. **Sync**: Changes upload to server, server broadcasts to other clients
4. **Merge**: [reconcile-text](https://schmelczer.dev/reconcile) automatically merges concurrent edits
No CRDT infrastructure. No operation logs. Just file comparison and smart merging.
## Key Advantages
**Editor agnostic**: Edit files with any tool. Other solutions break when you edit outside their ecosystem.
**Self-hosted**: Your data, your server. No third parties, no subscriptions, no surprises.
**Automatic merging**: Operational transformation handles conflicts without interrupting your workflow.
**Production-ready**: Comprehensive tests, E2E tests, battle-tested. Many alternatives have zero tests.
**Collaborative**: Real-time sync with cursor tracking. See where teammates are editing.
## Not Tied to Obsidian
VaultLink syncs Markdown files. Use it for:
- Obsidian vaults (Obsidian desktop + mobile + CLI)
- Technical documentation (VS Code, your-editor, CLI)
- Academic writing (multiple Markdown editors)
- Automated workflows (CLI client for backups/CI/CD)
The Obsidian plugin is just a convenience wrapper around the sync client.
## Quick Comparison
| Feature | VaultLink | Git | Cloud Sync | CRDT Solutions |
| ------------------- | --------- | --- | ---------- | -------------- |
| Self-hosted | ✅ | ✅ | ❌ | Varies |
| Any editor | ✅ | ✅ | ✅ | ❌ |
| No conflict markers | ✅ | ❌ | ❌ | ✅ |
| Real-time | ✅ | ❌ | ❌ | ✅ |
| No subscriptions | ✅ | ✅ | ❌ | Varies |
| Comprehensive tests | ✅ | N/A | N/A | ❌ |
[Detailed comparison with alternatives →](/guide/alternatives)
## Next Steps
- [Get started →](/guide/getting-started) (5 minute setup)
- [See the architecture →](/architecture/) (understand how it works)
- [Compare alternatives →](/guide/alternatives) (why VaultLink vs others)

50
docs/index.md
View file

@ -3,7 +3,7 @@ layout: home
hero:
name: VaultLink
text: Self-Hosted Obsidian Sync
text: Self-Hosted Sync & Collaboration for Obsidian and beyond
tagline: Edit with any tool. Automatic conflict-free merging. Your infrastructure.
image:
src: /logo.svg
@ -17,29 +17,42 @@ hero:
link: /guide/what-is-vaultlink
features:
- title: Edit Anywhere
details: Use Obsidian, Vim, VS Code, or any editor. VaultLink syncs files, not keystrokes—edit however you want
- title: Single-binary server meant for self-hosting
details: Simple Rust-powered WebSocket server with SQLite.
- title: Your Data, Your Server
details: Fully self-hosted. No third parties, no subscriptions, no data mining. Single Docker container or binary
- title: No Conflict Markers
details: Automatic merge using operational transformation. Never see conflict markers in your notes again
details: Built with self-hosting in mind. Single simple Docker container or binary.
- title: Real-Time Collaboration
details: See teammate cursors, merge edits instantly. Rust-powered WebSocket server with SQLite
details: See your collaborators' cursors and edits instantly. Or just use it to sync your files across devices.
- title: Obsidian plugin
details: First-class support for Obsidian through the [VaultLink]() plugin. Sync changes coming from both within and outside of your Obsidian Vault.
- title: Interoperability is the new default
details: VaultLink isn't limited to Obsidian. It comes with a client CLI, and can be easily embedded into other editors and knowledgebases as a plugin to unlock true interoperability.
- title: No Conflict Markers
details: Never see conflict markers in your notes again. Automatic smart merging for text without human intervention which never drops changes. See the [reconcile demo](https://schmelczer.dev/reconcile) for an intuitive visualisation.
- title: Open Source Everything
details: MIT licensed. Server, clients, and sync algorithm are all open source. No proprietary components
- title: Battle-Tested
details: Comprehensive test suite. E2E tests. Used in production. Unlike alternatives with zero tests
details: MIT licensed. Server, clients, and sync algorithm are all open source. No proprietary components.
---
## Why Self-Host?
## What is VaultLink?
**You own your knowledge base.** Commercial sync services can disappear, change pricing, or lock you out. VaultLink runs on your infrastructure—VPS, home server, or localhost.
VaultLink is an editor agnostic sync & collaboration engine consisting of:
- a self-hostable server
- an [Obsidian](https://obsidian.md) plugin
- and an optional plugin library & CLI client for integrating with other editors
**Edit with any tool.** Other solutions require CRDT-aware editors or break when you edit outside Obsidian. VaultLink uses differential sync: edit files however you want, sync handles the rest.
I'm deeply concived that the shape of a personal knwoledgebase, stack of notes, or "operating system for life" is a folder of Markdown files and attachments. I believe in a rich editing experience built on top of plain text files, owning your own data, and eliminating moats between propriety products.
**No conflict markers.** Git forces manual merging. Other tools use last-write-wins. VaultLink's operational transformation automatically merges Markdown and text files without conflict markers or workflow interruption. [See what's supported →](/guide/limitations)
Obsidian is a well-loved editor ([see the 2025 self-hosting survey]()). Even though it supports E2EE Syncing, it's done through a propriety plugin limited to Obsidian which also lacks support for collaboration. Existing 3rd-party syncing solutions fall short of delivering a stable and editor-agnostic experience. For more on this, see the [alternatives](/guide/alternatives).
VaultLink finally delivers a reliable, self-hosted, and editor-agnostic syncing solution with first-class support for Obsidian.
> And more editors to come in the future, see the [roadmap]().
[See how VaultLink compares to alternatives →](/guide/alternatives)
## Quick Start
@ -53,3 +66,10 @@ docker run -d -p 3000:3000 -v $(pwd)/data:/data \
Then install the [Obsidian plugin](/guide/obsidian-plugin) or [CLI client](/guide/cli-client).
[Full setup guide →](/guide/getting-started)
## What makes VaultLink special
VaultLink's architecture enables a few unique features:
- Its text merging algorithm handles conflicting (even offline & coming from outside the editor) edits without human oversights while ensuring no changes are dropped. Learn more about this in the [syncing algorithm section](architecture/sync-algorithm.md).
- Provides its full feature set through shallow integration meaning that it's straightforward to integrate into editors even ones which haven't been written with real-time collaboration and remote backups in mind. This is detailed in the [architecture section](architecture/index.md).
- There are various text editor/wiki/knowledgebase style apps & websites around. Users should be able to mix & match between them, try them out, and collaborate with people preferring a different editor than themselves. The long-term goal of VaultLink is to break down these barriers. As of today, the two clients are a CLI exectuable client and an Obsidian plugin. However, there's more to come. So for an overview, head to the [roadmap](roadmap.md) page.