This commit is contained in:
Andras Schmelczer 2025-11-22 11:19:08 +00:00
parent 56c1f4d58b
commit 50a95b114d
19 changed files with 4663 additions and 1 deletions

516
docs/guide/cli-client.md Normal file
View file

@ -0,0 +1,516 @@
# CLI Client
The VaultLink CLI client provides standalone synchronization without requiring Obsidian. Perfect for servers, automation, backups, or syncing vaults on headless systems.
## Installation
### Docker (Recommended)
Pull the latest image:
```bash
docker pull ghcr.io/schmelczer/vault-link-cli:latest
```
### npm
Install globally:
```bash
npm install -g @schmelczer/local-client-cli
```
Verify installation:
```bash
vaultlink --version
```
### From Source
Build from the repository:
```bash
git clone https://github.com/schmelczer/vault-link.git
cd vault-link/frontend/local-client-cli
npm install
npm run build
node dist/cli.js --help
```
## Usage
### Basic Usage
```bash
vaultlink \
--local-path /path/to/vault \
--remote-uri wss://sync.example.com \
--token your-auth-token \
--vault-name default
```
### Docker Usage
```bash
docker run -v /path/to/vault:/vault \
ghcr.io/schmelczer/vault-link-cli:latest \
-l /vault \
-r wss://sync.example.com \
-t your-auth-token \
-v default
```
### Docker Compose
Create `docker-compose.yml`:
```yaml
services:
vaultlink-cli:
image: ghcr.io/schmelczer/vault-link-cli:latest
restart: unless-stopped
volumes:
- ./vault:/vault
command:
- "-l"
- "/vault"
- "-r"
- "wss://sync.example.com"
- "-t"
- "your-token"
- "-v"
- "default"
```
Start the client:
```bash
docker compose up -d
```
## Configuration Options
### Required Arguments
| Argument | Short | Description | Example |
|----------|-------|-------------|---------|
| `--local-path` | `-l` | Local directory to sync | `/vault` |
| `--remote-uri` | `-r` | Server WebSocket URI | `wss://sync.example.com` |
| `--token` | `-t` | Authentication token | `abc123...` |
| `--vault-name` | `-v` | Vault name on server | `default` |
### Optional Arguments
| Argument | Default | Description |
|----------|---------|-------------|
| `--sync-concurrency` | `1` | Concurrent file operations |
| `--max-file-size-mb` | `10` | Max file size in MB |
| `--ignore-pattern` | - | Glob pattern to ignore (repeatable) |
| `--websocket-retry-interval-ms` | `3500` | Reconnection interval |
| `--log-level` | `INFO` | Log level: DEBUG, INFO, WARNING, ERROR |
### Environment Variables
Alternative to command-line arguments:
```bash
export VAULTLINK_LOCAL_PATH="/vault"
export VAULTLINK_REMOTE_URI="wss://sync.example.com"
export VAULTLINK_TOKEN="your-token"
export VAULTLINK_VAULT_NAME="default"
vaultlink
```
## Examples
### Basic Sync
Sync a local directory to the server:
```bash
vaultlink \
-l ./my-notes \
-r wss://sync.example.com \
-t my-secure-token \
-v personal
```
### With Ignore Patterns
Exclude specific files or directories:
```bash
vaultlink \
-l ./vault \
-r wss://sync.example.com \
-t token123 \
-v default \
--ignore-pattern "*.tmp" \
--ignore-pattern ".DS_Store" \
--ignore-pattern "node_modules/**"
```
### Debug Logging
Enable verbose logging:
```bash
vaultlink \
-l ./vault \
-r wss://sync.example.com \
-t token123 \
-v default \
--log-level DEBUG
```
### High Concurrency
Faster initial sync:
```bash
vaultlink \
-l ./vault \
-r wss://sync.example.com \
-t token123 \
-v default \
--sync-concurrency 5
```
### Large Files
Allow larger file uploads:
```bash
vaultlink \
-l ./vault \
-r wss://sync.example.com \
-t token123 \
-v default \
--max-file-size-mb 50
```
## Docker Deployment
### Long-Running Sync
Run as a daemon for continuous synchronization:
```bash
docker run -d \
--name vaultlink-sync \
--restart unless-stopped \
-v $(pwd)/vault:/vault \
ghcr.io/schmelczer/vault-link-cli:latest \
-l /vault \
-r wss://sync.example.com \
-t your-token \
-v default
```
Monitor logs:
```bash
docker logs -f vaultlink-sync
```
### Health Monitoring
The Docker image includes built-in health checks:
```bash
# Check health status
docker ps
# View detailed health info
docker inspect --format='{{json .State.Health}}' vaultlink-sync | jq
```
Health check verifies:
- Health file exists
- Status updated within last 30 seconds
- WebSocket connection is active
Configure custom health check:
```yaml
services:
vaultlink-cli:
image: ghcr.io/schmelczer/vault-link-cli:latest
healthcheck:
test: ["CMD", "node", "/app/healthcheck.js"]
interval: 15s
timeout: 5s
retries: 5
start_period: 20s
```
### Read-Only Vault
Mount vault as read-only to prevent local changes:
```bash
docker run -d \
-v $(pwd)/vault:/vault:ro \
ghcr.io/schmelczer/vault-link-cli:latest \
-l /vault \
-r wss://sync.example.com \
-t token \
-v default
```
::: warning
The CLI needs write access to create `.vaultlink` metadata directory. Mount as read-write or provide a separate writeable directory.
:::
## How It Works
### Initial Sync
On startup:
1. Creates `.vaultlink/` directory for metadata
2. Scans local filesystem
3. Uploads all local files to server
4. Downloads files from server not present locally
5. Resolves conflicts using operational transformation
### Real-Time Synchronization
After initial sync:
1. Watches filesystem for changes using `fs.watch`
2. Uploads changed files immediately
3. Receives real-time updates from server via WebSocket
4. Handles bidirectional sync automatically
### Graceful Shutdown
On SIGINT (Ctrl+C) or SIGTERM:
1. Completes pending uploads
2. Closes WebSocket connection cleanly
3. Flushes metadata to disk
4. Exits gracefully
## Use Cases
### Automated Backups
Continuously backup vaults to a remote server:
```bash
docker run -d \
--name vault-backup \
-v /important/notes:/vault:ro \
ghcr.io/schmelczer/vault-link-cli:latest \
-l /vault -r wss://backup.example.com -t backup-token -v backups
```
### CI/CD Documentation
Sync documentation in automated pipelines:
```bash
# In your CI pipeline
docker run \
-v $(pwd)/docs:/vault \
ghcr.io/schmelczer/vault-link-cli:latest \
-l /vault -r wss://docs.example.com -t ci-token -v prod-docs
```
### Multi-Location Sync
Sync between different geographic locations:
```bash
# Location A
vaultlink -l /data/vault -r wss://hub.example.com -t token -v shared
# Location B
vaultlink -l /backup/vault -r wss://hub.example.com -t token -v shared
```
### Development Environment
Keep documentation in sync across dev environments:
```bash
# In docker-compose.yml for your dev stack
services:
docs-sync:
image: ghcr.io/schmelczer/vault-link-cli:latest
volumes:
- ./docs:/vault
command: ["-l", "/vault", "-r", "wss://docs-server", "-t", "dev-token", "-v", "dev"]
```
## Troubleshooting
### Client won't connect
**Check server accessibility**:
```bash
curl https://sync.example.com/vaults/test/ping
```
**Verify WebSocket protocol**:
- Use `ws://` for HTTP servers
- Use `wss://` for HTTPS servers
**Check authentication**:
- Token must match server config
- User must have access to the vault
### Permission errors
**Docker volume permissions**:
```bash
# Ensure directory is writable
chmod 755 /path/to/vault
# Check Docker user ID
docker run --rm ghcr.io/schmelczer/vault-link-cli:latest id
```
**SELinux issues**:
```bash
# Add :z flag to volume mount
docker run -v /path/to/vault:/vault:z ...
```
### Files not syncing
**Check ignore patterns**:
- View logs to see which files are skipped
- Ensure patterns don't match unintentionally
**File size limits**:
- Check `--max-file-size-mb` setting
- Large files are skipped with a warning
**Check metadata**:
```bash
# View sync metadata
cat /path/to/vault/.vaultlink/metadata.json
```
### High memory usage
**Reduce concurrency**:
```bash
--sync-concurrency 1
```
**Limit file sizes**:
```bash
--max-file-size-mb 5
```
**Check vault size**:
- Very large vaults may need more resources
- Consider splitting into multiple vaults
### Connection keeps dropping
**Increase retry interval**:
```bash
--websocket-retry-interval-ms 5000
```
**Check network stability**:
```bash
# Monitor connection
docker logs -f vaultlink-sync | grep -i websocket
```
**Server timeout settings**:
- Verify reverse proxy WebSocket timeout
- Check server `response_timeout_seconds`
## Advanced Usage
### Custom Healthcheck Script
Create your own health monitoring:
```bash
#!/bin/bash
HEALTH_FILE="/tmp/vaultlink-health.json"
if [ ! -f "$HEALTH_FILE" ]; then
exit 1
fi
# Check file is recent (within 60 seconds)
if [ $(( $(date +%s) - $(stat -c %Y "$HEALTH_FILE") )) -gt 60 ]; then
exit 1
fi
# Check WebSocket is connected
if ! jq -e '.connected == true' "$HEALTH_FILE" > /dev/null; then
exit 1
fi
exit 0
```
### Automated Recovery
Restart on failure with exponential backoff:
```bash
#!/bin/bash
RETRY_DELAY=5
while true; do
vaultlink -l /vault -r wss://server -t token -v default
echo "Client exited, restarting in ${RETRY_DELAY}s..."
sleep $RETRY_DELAY
# Exponential backoff up to 5 minutes
RETRY_DELAY=$((RETRY_DELAY * 2))
if [ $RETRY_DELAY -gt 300 ]; then
RETRY_DELAY=300
fi
done
```
### Integration with systemd
Create `/etc/systemd/system/vaultlink-cli.service`:
```ini
[Unit]
Description=VaultLink CLI Sync
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
Restart=always
RestartSec=10
Environment="VAULTLINK_LOCAL_PATH=/data/vault"
Environment="VAULTLINK_REMOTE_URI=wss://sync.example.com"
Environment="VAULTLINK_TOKEN=your-token"
Environment="VAULTLINK_VAULT_NAME=default"
ExecStart=/usr/local/bin/vaultlink
[Install]
WantedBy=multi-user.target
```
Enable and start:
```bash
sudo systemctl daemon-reload
sudo systemctl enable vaultlink-cli
sudo systemctl start vaultlink-cli
```
## Next Steps
- [Configure server authentication →](/config/authentication)
- [Learn about the sync algorithm →](/architecture/sync-algorithm)
- [Set up Obsidian plugin →](/guide/obsidian-plugin)

View file

@ -0,0 +1,185 @@
# Getting Started
This guide will walk you through setting up VaultLink from scratch. You'll have a working sync server and connected client in under 10 minutes.
## Prerequisites
- Docker installed (recommended) or Rust toolchain for building from source
- Basic familiarity with command line
- A server or machine to host the sync server (can be localhost for testing)
## Quick Start
### Step 1: Deploy the Sync Server
The fastest way to get started is with Docker:
```bash
# Create a directory for server data
mkdir -p ~/vaultlink-data
cd ~/vaultlink-data
# Create a basic configuration file
cat > config.yml << 'EOF'
database:
databases_directory_path: databases
max_connections_per_vault: 12
cursor_timeout_seconds: 60
server:
host: 0.0.0.0
port: 3000
max_body_size_mb: 512
max_clients_per_vault: 256
response_timeout_seconds: 60
users:
user_configs:
- name: admin
token: change-this-to-a-secure-random-token
vault_access:
type: allow_access_to_all
logging:
log_directory: logs
log_rotation: 7days
EOF
# Run the server
docker run -d \
--name vaultlink-server \
--restart unless-stopped \
-p 3000:3000 \
-v $(pwd):/data \
ghcr.io/schmelczer/vault-link-server:latest \
/app/sync_server /data/config.yml
```
::: warning
Change the token in `config.yml` to a secure random value before deploying to production!
:::
Verify the server is running:
```bash
curl http://localhost:3000/vaults/test/ping
```
You should see: `pong`
### Step 2: Choose Your Client
You can connect to VaultLink using either the Obsidian plugin or the standalone CLI client.
#### Option A: Obsidian Plugin
1. Open Obsidian Settings → Community Plugins
2. Browse community plugins and search for "VaultLink"
3. Install and enable the plugin
4. Configure the plugin:
- **Server URL**: `ws://localhost:3000` (or your server address)
- **Token**: The token from your `config.yml`
- **Vault Name**: `default` (or any name you choose)
[Read the full Obsidian plugin guide →](/guide/obsidian-plugin)
#### Option B: CLI Client
Perfect for syncing vaults without Obsidian:
```bash
docker run -d \
--name vaultlink-cli \
--restart unless-stopped \
-v /path/to/your/vault:/vault \
ghcr.io/schmelczer/vault-link-cli:latest \
-l /vault \
-r ws://localhost:3000 \
-t change-this-to-a-secure-random-token \
-v default
```
Replace `/path/to/your/vault` with the directory containing your files.
[Read the full CLI client guide →](/guide/cli-client)
## Next Steps
### Production Deployment
For production use, you should:
1. **Use HTTPS/WSS**: Put the sync server behind a reverse proxy with SSL
2. **Secure tokens**: Generate cryptographically random tokens
3. **Configure backups**: Back up the SQLite databases regularly
4. **Set up monitoring**: Use Docker health checks and logging
[Learn about production deployment →](/guide/server-setup#production-deployment)
### Multiple Users
To add more users or restrict vault access:
```yaml
users:
user_configs:
- name: alice
token: alice-secure-token
vault_access:
type: allow_list
allowed:
- personal
- shared
- name: bob
token: bob-secure-token
vault_access:
type: allow_list
allowed:
- shared
```
[Learn about authentication configuration →](/config/authentication)
### Advanced Configuration
Explore advanced server options:
- Database tuning for large vaults
- Rate limiting and connection limits
- Custom logging and log rotation
- Multi-vault setups
[View configuration reference →](/config/server)
## Architecture Overview
Want to understand how VaultLink works under the hood?
[Read the architecture documentation →](/architecture/)
## Troubleshooting
### Server won't start
Check Docker logs:
```bash
docker logs vaultlink-server
```
Common issues:
- Port 3000 already in use: Change the port mapping `-p 3001:3000`
- Config file errors: Validate YAML syntax
- Permission issues: Ensure the volume mount is writable
### Client can't connect
1. Verify server is accessible: `curl http://your-server:3000/vaults/test/ping`
2. Check WebSocket connectivity (browser dev tools or wscat)
3. Verify token matches between client and server config
4. Check firewall rules allow port 3000
### Files not syncing
1. Check client logs for errors
2. Verify vault name matches on both server and client
3. Ensure user has access to the vault (check server config)
4. Check for file size limits (default 10MB for CLI)
For more help, [open an issue on GitHub](https://github.com/schmelczer/vault-link/issues).

View file

@ -0,0 +1,262 @@
# Obsidian Plugin
The VaultLink Obsidian plugin provides native real-time synchronization directly within Obsidian.
## Installation
### From Obsidian Community Plugins
1. Open Obsidian Settings
2. Navigate to **Community Plugins**
3. Click **Browse** and search for "VaultLink"
4. Click **Install**
5. Enable the plugin
### Manual Installation
1. Download the latest release from [GitHub Releases](https://github.com/schmelczer/vault-link/releases)
2. Extract `main.js`, `manifest.json`, and `styles.css`
3. Copy to `.obsidian/plugins/vault-link/` in your vault
4. Reload Obsidian
5. Enable VaultLink in Community Plugins settings
## Configuration
After installation, configure the plugin in **Settings → VaultLink**.
### Required Settings
#### Server URL
The WebSocket URL of your sync server.
- **Development/Local**: `ws://localhost:3000`
- **Production (SSL)**: `wss://sync.example.com`
::: tip
Use `ws://` for unencrypted connections and `wss://` for SSL connections (production).
:::
#### Authentication Token
Your authentication token from the server's `config.yml`.
Generate a secure token:
```bash
openssl rand -hex 32
```
#### Vault Name
The name of the vault on the server. Can be any string.
Multiple Obsidian vaults can sync to the same server vault name (for shared vaults), or use unique names for separate vaults.
### Optional Settings
#### Sync Concurrency
Number of files to sync simultaneously.
- **Default**: 1
- **Range**: 1-10
- Higher values = faster initial sync, more resource usage
#### Max File Size
Maximum file size to sync (in MB).
- **Default**: 10
- Files larger than this are skipped
#### Ignore Patterns
Glob patterns for files to exclude from sync.
Examples:
- `*.tmp` - Ignore temporary files
- `.trash/**` - Ignore trash folder
- `private/**` - Ignore private directory
#### WebSocket Retry Interval
Milliseconds between reconnection attempts when disconnected.
- **Default**: 3500ms
- Increase for flaky networks to avoid connection spam
## Usage
### Initial Sync
When first connecting:
1. The plugin uploads all local files to the server
2. Downloads any missing files from the server
3. Resolves any conflicts using operational transformation
4. Begins real-time synchronization
Initial sync time depends on vault size and `sync_concurrency` setting.
### Real-Time Sync
Once connected:
- **File changes**: Automatically synced when saved
- **File creation**: New files immediately uploaded
- **File deletion**: Deletions propagated to other clients
- **File renames**: Tracked and synchronized
The plugin watches your vault filesystem and syncs changes in real-time via WebSocket.
### Status Indicators
The plugin provides visual feedback:
- **Connected**: Green status in settings
- **Syncing**: Progress indicator during uploads
- **Disconnected**: Red status, automatic reconnection attempts
- **Error**: Error message in settings and console
Check the Obsidian console (Ctrl+Shift+I / Cmd+Option+I) for detailed logs.
## Features
### Automatic Conflict Resolution
When multiple users edit the same file simultaneously, operational transformation merges changes automatically:
- All edits are preserved
- No manual conflict resolution required
- Changes appear in real-time as others type
### Mobile Support
VaultLink works on Obsidian mobile (iOS and Android):
- Same configuration as desktop
- Real-time sync across all devices
- Handle network changes gracefully
::: warning
Ensure your sync server is accessible from mobile networks (use WSS with a public domain or VPN).
:::
### Offline Support
The plugin handles offline scenarios:
- Continue working when disconnected
- Changes queue locally
- Automatic sync when connection restored
- Conflict resolution if others edited the same files
## Collaboration Workflows
### Personal Multi-Device Sync
Sync the same vault across devices:
1. Configure each Obsidian instance with the same vault name
2. Use the same authentication token
3. All devices stay in sync automatically
### Team Shared Vault
Multiple users collaborating:
1. Each user has their own token (configured in server `config.yml`)
2. All users connect to the same vault name
3. Real-time collaborative editing with automatic conflict resolution
### Selective Sharing
Share specific folders while keeping others private:
1. Use different vault names for shared vs. private content
2. Configure access control on the server per vault
3. Use ignore patterns to exclude sensitive directories
## Troubleshooting
### Plugin won't connect
1. **Verify server is running**:
```bash
curl http://your-server:3000/vaults/test/ping
```
Should return `pong`
2. **Check URL format**:
- Local: `ws://localhost:3000`
- Remote (SSL): `wss://sync.example.com`
- Don't include `/vault/name` in the URL
3. **Verify token**:
- Must match server config exactly
- No extra spaces or quotes
- Check server logs for authentication errors
4. **Check firewall**:
- Ensure port is accessible from your network
- For mobile, server must be publicly accessible or use VPN
### Files not syncing
1. **Check ignore patterns**: File may match an exclusion pattern
2. **File size**: Check if file exceeds `max_file_size_mb`
3. **Permissions**: Ensure vault directory is readable/writable
4. **Console errors**: Open dev tools (Ctrl+Shift+I) and check console
### Slow initial sync
1. **Increase concurrency**: Set `sync_concurrency` higher (e.g., 5)
2. **Network speed**: Check internet connection
3. **Server resources**: Ensure server isn't overloaded
4. **Large files**: Consider increasing timeout settings
### Conflicts not resolving
Operational transformation should handle conflicts automatically. If issues persist:
1. Check console for sync errors
2. Verify both clients are connected
3. Check server logs for processing errors
4. Ensure files are text-based (binary files may not merge well)
### High CPU/Memory usage
1. **Reduce concurrency**: Lower `sync_concurrency`
2. **Add ignore patterns**: Exclude unnecessary files
3. **File watchers**: Large vaults may trigger many filesystem events
4. **Check for sync loops**: Ensure no circular dependencies
## Advanced Configuration
### Multiple Vaults
To sync multiple Obsidian vaults to different server vaults:
1. Each Obsidian vault has its own VaultLink plugin configuration
2. Use different vault names for each
3. Can use the same or different tokens (depending on access control)
### Custom Sync Patterns
Combine ignore patterns for fine-grained control:
```
# Ignore patterns
*.tmp
*.bak
.DS_Store
.trash/**
private/**
drafts/**/*.draft.md
```
### Development/Testing
For plugin development:
1. Clone the repository
2. `cd frontend && npm install`
3. `npm run dev` to build in watch mode
4. Plugin rebuilds automatically on changes
5. Reload Obsidian to test changes
## Next Steps
- [Learn about the sync algorithm →](/architecture/sync-algorithm)
- [Configure the server →](/config/server)
- [Set up the CLI client →](/guide/cli-client)

370
docs/guide/server-setup.md Normal file
View file

@ -0,0 +1,370 @@
# Server Setup
This guide covers deploying the VaultLink sync server in various environments, from local development to production infrastructure.
## Deployment Options
### Docker (Recommended)
Docker provides the easiest deployment path with built-in health checks and minimal dependencies.
#### Basic Docker Deployment
```bash
# Pull the latest image
docker pull ghcr.io/schmelczer/vault-link-server:latest
# Create data directory
mkdir -p ~/vaultlink-data
# Create config.yml (see Configuration section below)
# Run the container
docker run -d \
--name vaultlink-server \
--restart unless-stopped \
-p 3000:3000 \
-v ~/vaultlink-data:/data \
ghcr.io/schmelczer/vault-link-server:latest \
/app/sync_server /data/config.yml
```
#### Docker Compose
Create `docker-compose.yml`:
```yaml
services:
vaultlink-server:
image: ghcr.io/schmelczer/vault-link-server:latest
container_name: vaultlink-server
restart: unless-stopped
ports:
- "3000:3000"
volumes:
- ./data:/data
command: ["/app/sync_server", "/data/config.yml"]
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:3000/vaults/fake/ping"]
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
```
Start the server:
```bash
docker compose up -d
```
### Binary Installation
Download pre-built binaries from [GitHub Releases](https://github.com/schmelczer/vault-link/releases).
```bash
# Download the binary for your platform
wget https://github.com/schmelczer/vault-link/releases/latest/download/sync_server-linux-x86_64
# Make executable
chmod +x sync_server-linux-x86_64
# Run the server
./sync_server-linux-x86_64 config.yml
```
### Build from Source
Requirements:
- Rust 1.89.0 or later
- SQLite development headers
- SQLx CLI
```bash
# Clone the repository
git clone https://github.com/schmelczer/vault-link.git
cd vault-link/sync-server
# Install SQLx CLI
cargo install sqlx-cli
# Set up the database
sqlx database create --database-url sqlite://db.sqlite3
sqlx migrate run --source src/app_state/database/migrations --database-url sqlite://db.sqlite3
cargo sqlx prepare --workspace
# Build in release mode
cargo build --release
# Run the server
./target/release/sync_server config.yml
```
## Configuration
Create a `config.yml` file with your server configuration:
```yaml
database:
databases_directory_path: databases
max_connections_per_vault: 12
cursor_timeout_seconds: 60
server:
host: 0.0.0.0
port: 3000
max_body_size_mb: 512
max_clients_per_vault: 256
response_timeout_seconds: 60
users:
user_configs:
- name: admin
token: your-secure-random-token-here
vault_access:
type: allow_access_to_all
logging:
log_directory: logs
log_rotation: 7days
```
### Configuration Fields
#### Database
- `databases_directory_path`: Directory for SQLite databases (one per vault)
- `max_connections_per_vault`: Maximum concurrent database connections
- `cursor_timeout_seconds`: How long to keep database cursors alive
#### Server
- `host`: Bind address (use `0.0.0.0` for all interfaces)
- `port`: Port to listen on (default: 3000)
- `max_body_size_mb`: Maximum upload size
- `max_clients_per_vault`: Concurrent client limit per vault
- `response_timeout_seconds`: Request timeout
#### Users
See [Authentication Configuration →](/config/authentication) for detailed user setup.
#### Logging
- `log_directory`: Where to store log files
- `log_rotation`: How often to rotate logs (e.g., `7days`, `24hours`)
## Production Deployment
### SSL/TLS with Reverse Proxy
VaultLink doesn't handle SSL directly. Use a reverse proxy like Nginx or Caddy.
#### Nginx Configuration
```nginx
upstream vaultlink {
server localhost:3000;
}
server {
listen 443 ssl http2;
server_name sync.example.com;
ssl_certificate /path/to/cert.pem;
ssl_certificate_key /path/to/key.pem;
location / {
proxy_pass http://vaultlink;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# WebSocket specific
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
}
}
```
Reload Nginx:
```bash
sudo nginx -t
sudo systemctl reload nginx
```
#### Caddy Configuration
Caddy handles SSL automatically:
```caddy
sync.example.com {
reverse_proxy localhost:3000
}
```
Start Caddy:
```bash
caddy run --config Caddyfile
```
### Systemd Service
Create `/etc/systemd/system/vaultlink.service`:
```ini
[Unit]
Description=VaultLink Sync Server
After=network.target
[Service]
Type=simple
User=vaultlink
WorkingDirectory=/opt/vaultlink
ExecStart=/opt/vaultlink/sync_server /opt/vaultlink/config.yml
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
```
Enable and start:
```bash
sudo systemctl daemon-reload
sudo systemctl enable vaultlink
sudo systemctl start vaultlink
sudo systemctl status vaultlink
```
### Security Best Practices
1. **Use strong tokens**: Generate with `openssl rand -hex 32`
2. **Enable firewall**: Only expose port 3000 to reverse proxy
3. **Regular updates**: Keep Docker images and binaries updated
4. **Backup databases**: SQLite files in `databases_directory_path`
5. **Monitor logs**: Check log directory for errors and anomalies
6. **Limit access**: Use vault-specific access controls per user
### Backup Strategy
The SQLite databases contain all vault data and history:
```bash
# Backup script
#!/bin/bash
BACKUP_DIR="/backup/vaultlink/$(date +%Y%m%d)"
DATA_DIR="/data/databases"
mkdir -p "$BACKUP_DIR"
cp -r "$DATA_DIR" "$BACKUP_DIR/"
# Keep 30 days of backups
find /backup/vaultlink -type d -mtime +30 -exec rm -rf {} +
```
Run daily via cron:
```cron
0 2 * * * /opt/vaultlink/backup.sh
```
### Monitoring
#### Health Checks
The server exposes a ping endpoint:
```bash
curl http://localhost:3000/vaults/fake/ping
# Returns: pong
```
Docker health check is built-in and checks this endpoint every 30 seconds.
#### Prometheus Metrics
For advanced monitoring, collect Docker stats or implement custom metrics.
#### Log Monitoring
Logs are written to the configured `log_directory`. Monitor for:
- Connection failures
- Authentication errors
- Database errors
- WebSocket disconnections
Example log watching:
```bash
tail -f /data/logs/*.log | grep -i error
```
## Scaling
### Horizontal Scaling
VaultLink currently uses SQLite, which limits horizontal scaling. For multiple servers:
1. Run separate instances for different vaults
2. Use load balancer with sticky sessions (same vault → same server)
3. Consider database architecture for your scale needs
### Vertical Scaling
Increase resources for the server:
- More CPU for handling concurrent connections
- More RAM for database caching
- Faster storage (SSD) for database operations
Tune configuration:
- Increase `max_clients_per_vault` for more concurrent users
- Increase `max_connections_per_vault` for database performance
- Adjust `max_body_size_mb` based on typical file sizes
## Troubleshooting
### Server won't start
```bash
# Check Docker logs
docker logs vaultlink-server
# Common issues:
# - Port already in use: Change port mapping
# - Config syntax error: Validate YAML
# - Permission error: Check volume permissions
```
### High memory usage
- Reduce `max_connections_per_vault`
- Reduce `max_clients_per_vault`
- Check for large vaults (may need database optimization)
### Database corruption
```bash
# Verify database integrity
sqlite3 databases/your-vault.db "PRAGMA integrity_check;"
# If corrupted, restore from backup
cp /backup/databases/your-vault.db /data/databases/
```
### WebSocket connection drops
- Check reverse proxy timeout settings
- Verify firewall isn't closing connections
- Review client retry intervals
- Check server logs for errors
## Next Steps
- [Configure authentication and access control →](/config/authentication)
- [Set up Obsidian plugin →](/guide/obsidian-plugin)
- [Deploy CLI client →](/guide/cli-client)
- [Understand the architecture →](/architecture/)

View file

@ -0,0 +1,115 @@
# What is VaultLink?
VaultLink is a self-hosted real-time synchronization system for Obsidian vaults. It provides collaborative file syncing with automatic conflict resolution, designed for users who want complete control over their data.
## Overview
VaultLink consists of three main components:
### Sync Server
A Rust-based WebSocket server that handles:
- Real-time bidirectional synchronization
- Document versioning with SQLite
- User authentication and vault access control
- Operational transformation for conflict resolution
### Obsidian Plugin
A native Obsidian plugin that:
- Integrates sync directly into your Obsidian workflow
- Provides real-time updates as you edit
- Handles file watching and automatic synchronization
- Works across desktop and mobile platforms
### CLI Client
A standalone synchronization client that:
- Syncs vaults without requiring Obsidian
- Perfect for servers, automation, or backup systems
- Provides file watching and bidirectional sync
- Runs in Docker or as a standalone binary
## Key Features
### Real-Time Synchronization
Changes are synchronized immediately via WebSocket connections. When multiple users edit the same file, operational transformation ensures all edits are preserved without conflicts.
### Self-Hosted Architecture
Run the sync server on your own infrastructure:
- Full control over data storage and access
- No dependency on third-party services
- Configurable authentication and authorization
- Deploy anywhere: cloud VPS, home server, or localhost
### Operational Transformation
VaultLink uses the `reconcile-text` library for intelligent conflict resolution:
- Simultaneous edits are automatically merged
- No manual conflict resolution required
- Preserves intent of all contributors
- Works seamlessly in the background
### Flexible Authentication
Configure user access per vault:
- Token-based authentication
- Per-user vault access control
- Allow-list or deny-list patterns
- Support for multiple users and vaults
## Use Cases
### Personal Sync
Synchronize your Obsidian vault across multiple devices:
- Laptop, desktop, and mobile in real-time
- No cloud service subscription required
- Full privacy and data control
### Team Collaboration
Share knowledge bases with teammates:
- Real-time collaborative editing
- Granular access control per vault
- Self-hosted for enterprise security requirements
### Automated Backups
Use the CLI client for automated workflows:
- Scheduled backups to remote servers
- Integration with existing backup systems
- Headless operation without Obsidian
### Development & Testing
Synchronize documentation across environments:
- Keep docs in sync with development environments
- Automated deployment of documentation
- Version control integration
## How It Works
1. **Server Setup**: Deploy the sync server on your infrastructure
2. **Authentication**: Configure users and vault access in `config.yml`
3. **Client Connection**: Connect via Obsidian plugin or CLI client
4. **Initial Sync**: Client uploads local files to server
5. **Real-Time Updates**: Changes sync bidirectionally via WebSocket
6. **Conflict Resolution**: Operational transformation handles simultaneous edits
## Technology Stack
- **Server**: Rust with Axum framework, SQLite database, WebSocket protocol
- **Frontend**: TypeScript with WebSocket client, npm workspaces
- **Sync Algorithm**: reconcile-text operational transformation library
- **Deployment**: Docker images, binary releases, or source builds
## Next Steps
Ready to get started?
- [Getting Started Guide →](/guide/getting-started)
- [Server Setup →](/guide/server-setup)
- [Architecture Overview →](/architecture/)