andras/vault-link
andras/vault-link
1
0
Fork 0
Code Issues Pull requests 19 Projects Releases 55 Packages Wiki Activity Actions 27

Format without eclint

Browse source
This commit is contained in:
Andras Schmelczer 2025-12-14 14:14:07 +00:00
commit d13abc115d
35 changed files with 3276 additions and 3301 deletions
Download patch file Download diff file Expand all files Collapse all files

4
.github/workflows/deploy-docs.yml vendored
View file

@ -5,8 +5,8 @@ on:
branches: branches:
- main - main
paths: paths:
- 'docs/**' - "docs/**"
- '.github/workflows/deploy-docs.yml' - ".github/workflows/deploy-docs.yml"
workflow_dispatch: workflow_dispatch:
permissions: permissions:

2
.github/workflows/e2e.yml vendored
View file

@ -6,7 +6,7 @@ on:
pull_request: pull_request:
branches: ["main"] branches: ["main"]
schedule: schedule:
- cron: '0 * * * *' - cron: "0 * * * *"
workflow_dispatch: workflow_dispatch:
concurrency: concurrency:

4
.vscode/settings.json vendored
View file

@ -5,6 +5,6 @@
"**/dist": true, "**/dist": true,
"**/node_modules": true, "**/node_modules": true,
"**/.sqlx": true, "**/.sqlx": true,
"**/target": true, "**/target": true
}, }
} }

13
CLAUDE.md
View file

@ -24,6 +24,7 @@ VaultLink is a self-hosted Obsidian plugin for real-time collaborative file sync
## Development Commands ## Development Commands
### Server Development ### Server Development
```bash ```bash
cd sync-server cd sync-server
cargo run config-e2e.yml # Start development server cargo run config-e2e.yml # Start development server
@ -36,6 +37,7 @@ cargo machete --with-metadata # Detect unused dependencies
``` ```
### Frontend Development ### Frontend Development
```bash ```bash
cd frontend cd frontend
npm run dev # Start development mode (watches sync-client and obsidian-plugin) npm run dev # Start development mode (watches sync-client and obsidian-plugin)
@ -45,6 +47,7 @@ npm run lint # Lint and format TypeScript code
``` ```
### Database Setup (Development) ### Database Setup (Development)
```bash ```bash
cd sync-server cd sync-server
sqlx database create --database-url sqlite://db.sqlite3 sqlx database create --database-url sqlite://db.sqlite3
@ -53,12 +56,14 @@ cargo sqlx prepare --workspace
``` ```
### Initial Setup ### Initial Setup
```bash ```bash
# Install required cargo tools # Install required cargo tools
cargo install sqlx-cli cargo-machete cargo-edit cargo install sqlx-cli cargo-machete cargo-edit
``` ```
### Scripts ### Scripts
- `scripts/check.sh`: Full CI check (builds, lints, tests both server and frontend) - `scripts/check.sh`: Full CI check (builds, lints, tests both server and frontend)
- `scripts/check.sh --fix`: Same as above but auto-fixes linting and formatting issues - `scripts/check.sh --fix`: Same as above but auto-fixes linting and formatting issues
- `scripts/e2e.sh`: End-to-end testing - `scripts/e2e.sh`: End-to-end testing
@ -69,16 +74,20 @@ cargo install sqlx-cli cargo-machete cargo-edit
## Code Structure ## Code Structure
### Workspace Configuration ### Workspace Configuration
The frontend uses npm workspaces with four packages: The frontend uses npm workspaces with four packages:
- `sync-client`: Core synchronization logic - `sync-client`: Core synchronization logic
- `obsidian-plugin`: Obsidian-specific integration - `obsidian-plugin`: Obsidian-specific integration
- `test-client`: Testing utilities - `test-client`: Testing utilities
- `local-client-cli`: Standalone CLI for VaultLink sync client - `local-client-cli`: Standalone CLI for VaultLink sync client
### Type Generation ### Type Generation
Rust structs generate TypeScript types via ts-rs crate, stored in `sync-server/bindings/` and used by frontend packages. Rust structs generate TypeScript types via ts-rs crate, stored in `sync-server/bindings/` and used by frontend packages.
### Key Files ### Key Files
- `sync-server/src/`: Rust server implementation with WebSocket handlers - `sync-server/src/`: Rust server implementation with WebSocket handlers
- `frontend/sync-client/src/sync-client.ts`: Main sync client entry point - `frontend/sync-client/src/sync-client.ts`: Main sync client entry point
- `frontend/obsidian-plugin/src/vault-link-plugin.ts`: Main Obsidian plugin class - `frontend/obsidian-plugin/src/vault-link-plugin.ts`: Main Obsidian plugin class
@ -87,11 +96,13 @@ Rust structs generate TypeScript types via ts-rs crate, stored in `sync-server/b
## Testing ## Testing
### Running Tests ### Running Tests
- Server: `cargo test --verbose` - Server: `cargo test --verbose`
- Frontend: `npm run test` (runs Jest across all workspaces) - Frontend: `npm run test` (runs Jest across all workspaces)
- E2E: `scripts/e2e.sh` - E2E: `scripts/e2e.sh`
### Test Structure ### Test Structure
- Rust: Unit tests alongside source files - Rust: Unit tests alongside source files
- TypeScript: `.test.ts` files using Jest - TypeScript: `.test.ts` files using Jest
- E2E: Uses test-client to simulate multiple concurrent users - E2E: Uses test-client to simulate multiple concurrent users
@ -99,12 +110,14 @@ Rust structs generate TypeScript types via ts-rs crate, stored in `sync-server/b
## Code Style ## Code Style
### Rust ### Rust
- Uses extensive Clippy lints (see Cargo.toml) - Uses extensive Clippy lints (see Cargo.toml)
- Follows pedantic linting rules - Follows pedantic linting rules
- Forbids unsafe code - Forbids unsafe code
- Uses cargo fmt with default settings - Uses cargo fmt with default settings
### TypeScript ### TypeScript
- Prettier configuration: 4-space tabs, trailing commas removed, LF line endings - Prettier configuration: 4-space tabs, trailing commas removed, LF line endings
- ESLint with unused imports plugin - ESLint with unused imports plugin
- Consistent across all three frontend packages - Consistent across all three frontend packages

8
README.md
View file

@ -8,12 +8,12 @@
## Develop ## Develop
### Install [nvm](https://github.com/nvm-sh/nvm) ### Set up Node.JS 25 with [nvm](https://github.com/nvm-sh/nvm)
- `curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash` - `curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash`
- `nvm install 22` - `nvm install 25`
- `nvm use 22` - `nvm use 25`
- Optionally set the system-wide default: `nvm alias default 22` - Optionally, set the system-wide default: `nvm alias default 25`
### Set up Rust ### Set up Rust

7
docs/.cspell.json
View file

@ -2,12 +2,7 @@
"version": "0.2", "version": "0.2",
"language": "en-GB", "language": "en-GB",
"dictionaries": ["en-gb"], "dictionaries": ["en-gb"],
"ignorePaths": [ "ignorePaths": ["node_modules", ".vitepress/dist", ".vitepress/cache", "package-lock.json"],
"node_modules",
".vitepress/dist",
".vitepress/cache",
"package-lock.json"
],
"words": [ "words": [
"VaultLink", "VaultLink",
"Obsidian", "Obsidian",

58
docs/architecture/data-flow.md
View file

@ -361,11 +361,11 @@ VALUES (?, ?, ?);
```json ```json
{ {
"type": "upload_file", "type": "upload_file",
"path": "notes/example.md", "path": "notes/example.md",
"content": "File content here...", "content": "File content here...",
"base_version": 10, "base_version": 10,
"timestamp": "2024-01-01T12:00:00Z" "timestamp": "2024-01-01T12:00:00Z"
} }
``` ```
@ -373,8 +373,8 @@ VALUES (?, ?, ?);
```json ```json
{ {
"type": "download_file", "type": "download_file",
"path": "notes/example.md" "path": "notes/example.md"
} }
``` ```
@ -382,8 +382,8 @@ VALUES (?, ?, ?);
```json ```json
{ {
"type": "delete_file", "type": "delete_file",
"path": "notes/old.md" "path": "notes/old.md"
} }
``` ```
@ -391,8 +391,8 @@ VALUES (?, ?, ?);
```json ```json
{ {
"type": "list_files", "type": "list_files",
"since_version": 0 "since_version": 0
} }
``` ```
@ -402,11 +402,11 @@ VALUES (?, ?, ?);
```json ```json
{ {
"type": "file_updated", "type": "file_updated",
"path": "notes/example.md", "path": "notes/example.md",
"version": 11, "version": 11,
"size": 1024, "size": 1024,
"hash": "abc123..." "hash": "abc123..."
} }
``` ```
@ -414,10 +414,10 @@ VALUES (?, ?, ?);
```json ```json
{ {
"type": "file_content", "type": "file_content",
"path": "notes/example.md", "path": "notes/example.md",
"content": "Updated content...", "content": "Updated content...",
"version": 11 "version": 11
} }
``` ```
@ -425,9 +425,9 @@ VALUES (?, ?, ?);
```json ```json
{ {
"type": "file_deleted", "type": "file_deleted",
"path": "notes/old.md", "path": "notes/old.md",
"version": 12 "version": 12
} }
``` ```
@ -435,9 +435,9 @@ VALUES (?, ?, ?);
```json ```json
{ {
"type": "sync_complete", "type": "sync_complete",
"total_files": 150, "total_files": 150,
"current_version": 200 "current_version": 200
} }
``` ```
@ -445,9 +445,9 @@ VALUES (?, ?, ?);
```json ```json
{ {
"type": "error", "type": "error",
"message": "File too large", "message": "File too large",
"code": "FILE_TOO_LARGE" "code": "FILE_TOO_LARGE"
} }
``` ```

6
docs/config/authentication.md
View file

@ -243,9 +243,9 @@ users:
2. Client sends authentication message: 2. Client sends authentication message:
```json ```json
{ {
"type": "auth", "type": "auth",
"token": "user-token", "token": "user-token",
"vault": "vault-name" "vault": "vault-name"
} }
``` ```
3. Server validates: 3. Server validates:

5960
docs/package-lock.json generated
View file

File diff suppressed because it is too large Load diff

36
frontend/local-client-cli/README.md
View file

@ -47,24 +47,24 @@ vaultlink \
### Required ### Required
| Option | Description | | Option | Description |
|--------|-------------| | ------------------------- | --------------------------------------------- |
| `-l, --local-path <path>` | Local directory to sync | | `-l, --local-path <path>` | Local directory to sync |
| `-r, --remote-uri <uri>` | Remote server WebSocket URI (ws:// or wss://) | | `-r, --remote-uri <uri>` | Remote server WebSocket URI (ws:// or wss://) |
| `-t, --token <token>` | Authentication token | | `-t, --token <token>` | Authentication token |
| `-v, --vault-name <name>` | Vault name on server | | `-v, --vault-name <name>` | Vault name on server |
### Optional ### Optional
| Option | Default | Description | | Option | Default | Description |
|--------|---------|-------------| | ------------------------------------ | ------- | -------------------------------------- |
| `--sync-concurrency <number>` | `1` | Concurrent sync operations | | `--sync-concurrency <number>` | `1` | Concurrent sync operations |
| `--max-file-size-mb <number>` | `10` | Maximum file size in MB | | `--max-file-size-mb <number>` | `10` | Maximum file size in MB |
| `--ignore-pattern <pattern>` | - | Glob pattern to ignore (repeatable) | | `--ignore-pattern <pattern>` | - | Glob pattern to ignore (repeatable) |
| `--websocket-retry-interval-ms <ms>` | `3500` | WebSocket reconnection interval | | `--websocket-retry-interval-ms <ms>` | `3500` | WebSocket reconnection interval |
| `--log-level <level>` | `INFO` | Log level: DEBUG, INFO, WARNING, ERROR | | `--log-level <level>` | `INFO` | Log level: DEBUG, INFO, WARNING, ERROR |
| `-h, --help` | - | Show help | | `-h, --help` | - | Show help |
| `-V, --version` | - | Show version | | `-V, --version` | - | Show version |
### Auto-Ignored Patterns ### Auto-Ignored Patterns
@ -74,11 +74,13 @@ vaultlink \
### Examples ### Examples
Basic usage: Basic usage:
```bash ```bash
vaultlink -l ./vault -r wss://sync.example.com -t token123 -v default vaultlink -l ./vault -r wss://sync.example.com -t token123 -v default
``` ```
With ignore patterns: With ignore patterns:
```bash ```bash
vaultlink -l ./vault -r wss://sync.example.com -t token123 -v default \ vaultlink -l ./vault -r wss://sync.example.com -t token123 -v default \
--ignore-pattern "*.tmp" \ --ignore-pattern "*.tmp" \
@ -87,6 +89,7 @@ vaultlink -l ./vault -r wss://sync.example.com -t token123 -v default \
``` ```
With debug logging: With debug logging:
```bash ```bash
vaultlink -l ./vault -r wss://sync.example.com -t token123 -v default \ vaultlink -l ./vault -r wss://sync.example.com -t token123 -v default \
--log-level DEBUG --log-level DEBUG
@ -176,6 +179,7 @@ services:
## Development ## Development
Build: Build:
```bash ```bash
npm run build npm run build
# or from the parent folder, run # or from the parent folder, run
@ -183,11 +187,13 @@ docker build -f local-client-cli/Dockerfile .
``` ```
Test: Test:
```bash ```bash
npm test npm test
``` ```
Docker build: Docker build:
```bash ```bash
cd frontend cd frontend
docker build -f local-client-cli/Dockerfile -t vault-link-cli:test . docker build -f local-client-cli/Dockerfile -t vault-link-cli:test .

4
frontend/local-client-cli/src/file-watcher.ts
View file

@ -106,8 +106,8 @@ export class FileWatcher {
} }
/** /**
* Convert a native platform path to forward slashes * Convert a native platform path to forward slashes
*/ */
private toUnixPath(nativePath: string): string { private toUnixPath(nativePath: string): string {
if (path.sep === "\\") { if (path.sep === "\\") {
return nativePath.replace(/\\/g, "/"); return nativePath.replace(/\\/g, "/");

8
frontend/local-client-cli/src/node-filesystem.ts
View file

@ -185,8 +185,8 @@ export class NodeFileSystemOperations implements FileSystemOperations {
} }
/** /**
* Convert a forward-slash path to native platform path separators * Convert a forward-slash path to native platform path separators
*/ */
private toNativePath(relativePath: string): string { private toNativePath(relativePath: string): string {
if (path.sep === "\\") { if (path.sep === "\\") {
return relativePath.replace(/\//g, "\\"); return relativePath.replace(/\//g, "\\");
@ -195,8 +195,8 @@ export class NodeFileSystemOperations implements FileSystemOperations {
} }
/** /**
* Convert a native platform path to forward slashes * Convert a native platform path to forward slashes
*/ */
private toUnixPath(nativePath: string): string { private toUnixPath(nativePath: string): string {
if (path.sep === "\\") { if (path.sep === "\\") {
return nativePath.replace(/\\/g, "/"); return nativePath.replace(/\\/g, "/");

4
frontend/local-client-cli/tsconfig.json
View file

@ -18,7 +18,5 @@
"declarationMap": true, "declarationMap": true,
"sourceMap": true "sourceMap": true
}, },
"exclude": [ "exclude": ["dist"]
"dist"
]
} }

26
frontend/obsidian-plugin/README.md
View file

@ -8,6 +8,7 @@ The repo depends on the latest plugin API (obsidian.d.ts) in TypeScript Definiti
**Note:** The Obsidian API is still in early alpha and is subject to change at any time! **Note:** The Obsidian API is still in early alpha and is subject to change at any time!
This sample plugin demonstrates some of the basic functionality the plugin API can do. This sample plugin demonstrates some of the basic functionality the plugin API can do.
- Adds a ribbon icon, which shows a Notice when clicked. - Adds a ribbon icon, which shows a Notice when clicked.
- Adds a command "Open Sample Modal" which opens a Modal. - Adds a command "Open Sample Modal" which opens a Modal.
- Adds a plugin setting tab to the settings page. - Adds a plugin setting tab to the settings page.
@ -57,31 +58,6 @@ Quick starting guide for new plugin devs:
- Copy over `main.js`, `styles.css`, `manifest.json` to your vault `VaultFolder/.obsidian/plugins/your-plugin-id/`. - Copy over `main.js`, `styles.css`, `manifest.json` to your vault `VaultFolder/.obsidian/plugins/your-plugin-id/`.
## Funding URL
You can include funding URLs where people who use your plugin can financially support it.
The simple way is to set the `fundingUrl` field to your link in your `manifest.json` file:
```json
{
"fundingUrl": "https://buymeacoffee.com"
}
```
If you have multiple URLs, you can also do:
```json
{
"fundingUrl": {
"Buy Me a Coffee": "https://buymeacoffee.com",
"GitHub Sponsor": "https://github.com/sponsors",
"Patreon": "https://www.patreon.com/"
}
}
```
## API Documentation ## API Documentation
See https://github.com/obsidianmd/obsidian-api See https://github.com/obsidianmd/obsidian-api

6
frontend/obsidian-plugin/src/vault-link-plugin.ts
View file

@ -135,9 +135,9 @@ export default class VaultLinkPlugin extends Plugin {
nativeLineEndings: Platform.isWin ? "\r\n" : "\n", nativeLineEndings: Platform.isWin ? "\r\n" : "\n",
...(IS_DEBUG_BUILD ...(IS_DEBUG_BUILD
? { ? {
fetch: debugging.slowFetchFactory(1), fetch: debugging.slowFetchFactory(1),
webSocket: debugging.slowWebSocketFactory(1, new Logger()) webSocket: debugging.slowWebSocketFactory(1, new Logger())
} }
: {}) : {})
}); });

5
frontend/obsidian-plugin/src/views/settings/settings-tab.ts
View file

@ -266,9 +266,8 @@ export class SyncSettingsTab extends PluginSettingTab {
new Notice("Checking connection to the server..."); new Notice("Checking connection to the server...");
new Notice( new Notice(
( (await this.syncClient.checkConnection())
await this.syncClient.checkConnection() .serverMessage
).serverMessage
); );
await this.statusDescription.updateConnectionState(); await this.statusDescription.updateConnectionState();
} else { } else {

9
frontend/obsidian-plugin/tsconfig.json
View file

@ -6,12 +6,7 @@
"strict": true, "strict": true,
"moduleResolution": "bundler", "moduleResolution": "bundler",
"allowSyntheticDefaultImports": true, "allowSyntheticDefaultImports": true,
"lib": [ "lib": ["DOM", "ES2024"]
"DOM",
"ES2024"
]
}, },
"exclude": [ "exclude": ["./dist"]
"./dist"
]
} }

2
frontend/obsidian-plugin/webpack.config.js
View file

@ -46,7 +46,7 @@ module.exports = (env, argv) => ({
const source = path.resolve(__dirname, "dist"); const source = path.resolve(__dirname, "dist");
const destinations = [ const destinations = [
"/volumes/syncthing/Desktop/test/test/.obsidian/plugins/vault-link", "/volumes/syncthing/Desktop/test/test/.obsidian/plugins/vault-link",
"/volumes/syncthing/Desktop/test/test2/.obsidian/plugins/vault-link", "/volumes/syncthing/Desktop/test/test2/.obsidian/plugins/vault-link"
// "/home/andras/obsidian-test/.obsidian/plugins/vault-link" // "/home/andras/obsidian-test/.obsidian/plugins/vault-link"
]; ];
destinations.forEach((destination) => { destinations.forEach((destination) => {

32
frontend/sync-client/src/file-operations/file-operations.ts
View file

@ -45,11 +45,11 @@ export class FileOperations {
} }
/** /**
* Create a file at the specified path. * Create a file at the specified path.
* *
* If a file with the same name already exists, it is moved before creating the new one. * If a file with the same name already exists, it is moved before creating the new one.
* Parent directories are created if necessary. * Parent directories are created if necessary.
*/ */
public async create( public async create(
path: RelativePath, path: RelativePath,
newContent: Uint8Array newContent: Uint8Array
@ -77,11 +77,11 @@ export class FileOperations {
} }
/** /**
* Update the file at the given path. * Update the file at the given path.
* *
* Performs a 3-way merge before writing if the file's content differs from `expectedContent`. * Performs a 3-way merge before writing if the file's content differs from `expectedContent`.
* Does not recreate the file if it no longer exists, returning an empty array instead. * Does not recreate the file if it no longer exists, returning an empty array instead.
*/ */
public async write( public async write(
path: RelativePath, path: RelativePath,
expectedContent: Uint8Array, expectedContent: Uint8Array,
@ -239,12 +239,12 @@ export class FileOperations {
} }
/** /**
* Deconflicts the given path by appending (1), (2), etc. before the file extension until a non-existent path is found. * Deconflicts the given path by appending (1), (2), etc. before the file extension until a non-existent path is found.
* The returned path has a lock acquired on it; it must be released by the caller when no longer needed. * The returned path has a lock acquired on it; it must be released by the caller when no longer needed.
* *
* @param path The starting path to deconflict * @param path The starting path to deconflict
* @returns a non-existent path with a lock acquired on it * @returns a non-existent path with a lock acquired on it
*/ */
private async deconflictPath(path: RelativePath): Promise<RelativePath> { private async deconflictPath(path: RelativePath): Promise<RelativePath> {
// eslint-disable-next-line prefer-const // eslint-disable-next-line prefer-const
let [directory, fileName] = FileOperations.getParentDirAndFile(path); let [directory, fileName] = FileOperations.getParentDirAndFile(path);

8
frontend/sync-client/src/file-operations/safe-filesystem-operations.ts
View file

@ -135,10 +135,10 @@ export class SafeFileSystemOperations implements FileSystemOperations {
} }
/** /**
* Decorate an operation to ensure that the file exists before running it. * Decorate an operation to ensure that the file exists before running it.
* If the operation fails, it will check if the file still exists and throw * If the operation fails, it will check if the file still exists and throw
* a FileNotFoundError if it doesn't. * a FileNotFoundError if it doesn't.
*/ */
private async safeOperation<T>( private async safeOperation<T>(
path: RelativePath, path: RelativePath,
operation: () => Promise<T>, operation: () => Promise<T>,

2
frontend/sync-client/src/persistence/database.ts
View file

@ -114,7 +114,7 @@ export class Database {
i === 0 i === 0
? false ? false
: records[i - 1].parallelVersion === : records[i - 1].parallelVersion ===
current.parallelVersion current.parallelVersion
) )
) { ) {
throw new Error( throw new Error(

52
frontend/sync-client/src/services/fetch-controller.ts
View file

@ -25,18 +25,18 @@ export class FetchController {
} }
/** /**
* Whether the fetch implementation can immediately send requests once outside of a reset. * Whether the fetch implementation can immediately send requests once outside of a reset.
*/ */
public get canFetch(): boolean { public get canFetch(): boolean {
return this._canFetch; return this._canFetch;
} }
/** /**
* Allow or disallow fetching. The changes only take effect if not resetting. * Allow or disallow fetching. The changes only take effect if not resetting.
* When called during a reset, its effect is deferred until the reset is finished. * When called during a reset, its effect is deferred until the reset is finished.
* *
* @param canFetch Whether fetching is enabled * @param canFetch Whether fetching is enabled
*/ */
public set canFetch(canFetch: boolean) { public set canFetch(canFetch: boolean) {
this._canFetch = canFetch; this._canFetch = canFetch;
@ -59,9 +59,9 @@ export class FetchController {
} }
/** /**
* Starts a reset, causing all ongoing and future fetches to be rejected * Starts a reset, causing all ongoing and future fetches to be rejected
* with a SyncResetError until finishReset is called. * with a SyncResetError until finishReset is called.
*/ */
public startReset(): void { public startReset(): void {
this.isResetting = true; this.isResetting = true;
this.rejectUntil(new SyncResetError()); this.rejectUntil(new SyncResetError());
@ -72,9 +72,9 @@ export class FetchController {
} }
/** /**
* Finishes a reset, allowing fetches to proceed or wait again depending on * Finishes a reset, allowing fetches to proceed or wait again depending on
* the current sync settings. * the current sync settings.
*/ */
public finishReset(): void { public finishReset(): void {
if (!this.isResetting) { if (!this.isResetting) {
return; return;
@ -85,19 +85,19 @@ export class FetchController {
} }
/** /**
* *
* |------------------|---------------|-----------------------------------------------------| * |------------------|---------------|-----------------------------------------------------|
* | | Sync enabled | Sync disabled | * | | Sync enabled | Sync disabled |
* |------------------|-------------- |-----------------------------------------------------| * |------------------|-------------- |-----------------------------------------------------|
* | During reset | Rejects with SyncResetError without sending request | * | During reset | Rejects with SyncResetError without sending request |
* |------------------|-------------- |-----------------------------------------------------| * |------------------|-------------- |-----------------------------------------------------|
* | Outside of reset | Same as fetch | Blocks until sync is enabled and then same as fetch | * | Outside of reset | Same as fetch | Blocks until sync is enabled and then same as fetch |
* |------------------|---------------|-----------------------------------------------------| * |------------------|---------------|-----------------------------------------------------|
* *
* @param logger for errors * @param logger for errors
* @param fetch to wrap * @param fetch to wrap
* @returns a wrapped fetch implementation affected by the FetchController state * @returns a wrapped fetch implementation affected by the FetchController state
*/ */
public getControlledFetchImplementation( public getControlledFetchImplementation(
logger: Logger, logger: Logger,
fetch: typeof globalThis.fetch = globalThis.fetch fetch: typeof globalThis.fetch = globalThis.fetch

10
frontend/sync-client/src/services/types/CreateDocumentVersion.ts
View file

@ -2,11 +2,11 @@
export interface CreateDocumentVersion { export interface CreateDocumentVersion {
/** /**
* The client can decide the document id (if it wishes to) in order * The client can decide the document id (if it wishes to) in order
* to help with syncing. If the client does not provide a document id, * to help with syncing. If the client does not provide a document id,
* the server will generate one. If the client provides a document id * the server will generate one. If the client provides a document id
* it must not already exist in the database. * it must not already exist in the database.
*/ */
document_id: string | null; document_id: string | null;
relative_path: string; relative_path: string;
content: number[]; content: number[];

4
frontend/sync-client/src/services/types/FetchLatestDocumentsResponse.ts
View file

@ -7,7 +7,7 @@ import type { DocumentVersionWithoutContent } from "./DocumentVersionWithoutCont
export interface FetchLatestDocumentsResponse { export interface FetchLatestDocumentsResponse {
latestDocuments: DocumentVersionWithoutContent[]; latestDocuments: DocumentVersionWithoutContent[];
/** /**
* The update ID of the latest document in the response. * The update ID of the latest document in the response.
*/ */
lastUpdateId: bigint; lastUpdateId: bigint;
} }

20
frontend/sync-client/src/services/types/PingResponse.ts
View file

@ -5,21 +5,21 @@
*/ */
export interface PingResponse { export interface PingResponse {
/** /**
* Semantic version of the server. * Semantic version of the server.
*/ */
serverVersion: string; serverVersion: string;
/** /**
* Whether the client is authenticated based on the sent Authorization * Whether the client is authenticated based on the sent Authorization
* header. * header.
*/ */
isAuthenticated: boolean; isAuthenticated: boolean;
/** /**
* List of file extensions that are allowed to be merged. * List of file extensions that are allowed to be merged.
*/ */
mergeableFileExtensions: string[]; mergeableFileExtensions: string[];
/** /**
* API version ensuring backwards & forwards compatibility between the client * API version ensuring backwards & forwards compatibility between the client
* and server. * and server.
*/ */
supportedApiVersion: number; supportedApiVersion: number;
} }

22
frontend/sync-client/src/sync-client.ts
View file

@ -285,10 +285,10 @@ export class SyncClient {
} }
/** /**
* Reload settings from disk overriding current in-memory settings. * Reload settings from disk overriding current in-memory settings.
* Missing values will be filled in from DEFAULT_SETTINGS rather than * Missing values will be filled in from DEFAULT_SETTINGS rather than
* retaining current in-memory settings. * retaining current in-memory settings.
*/ */
public async reloadSettings(): Promise<void> { public async reloadSettings(): Promise<void> {
this.checkIfDestroyed("reloadSettings"); this.checkIfDestroyed("reloadSettings");
@ -320,10 +320,10 @@ export class SyncClient {
} }
/** /**
* Wait for the in-flight operations to finish, reset all tracking, * Wait for the in-flight operations to finish, reset all tracking,
* and the local database but retain the settings. * and the local database but retain the settings.
* The SyncClient can be used again after calling this method. * The SyncClient can be used again after calling this method.
*/ */
public async reset(): Promise<void> { public async reset(): Promise<void> {
this.checkIfDestroyed("reset"); this.checkIfDestroyed("reset");
@ -436,9 +436,9 @@ export class SyncClient {
} }
/** /**
* Completely destroy the SyncClient, cancelling all in-progress operations. * Completely destroy the SyncClient, cancelling all in-progress operations.
* After calling this method, the SyncClient cannot be used again. * After calling this method, the SyncClient cannot be used again.
*/ */
public async destroy(): Promise<void> { public async destroy(): Promise<void> {
this.checkIfDestroyed("destroy"); this.checkIfDestroyed("destroy");

8
frontend/sync-client/src/sync-operations/syncer.ts
View file

@ -484,10 +484,10 @@ export class Syncer {
} }
/** /**
* Create fake documents in the database for all files that are present locally * Create fake documents in the database for all files that are present locally
* and also exist remotely. This will stop the subequent syncs from duplicating * and also exist remotely. This will stop the subequent syncs from duplicating
* the documents by creating the same documents from multiple clients. * the documents by creating the same documents from multiple clients.
*/ */
private async createFakeDocumentsFromRemoteState(): Promise<void> { private async createFakeDocumentsFromRemoteState(): Promise<void> {
if (this.database.getHasInitialSyncCompleted()) { if (this.database.getHasInitialSyncCompleted()) {
return; return;

58
frontend/sync-client/src/sync-operations/unrestricted-syncer.ts
View file

@ -170,14 +170,14 @@ export class UnrestrictedSyncer {
const updateDetails: SyncUpdateDetails | SyncMovedDetails = const updateDetails: SyncUpdateDetails | SyncMovedDetails =
oldPath !== undefined oldPath !== undefined
? { ? {
type: SyncType.MOVE, type: SyncType.MOVE,
relativePath: document.relativePath, relativePath: document.relativePath,
movedFrom: oldPath movedFrom: oldPath
} }
: { : {
type: SyncType.UPDATE, type: SyncType.UPDATE,
relativePath: document.relativePath relativePath: document.relativePath
}; };
await this.executeSync(updateDetails, async () => { await this.executeSync(updateDetails, async () => {
const originalRelativePath = document.relativePath; const originalRelativePath = document.relativePath;
@ -216,22 +216,22 @@ export class UnrestrictedSyncer {
response = response =
isText && cachedVersion !== undefined isText && cachedVersion !== undefined
? await this.syncService.putText({ ? await this.syncService.putText({
documentId: document.documentId, documentId: document.documentId,
parentVersionId: parentVersionId:
document.metadata.parentVersionId, document.metadata.parentVersionId,
relativePath: document.relativePath, relativePath: document.relativePath,
content: diff( content: diff(
new TextDecoder().decode(cachedVersion), new TextDecoder().decode(cachedVersion),
new TextDecoder().decode(contentBytes) new TextDecoder().decode(contentBytes)
) )
}) })
: await this.syncService.putBinary({ : await this.syncService.putBinary({
documentId: document.documentId, documentId: document.documentId,
parentVersionId: parentVersionId:
document.metadata.parentVersionId, document.metadata.parentVersionId,
relativePath: document.relativePath, relativePath: document.relativePath,
contentBytes contentBytes
}); });
} else { } else {
if (!force) { if (!force) {
this.logger.debug( this.logger.debug(
@ -336,14 +336,14 @@ export class UnrestrictedSyncer {
oldPath !== undefined || oldPath !== undefined ||
response.relativePath != originalRelativePath response.relativePath != originalRelativePath
? { ? {
type: SyncType.MOVE, type: SyncType.MOVE,
relativePath: response.relativePath, relativePath: response.relativePath,
movedFrom: originalRelativePath movedFrom: originalRelativePath
} }
: { : {
type: SyncType.UPDATE, type: SyncType.UPDATE,
relativePath: response.relativePath relativePath: response.relativePath
}; };
if (areThereLocalChanges) { if (areThereLocalChanges) {
this.history.addHistoryEntry({ this.history.addHistoryEntry({

10
frontend/sync-client/src/tracing/sync-history.ts
View file

@ -88,11 +88,11 @@ export class SyncHistory {
} }
/** /**
* Insert the entry at the beginning of the history list. If the entry * Insert the entry at the beginning of the history list. If the entry
* already in the list, it will get moved to the beginning and updated. * already in the list, it will get moved to the beginning and updated.
* *
* If the entry list is too long, the oldest entry will be removed. * If the entry list is too long, the oldest entry will be removed.
*/ */
public addHistoryEntry(entry: CommonHistoryEntry): void { public addHistoryEntry(entry: CommonHistoryEntry): void {
const historyEntry = { const historyEntry = {
...entry, ...entry,

4
frontend/sync-client/src/utils/create-client-id.ts
View file

@ -8,8 +8,8 @@ export function createClientId(): string {
typeof navigator !== "undefined" typeof navigator !== "undefined"
? navigator.platform // eslint-disable-line @typescript-eslint/no-deprecated ? navigator.platform // eslint-disable-line @typescript-eslint/no-deprecated
: typeof process !== "undefined" : typeof process !== "undefined"
? process.platform ? process.platform
: "unknown"; : "unknown";
return `vault-link/${packageVersion} (${uuidv4()}; ${platform})`; return `vault-link/${packageVersion} (${uuidv4()}; ${platform})`;
} }

42
frontend/sync-client/src/utils/data-structures/event-listeners.ts
View file

@ -13,32 +13,32 @@ export class EventListeners<TListener extends (...args: any[]) => any> {
} }
/** /**
* Adds a new listener to the collection. * Adds a new listener to the collection.
* *
* @param listener The listener callback to add * @param listener The listener callback to add
* @returns An unsubscribe function that removes this listener when called * @returns An unsubscribe function that removes this listener when called
*/ */
public add(listener: TListener): () => void { public add(listener: TListener): () => void {
this.listeners.push(listener); this.listeners.push(listener);
return () => this.remove(listener); return () => this.remove(listener);
} }
/** /**
* Removes a listener from the collection. * Removes a listener from the collection.
* *
* @param listener The listener callback to remove * @param listener The listener callback to remove
* @returns true if the listener was found and removed, false otherwise * @returns true if the listener was found and removed, false otherwise
*/ */
public remove(listener: TListener): boolean { public remove(listener: TListener): boolean {
return removeFromArray(this.listeners, listener); return removeFromArray(this.listeners, listener);
} }
/** /**
* Triggers all listeners synchronously with the provided arguments. * Triggers all listeners synchronously with the provided arguments.
* Any returned promises are ignored. Use triggerAsync() to await them. * Any returned promises are ignored. Use triggerAsync() to await them.
* *
* @param args The arguments to pass to each listener * @param args The arguments to pass to each listener
*/ */
public trigger(...args: Parameters<TListener>): void { public trigger(...args: Parameters<TListener>): void {
this.listeners.forEach((listener) => { this.listeners.forEach((listener) => {
listener(...args); listener(...args);
@ -46,12 +46,12 @@ export class EventListeners<TListener extends (...args: any[]) => any> {
} }
/** /**
* Triggers all listeners and awaits any promises they return. * Triggers all listeners and awaits any promises they return.
* Synchronous listeners are called immediately, and any async listeners * Synchronous listeners are called immediately, and any async listeners
* are awaited in parallel. * are awaited in parallel.
* *
* @param args The arguments to pass to each listener * @param args The arguments to pass to each listener
*/ */
public async triggerAsync(...args: Parameters<TListener>): Promise<void> { public async triggerAsync(...args: Parameters<TListener>): Promise<void> {
await awaitAll( await awaitAll(
this.listeners this.listeners

92
frontend/sync-client/src/utils/data-structures/locks.ts
View file

@ -21,34 +21,34 @@ export class Locks<T> {
public constructor(private readonly logger?: Logger) {} public constructor(private readonly logger?: Logger) {}
/** /**
* Executes a function while holding exclusive locks on one or more keys. * Executes a function while holding exclusive locks on one or more keys.
* *
* This method ensures that the provided function runs with exclusive access to the * This method ensures that the provided function runs with exclusive access to the
* specified key(s). Multiple keys are sorted to prevent deadlocks when different * specified key(s). Multiple keys are sorted to prevent deadlocks when different
* operations request the same keys in different orders. * operations request the same keys in different orders.
* *
* @template R The return type of the function to execute * @template R The return type of the function to execute
* @param keyOrKeys A single key or array of keys to lock during function execution * @param keyOrKeys A single key or array of keys to lock during function execution
* @param fn The function to execute while holding the lock(s). Can be sync or async. * @param fn The function to execute while holding the lock(s). Can be sync or async.
* @returns A Promise that resolves to the return value of the executed function * @returns A Promise that resolves to the return value of the executed function
* *
* @example * @example
* ```typescript * ```typescript
* // Lock a single key * // Lock a single key
* const result = await locks.withLock('file1', () => { * const result = await locks.withLock('file1', () => {
* // Critical section - only one operation can access 'file1' at a time * // Critical section - only one operation can access 'file1' at a time
* return processFile('file1'); * return processFile('file1');
* }); * });
* *
* // Lock multiple keys (prevents deadlocks through consistent ordering) * // Lock multiple keys (prevents deadlocks through consistent ordering)
* await locks.withLock(['file1', 'file2'], async () => { * await locks.withLock(['file1', 'file2'], async () => {
* // Critical section - exclusive access to both files * // Critical section - exclusive access to both files
* await moveFile('file1', 'file2'); * await moveFile('file1', 'file2');
* }); * });
* ``` * ```
* *
* @throws Any error thrown by the provided function will be propagated after locks are released * @throws Any error thrown by the provided function will be propagated after locks are released
*/ */
public async withLock<R>( public async withLock<R>(
keyOrKeys: T | T[], keyOrKeys: T | T[],
fn: () => R | Promise<R> fn: () => R | Promise<R>
@ -83,12 +83,12 @@ export class Locks<T> {
} }
/** /**
* Attempts to acquire a lock immediately without waiting. * Attempts to acquire a lock immediately without waiting.
* Must call `unlock()` if successful. * Must call `unlock()` if successful.
* *
* @param key The key to lock * @param key The key to lock
* @returns `true` if lock acquired, `false` if already locked * @returns `true` if lock acquired, `false` if already locked
*/ */
public tryLock(key: T): boolean { public tryLock(key: T): boolean {
if (this.locked.has(key)) { if (this.locked.has(key)) {
return false; return false;
@ -100,12 +100,12 @@ export class Locks<T> {
} }
/** /**
* Waits to acquire a lock, blocking until available. * Waits to acquire a lock, blocking until available.
* Operations are queued in FIFO order. Must call `unlock()` when done. * Operations are queued in FIFO order. Must call `unlock()` when done.
* *
* @param key The key to wait for and lock * @param key The key to wait for and lock
* @returns Promise that resolves when lock is acquired * @returns Promise that resolves when lock is acquired
*/ */
public async waitForLock(key: T): Promise<void> { public async waitForLock(key: T): Promise<void> {
if (this.tryLock(key)) { if (this.tryLock(key)) {
return Promise.resolve(); return Promise.resolve();
@ -126,12 +126,12 @@ export class Locks<T> {
} }
/** /**
* Releases a lock and grants access to the next waiting operation in FIFO order. * Releases a lock and grants access to the next waiting operation in FIFO order.
* Removes the key from locked set if no waiters. * Removes the key from locked set if no waiters.
* *
* @param key The key to unlock * @param key The key to unlock
* @throws {Error} If key is not currently locked * @throws {Error} If key is not currently locked
*/ */
public unlock(key: T): void { public unlock(key: T): void {
if (!this.locked.has(key)) { if (!this.locked.has(key)) {
return; return;

4
frontend/sync-client/tsconfig.json
View file

@ -12,7 +12,5 @@
"declaration": true, "declaration": true,
"declarationDir": "./dist/types" "declarationDir": "./dist/types"
}, },
"exclude": [ "exclude": ["./dist"]
"./dist"
]
} }

9
frontend/test-client/tsconfig.json
View file

@ -5,13 +5,8 @@
"target": "ES2022", "target": "ES2022",
"module": "CommonJS", "module": "CommonJS",
"esModuleInterop": true, "esModuleInterop": true,
"lib": [ "lib": ["DOM", "ES2024"],
"DOM",
"ES2024",
],
"moduleResolution": "node" "moduleResolution": "node"
}, },
"exclude": [ "exclude": ["./dist"]
"./dist"
]
} }

32
sync-server/config-e2e.yml
View file

@ -9,24 +9,24 @@ server:
max_clients_per_vault: 256 max_clients_per_vault: 256
response_timeout: 30m response_timeout: 30m
mergeable_file_extensions: mergeable_file_extensions:
- md - md
- txt - txt
users: users:
user_configs: user_configs:
- name: admin - name: admin
token: test-token-change-me token: test-token-change-me
vault_access: vault_access:
type: allow_access_to_all type: allow_access_to_all
- name: other-admin - name: other-admin
token: test-token-change-me2 token: test-token-change-me2
vault_access: vault_access:
type: allow_access_to_all type: allow_access_to_all
- name: test - name: test
token: other-test-token token: other-test-token
vault_access: vault_access:
type: allow_list type: allow_list
allowed: allowed:
- default - default
logging: logging:
log_directory: logs log_directory: logs
log_rotation: 7days log_rotation: 7days