reconcile/README.md

Reconcile: conflict-free 3-way text merging

diff3 but with automatic conflict resolution.

TODO: add links for crates and npm

Reconcile is a Rust and JavaScript (through WebAssembly) library for merging text without user intervention. It automatically resolves conflicts that would typically require user action in traditional 3-way merge tools.

Features

• Conflict-free output - No more git conflict markers in the result
• Cursor/selection position tracking - Automatically updates cursor positions during merging
• Pluggable tokenizer - Choose between word-level, character-level, or custom tokenization
• Full UTF-8 support - Handles Unicode text correctly
• WebAssembly support - Use from JavaScript/TypeScript applications

Quick Start

Rust

Add to your Cargo.toml:

[dependencies]
reconcile = "0.4"

use reconcile::{reconcile, BuiltinTokenizer};

let parent = "Hello world";
let left = "Hello beautiful world";
let right = "Hi world";

let result = reconcile(parent, &left.into(), &right.into(), &*BuiltinTokenizer::Word);
assert_eq!(result.apply().text(), "Hi beautiful world");

JavaScript/TypeScript

npm install reconcile

import { init, reconcile } from "reconcile";

// Initialize the WASM module (required before first use)
await init();

const parent = "Hello world";
const left = "Hello beautiful world";
const right = "Hi world";

const result = reconcile(parent, left, right);
console.log(result.text); // "Hi beautiful world"

API

Tokenizers

Reconcile supports different tokenization strategies:

• Word tokenizer (BuiltinTokenizer::Word): Splits text into words (default, recommended for most use cases)
• Character tokenizer (BuiltinTokenizer::Character): Splits text into individual characters (fine-grained merging)
• Custom tokenizer: Implement your own tokenization logic

Cursor Tracking

Reconcile can automatically update cursor and selection positions during merging:

const result = reconcile(
    "Hello world",
    {
        text: "Hello beautiful world",
        cursors: [{ id: 1, position: 6 }], // After "Hello "
    },
    {
        text: "Hi world",
        cursors: [{ id: 2, position: 0 }], // At beginning
    }
);

// Result includes updated cursor positions
console.log(result.cursors); // [{ id: 1, position: 3 }, { id: 2, position: 0 }]

History Tracking

Use reconcileWithHistory to get detailed information about the merge process:

const result = reconcileWithHistory(parent, left, right);
console.log(result.history); // Array of spans with their origins

Algorithm

The algorithm starts similarly to diff3. Its inputs are a parent document and two conflicting versions: left and right which have been created from the parent through any series of concurrent edits.

1. Diff calculation: First, 2-way diffs of (parent & left) and (parent & right) are computed using Myers' algorithm
2. Tokenization: The text is split into tokens (words, characters, etc.) for granular merging
3. Diff cleaning: The tokens of the same diff are reordered and merged to end up to maximise patch sizes
4. Operation transformation (OT): The resulting edits are weaved together using operational transformation principles, ensuring no changes are lost

EditedText (at least in the Rust library) exposes an implementation of OT. The primary purpose of this library isn't to implement OT but to provide automated text merging, howver, OT happens to provide an easy way of merging the output of Myers' diff. The same result could be achieved through many CRDT implementations as well. However, the merging quality is only as good as the 2-way diffs are. For instance, reconcile doesn't support move semantics as these are decomposed into an insert and delete operation by Myers'.

Motivation

Sometimes documents get edited concurrently by multiple users (or the same user from multiple devices) resulting in divergent changes.

To allow for offline editing, we could use CRDTs or Operational Transformation (OT) to come to a consistent resolution of the competing version. However, this requires capturing all user actions: insertions, deletes, move, copies, and pastes. In some applications, this is trivial if the document can only be edited through an editor that's in our control. But this isn't always the case. Users enjoy composable systems that don't lock them in. For example, one of the unique selling points of Obsidian is to provide an editor experience over a folder of Markdown files leaving the user free to change their technology of choice on a whim.

This means that files can be edited out-of-channel and the only information a text synchronization system can know is the current content of each tracked file. This is described as Differential Synchronization 1. This is the same problem as what Git and similar version control systems solve but in a manual way. Although the problem is similar, there's a relevant difference between syncing source code and personal notes: in the case of the former, a semantically incorrect conflict resolution can wreak havoc in a code base, or worse, introduce a correctness bug unnoticed. Text notes are different though, humans are well-equipped to finding the signal in a noisy environment and "bad merges" might result in a clumsy sentence but the reader will likely still understand the gist and can fix it if necessary.

There are domains of human text which are less tolerant of mis-merges: for instance, two conflicting changes to a contract could result in a term getting negated in different ways from both sides, resulting in a double-negation, thus unknowingly changing the meaning.

Development

Prerequisites

Install Node.js

• Install nvm: curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash
• nvm install 22
• nvm use 22
• Optionally set the system-wide default: nvm alias default 22

Set up Rust

• Install rustup: curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
• cargo install wasm-pack cargo-insta cargo-edit

Scripts

• Running tests: scripts/test.sh
• Formatting: scripts/lint.sh
• Building website: scripts/dev-website.sh
• Publishing new version: scripts/bump-version.sh patch

TODO: license