andras/reconcile
andras/reconcile
1
0
Fork 0
Code Issues Pull requests 6 Projects Releases Packages Wiki Activity Actions

Merge branch 'asch/better-api'

Browse source
This commit is contained in:
Andras Schmelczer 2025-07-06 22:47:58 +01:00
commit fb3c14b6ef
No known key found for this signature in database
GPG key ID: FC8F2C3D3D1A718C
32 changed files with 6334 additions and 6241 deletions
Download patch file Download diff file Expand all files Collapse all files

41
.github/workflows/check.yml vendored
View file

@ -43,3 +43,44 @@ jobs:
cargo test --features serde cargo test --features serde
cargo test --features wasm cargo test --features wasm
wasm-pack test --node --features wasm wasm-pack test --node --features wasm
publish:
needs: build
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v4
- name: Setup Node.js environment
uses: actions/setup-node@v4.2.0
with:
node-version: "22.x"
check-latest: true
registry-url: 'https://registry.npmjs.org'
- name: Setup rust
run: |
cargo install wasm-pack
- name: Build wasm
run: |
wasm-pack build --target web --features wasm
- name: Publish to crates.io
run: cargo publish --token ${{ secrets.CRATES_TOKEN }}
continue-on-error: true
- name: Build reconcile-js
run: |
cd reconcile-js
npm ci
npm run build
- name: Publish reconcile-js to NPM
run: |
cd reconcile-js
npm publish
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
continue-on-error: true

12
.github/workflows/gh-pages.yml vendored
View file

@ -15,7 +15,7 @@ permissions:
# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued. # Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete. # However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
concurrency: concurrency:
group: "pages" group: 'pages'
cancel-in-progress: false cancel-in-progress: false
jobs: jobs:
@ -31,13 +31,17 @@ jobs:
run: | run: |
cargo install wasm-pack cargo install wasm-pack
wasm-pack build --target web --features wasm wasm-pack build --target web --features wasm
cp -R pkg/reconcile.js examples/website/ cd reconcile-js
cp -R pkg/reconcile_bg.wasm examples/website/ npm ci
npm run build
cd ../examples/website
npm ci
npm run build
- name: Upload artifact - name: Upload artifact
uses: actions/upload-pages-artifact@v3 uses: actions/upload-pages-artifact@v3
with: with:
path: examples/website path: examples/website/dist
deploy: deploy:
environment: environment:

5
.prettierrc
View file

@ -3,8 +3,5 @@
"printWidth": 90, "printWidth": 90,
"tabWidth": 2, "tabWidth": 2,
"singleQuote": true, "singleQuote": true,
"endOfLine": "lf", "endOfLine": "lf"
"importOrder": ["^[./]", ".*", ".scss$"],
"importOrderSeparation": true,
"importOrderSortSpecifiers": true
} }

180
README.md
View file

@ -3,148 +3,172 @@
[![Check](https://github.com/schmelczer/reconcile/actions/workflows/check.yml/badge.svg)](https://github.com/schmelczer/reconcile/actions/workflows/check.yml) [![Check](https://github.com/schmelczer/reconcile/actions/workflows/check.yml/badge.svg)](https://github.com/schmelczer/reconcile/actions/workflows/check.yml)
[![Publish to GitHub Pages](https://github.com/schmelczer/reconcile/actions/workflows/gh-pages.yml/badge.svg)](https://github.com/schmelczer/reconcile/actions/workflows/gh-pages.yml) [![Publish to GitHub Pages](https://github.com/schmelczer/reconcile/actions/workflows/gh-pages.yml/badge.svg)](https://github.com/schmelczer/reconcile/actions/workflows/gh-pages.yml)
> [`diff3`](https://www.gnu.org/software/diffutils/manual/html_node/Invoking-diff3.html) (or `git merge`) but with automatic conflict resolution. > Think [`diff3`](https://www.gnu.org/software/diffutils/manual/html_node/Invoking-diff3.html) or `git merge`, but with intelligent conflict resolution that just works.
Reconcile is a Rust and JavaScript (through WebAssembly) library for merging text without user intervention. It automatically resolves conflicts that would typically require user action in traditional 3-way merge tools. Reconcile is a Rust and JavaScript (via WebAssembly) library that merges conflicting text edits without requiring manual intervention. Where traditional 3-way merge tools would leave you with conflict markers to resolve by hand, Reconcile automatically weaves changes together using sophisticated algorithms inspired by Operational Transformation.
Try out the [interactive demo](https://schmelczer.dev/reconcile)! ✨ **[Try the interactive demo](https://schmelczer.dev/reconcile)** to see it in action!
TODO: add links for crates and npm TODO: add links for crates and npm
## Features ## What makes Reconcile special?
- **Conflict-free output** - No more git conflict markers in the result - **🚫 No conflict markers** — Clean, merged output without Git's `<<<<<<<` noise
- **Cursor/selection position tracking** - Automatically updates cursor positions during merging - **📍 Cursor tracking** — Automatically repositions cursors and selections during merging
- **Pluggable tokenizer** - Choose between word-level, character-level, or custom tokenization - **🔧 Flexible tokenisation** — Word-level (default), character-level, or custom strategies
- **Full UTF-8 support** - Handles Unicode text correctly - **🌍 Unicode-first** — Full UTF-8 support
- **WebAssembly support** - Use from JavaScript/TypeScript applications - **🕸️ Cross-platform** — Native Rust performance with WebAssembly for JavaScript
## Quick Start ## Quick start
### Rust ### Rust
Add to your `Cargo.toml`: Add `reconcile` to your `Cargo.toml`:
```toml ```toml
[dependencies] [dependencies]
reconcile = "0.4" reconcile = "0.4"
``` ```
Then merge away:
```rust ```rust
use reconcile::{reconcile, BuiltinTokenizer}; use reconcile::{reconcile, BuiltinTokenizer};
// Start with original text
let parent = "Hello world"; let parent = "Hello world";
let left = "Hello beautiful world"; // Two people edit simultaneously
let right = "Hi world"; let left = "Hello beautiful world"; // Added "beautiful"
let right = "Hi world"; // Changed "Hello" to "Hi"
// Reconcile combines both changes intelligently
let result = reconcile(parent, &left.into(), &right.into(), &*BuiltinTokenizer::Word); let result = reconcile(parent, &left.into(), &right.into(), &*BuiltinTokenizer::Word);
assert_eq!(result.apply().text(), "Hi beautiful world"); assert_eq!(result.apply().text(), "Hi beautiful world");
``` ```
### JavaScript/TypeScript ### JavaScript/TypeScript
Install via npm:
```bash ```bash
npm install reconcile npm install reconcile
``` ```
```javascript Then use in your application:
import { init, reconcile } from "reconcile";
// Initialize the WASM module (required before first use) ```javascript
import { init, reconcile } from 'reconcile';
// One-time setup: initialise the WASM module
await init(); await init();
const parent = "Hello world"; // Same example as above
const left = "Hello beautiful world"; const parent = 'Hello world';
const right = "Hi world"; const left = 'Hello beautiful world';
const right = 'Hi world';
const result = reconcile(parent, left, right); const result = reconcile(parent, left, right);
console.log(result.text); // "Hi beautiful world" console.log(result.text); // "Hi beautiful world"
``` ```
## API ## Advanced usage
### Tokenizers ### Edit provenance
Reconcile supports different tokenization strategies: Track which changes came from where using `reconcileWithHistory`:
- **Word tokenizer** (`BuiltinTokenizer::Word`): Splits text into words (default, recommended for most use cases)
- **Character tokenizer** (`BuiltinTokenizer::Character`): Splits text into individual characters (fine-grained merging)
- **Custom tokenizer**: Implement your own tokenization logic
### Cursor Tracking
Reconcile can automatically update cursor and selection positions during merging:
```javascript
const result = reconcile(
"Hello world",
{
text: "Hello beautiful world",
cursors: [{ id: 1, position: 6 }], // After "Hello "
},
{
text: "Hi world",
cursors: [{ id: 2, position: 0 }], // At beginning
}
);
// Result includes updated cursor positions
console.log(result.cursors); // [{ id: 1, position: 3 }, { id: 2, position: 0 }]
```
### History Tracking
Use `reconcileWithHistory` to get detailed information about the merge process:
```javascript ```javascript
const result = reconcileWithHistory(parent, left, right); const result = reconcileWithHistory(parent, left, right);
console.log(result.history); // Array of spans with their origins console.log(result.history); // Detailed breakdown of each text span's origin
``` ```
## Algorithm ### Tokenisation strategies
The algorithm starts similarly to `diff3`. Its inputs are a **parent** document and two conflicting versions: `left` and `right` which have been created from the parent through any series of concurrent edits. Reconcile offers different ways to split text for merging:
1. **Diff calculation**: First, 2-way diffs of (parent & left) and (parent & right) are computed using Myers' algorithm - **Word tokeniser** (`BuiltinTokenizer::Word`) — Splits on word boundaries (recommended for prose)
2. **Tokenization**: The text is split into tokens (words, characters, etc.) for granular merging - **Character tokeniser** (`BuiltinTokenizer::Character`) — Individual characters (fine-grained control)
3. **Diff cleaning**: The tokens of the same diff are reordered and merged to end up to maximise patch sizes - **Line tokeniser** (`BuiltinTokenizer::Line`) — Line-by-line (similar to `git merge`)
4. **Operation transformation (OT)**: The resulting edits are weaved together using operational transformation principles, ensuring no changes are lost - **Custom tokeniser** — Roll your own for specialised use cases
`EditedText` (at least in the Rust library) exposes an implementation of OT. The primary purpose of this library isn't to implement OT but to provide automated text merging, howver, OT happens to provide an easy way of merging the output of Myers' diff. The same result could be achieved through many CRDT implementations as well. However, the merging quality is only as good as the 2-way diffs are. For instance, `reconcile` doesn't support `move` semantics as these are decomposed into an `insert` and `delete` operation by Myers'. ### Cursor tracking
## Motivation Ideal for collaborative editors — Reconcile tracks cursor positions through merges:
Sometimes documents get edited concurrently by multiple users (or the same user from multiple devices) resulting in divergent changes. ```javascript
const result = reconcile(
'Hello world',
{
text: 'Hello beautiful world',
cursors: [{ id: 1, position: 6 }], // After "Hello "
},
{
text: 'Hi world',
cursors: [{ id: 2, position: 0 }], // At the beginning
}
);
To allow for offline editing, we could use CRDTs or Operational Transformation (OT) to come to a consistent resolution of the competing version. However, this requires capturing all user actions: insertions, deletes, move, copies, and pastes. In some applications, this is trivial if the document can only be edited through an editor that's in our control. But this isn't always the case. Users enjoy composable systems that don't lock them in. For example, one of the unique selling points of Obsidian is to provide an editor experience over a folder of Markdown files leaving the user free to change their technology of choice on a whim. // Cursors are automatically repositioned in the merged text
console.log(result.cursors); // [{ id: 1, position: 3 }, { id: 2, position: 0 }]
```
This means that files can be edited out-of-channel and the only information a text synchronization system can know is the current content of each tracked file. This is described as Differential Synchronization [1]. This is the same problem as what Git and similar version control systems solve but in a manual way. Although the problem is similar, there's a relevant difference between syncing source code and personal notes: in the case of the former, a semantically incorrect conflict resolution can wreak havoc in a code base, or worse, introduce a correctness bug unnoticed. Text notes are different though, humans are well-equipped to finding the signal in a noisy environment and "bad merges" might result in a clumsy sentence but the reader will likely still understand the gist and can fix it if necessary. ## How it works
> There are domains of human text which are less tolerant of mis-merges: for instance, two conflicting changes to a contract could result in a term getting negated in different ways from both sides, resulting in a double-negation, thus unknowingly changing the meaning. Reconcile builds upon the foundation of `diff3` but adds intelligent conflict resolution. Given a **parent** document and two modified versions (`left` and `right`), here's what happens:
1. **Diff computation** — Myers' algorithm calculates differences between (parent ↔ left) and (parent ↔ right)
2. **Tokenisation** — Text splits into meaningful units (words, characters, etc.) for granular merging
3. **Diff optimisation** — Operations are reordered and consolidated to maximise coherent changes
4. **Operational Transformation** — Edits are woven together using OT principles, preserving all modifications
Whilst Reconcile's primary goal isn't implementing Operational Transformation, OT provides an elegant way to merge Myers' diff output. The same could be achieved with CRDTs, though the quality depends entirely on the underlying 2-way diffs. Note that `move` operations aren't supported, as Myers' algorithm decomposes them into separate `insert` and `delete` operations.
## Why Reconcile exists
Collaborative editing is everywhere — multiple users editing documents simultaneously, or the same person working across devices. This creates the inevitable challenge of conflicting changes.
Traditional solutions like CRDTs or Operational Transformation work brilliantly when you control the entire editing environment. They capture every keystroke, cursor movement, and operation. But real-world workflows are messier: users love tools that don't lock them in. Take Obsidian's approach with plain Markdown files — users can edit with any tool they fancy, from Vim to Word.
This creates what's known as **Differential Synchronisation** [¹]: you only know the final state of each document, not how it got there. It's the same challenge Git tackles, but Git expects humans to resolve conflicts manually.
Here's the key insight: whilst incorrect merges in source code can introduce devastating bugs, human text is more forgiving. People excel at extracting meaning from imperfect text — a slightly clumsy sentence is preferable to conflict markers interrupting the flow.
> **Caveat**: Some text domains are less tolerant of imperfect merges. Legal contracts, for instance, could have unintended meaning changes from double-negations created by conflicting edits.
## Development ## Development
### Prerequisites ### Prerequisites
#### Install Node.js #### Node.js setup
- Install [nvm](https://github.com/nvm-sh/nvm): `curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash` 1. Install [nvm](https://github.com/nvm-sh/nvm):
- `nvm install 22` ```bash
- `nvm use 22` curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash
- Optionally set the system-wide default: `nvm alias default 22` ```
2. Install and use Node 22:
```bash
nvm install 22 && nvm use 22
```
3. Optionally set as default: `nvm alias default 22`
#### Set up Rust #### Rust toolchain
- Install [`rustup`](https://rustup.rs): `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` 1. Install [rustup](https://rustup.rs):
- `cargo install wasm-pack cargo-insta cargo-edit` ```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```
2. Install additional tools:
```bash
cargo install wasm-pack cargo-insta cargo-edit
```
### Scripts ### Development scripts
- **Running tests**: `scripts/test.sh` - **Run tests**: `scripts/test.sh`
- **Formatting**: `scripts/lint.sh` - **Lint and format**: `scripts/lint.sh`
- **Building website**: `scripts/dev-website.sh` - **Build demo website**: `scripts/dev-website.sh`
- **Publishing new version**: `scripts/bump-version.sh patch` - **Publish new version**: `scripts/bump-version.sh patch`
TODO: license TODO: license
[1]: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/35605.pdf [¹]: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/35605.pdf

105
examples/website/index.html
View file

@ -8,18 +8,18 @@
/> />
<meta <meta
name="description" name="description"
content="Easily merge three versions of a text document with this 3-way text merge tool." content="3-way text merging that automatically resolves conflicts. No more Git conflict markers — just clean, merged results."
/> />
<meta property="og:title" content="3-Way Text Merge" /> <meta property="og:title" content="3-Way Text Merge" />
<meta <meta
property="og:description" property="og:description"
content="Easily merge three versions of a text document with this 3-way text merge tool." content="3-way text merging that automatically resolves conflicts. No more Git conflict markers — just clean, merged results."
/> />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://github.com/schmelczer/reconcile" /> <meta property="og:url" content="https://schmelczer.dev/reconcile" />
<meta property="og:image" content="/favicon.ico" /> <meta property="og:image" content="/favicon.ico" />
<link rel="icon" type="image/x-icon" href="favicon.ico" /> <link rel="icon" type="image/x-icon" href="favicon.ico" />
<title>3-Way Text Merge</title> <title>Reconcile: conflict-free text merging</title>
<link inline inline-asset="index.css" inline-asset-delete /> <link inline inline-asset="index.css" inline-asset-delete />
</head> </head>
<body> <body>
@ -28,49 +28,58 @@
<div class="scroll-container"> <div class="scroll-container">
<div class="page-wrapper"> <div class="page-wrapper">
<header> <header>
<h1>Reconcile: automated 3-way text merge</h1> <h1>Reconcile: conflict-free 3-way text merging</h1>
<p> <p>
The Think
<a
href="https://github.com/schmelczer/reconcile"
target="_blank"
rel="noopener noreferrer"
>reconcile</a
>
library solves a fundamental challenge in collaborative editing: what happens
when multiple users edit the same text simultaneously but we can only capture
the end result, not the intermediary edits? Essentially, it's
<a <a
href="https://www.gnu.org/software/diffutils/manual/html_node/Invoking-diff3.html" href="https://www.gnu.org/software/diffutils/manual/html_node/Invoking-diff3.html"
target="_blank" target="_blank"
rel="noopener noreferrer" rel="noopener noreferrer"
>diff3</a >diff3</a
> >
(or <code>git merge</code>) but with automatic conflict resolution. or <code>git merge</code>, but with intelligent conflict resolution that
</p> requires no user intervention. The
<p> <a
The href="https://github.com/schmelczer/reconcile"
<code>reconcile(parent: str, left: str, right: str) -> str</code> target="_blank"
takes conflicting concurrent edits and intelligently merges them into a rel="noopener noreferrer"
unified result. Beyond basic conflict resolution, it offers sophisticated >Reconcile</a
merging heuristics, flexible tokenization options, and cursor position >
tracking. library tackles a fundamental challenge in collaborative editing: what happens
</p> when multiple users edit the same text simultaneously, but the conflict
<p> resolver only has access to the final results, not the intermediate steps?
The algorithm begins with your chosen tokenizer, then applies Myers' diff
algorithm to compare the original text with both conflicting versions. These
diffs undergo transformation to preserve meaningful change sequences, before a
final merge strategy—inspired by Operational Transformation reconciles all
conflicting modifications without losing any edits.
</p>
<p>
For more details, see the
<a href="https://github.com/schmelczer/reconcile" target="_blank">README</a>.
</p> </p>
<p> <p>
Use the tokenization options below to experiment with different strategies. Where traditional merge tools leave you with conflict markers to resolve
The library supports user-defined tokenizers as well. manually, Reconcile automatically weaves changes together. The
<code>reconcile(parent, left, right)</code> function takes conflicting edits
and produces clean, unified results using an algorithm inspired by Operational
Transformation. No more <code>&lt;&lt;&lt;&lt;&lt;&lt;&lt;</code> markers
cluttering your text.
</p>
<p>
The process starts with your chosen tokenisation strategy, then applies Myers'
diff algorithm to compare the original with both modified versions. These
diffs are optimised and transformed to preserve meaningful changes, before a
final merge strategy combines all modifications without losing any edits.
</p>
<p>
Ready to dive deeper? Check out the
<a
href="https://github.com/schmelczer/reconcile"
target="_blank"
rel="noopener noreferrer"
>documentation</a
>
or try editing the text boxes below to see Reconcile in action.
</p>
<p>
Use the tokenisation options below to experiment with different approaches —
the library also supports custom tokenisers.
</p> </p>
</header> </header>
@ -87,7 +96,9 @@
<span class="radio-custom" aria-hidden="true"></span> <span class="radio-custom" aria-hidden="true"></span>
<div class="radio-content"> <div class="radio-content">
<span class="radio-label">Character</span> <span class="radio-label">Character</span>
<span class="radio-description">Split by individual characters</span> <span class="radio-description"
>Fine-grained character-level merging</span
>
</div> </div>
</label> </label>
<label class="radio-option"> <label class="radio-option">
@ -101,7 +112,7 @@
<span class="radio-custom" aria-hidden="true"></span> <span class="radio-custom" aria-hidden="true"></span>
<div class="radio-content"> <div class="radio-content">
<span class="radio-label">Word</span> <span class="radio-label">Word</span>
<span class="radio-description">Split by words (default)</span> <span class="radio-description">Retain full words (default)</span>
</div> </div>
</label> </label>
<label class="radio-option"> <label class="radio-option">
@ -110,7 +121,7 @@
<div class="radio-content"> <div class="radio-content">
<span class="radio-label">Line</span> <span class="radio-label">Line</span>
<span class="radio-description" <span class="radio-description"
>Split by lines similarly to <code>git merge</code></span >Line-by-line like <code>git merge</code></span
> >
</div> </div>
</label> </label>
@ -120,7 +131,7 @@
<div class="text-area-card diamond-parent"> <div class="text-area-card diamond-parent">
<label <label
for="original" for="original"
title="The text document's content before any concurrent edits occurred." title="The original text before any concurrent edits were made."
>Original</label >Original</label
> >
<textarea id="original" name="original"></textarea> <textarea id="original" name="original"></textarea>
@ -129,9 +140,9 @@
<div class="text-area-card diamond-left"> <div class="text-area-card diamond-left">
<label <label
for="left" for="left"
title="Colour-coded tokens mark the origin of each token in the result. This text box is marked with the colour green." title="First user's edits — changes from this box appear in green in the result."
> >
First concurrent edit First user's edits
<div class="box Left"></div> <div class="box Left"></div>
</label> </label>
<textarea id="left" name="left"></textarea> <textarea id="left" name="left"></textarea>
@ -140,9 +151,9 @@
<div class="text-area-card diamond-right"> <div class="text-area-card diamond-right">
<label <label
for="right" for="right"
title="Colour-coded tokens mark the origin of each token in the result. This text box is marked with the colour blue." title="Second user's edits — changes from this box appear in blue in the result."
> >
Second concurrent edit Second user's edits
<div class="box Right"></div> <div class="box Right"></div>
</label> </label>
<textarea id="right" name="right"></textarea> <textarea id="right" name="right"></textarea>
@ -151,9 +162,9 @@
<div class="text-area-card diamond-result"> <div class="text-area-card diamond-result">
<label <label
for="merged" for="merged"
title="Read-only. Change the above text boxes to change the content of this box." title="The automatically merged result — edit the boxes above to see changes in real-time."
> >
Deconflicted result Merged result
<svg <svg
xmlns="http://www.w3.org/2000/svg" xmlns="http://www.w3.org/2000/svg"
width="24" width="24"

22
examples/website/package.json
View file

@ -1,38 +1,32 @@
{ {
"name": "portfolio", "name": "reconcile-example-website",
"description": "An easily configurable timeline of projects.", "description": "",
"private": true, "private": true,
"scripts": { "scripts": {
"start": "webpack serve --open --mode development", "start": "webpack serve --open --mode development",
"format": "prettier --write \"src/**/*.(ts|scss|json|html)\"", "format": "prettier --write \"./**/*.(ts|scss|json|html)\"",
"build": "webpack --mode production" "build": "webpack --mode production"
}, },
"repository": { "repository": {
"type": "git", "type": "git",
"url": "git+https://github.com/schmelczer/schmelczer.github.io.git" "url": "git+https://github.com/schmelczer/reconcile.git"
}, },
"keywords": [ "keywords": [],
"CV",
"curriculum",
"vitae",
"portfolio",
"resumé"
],
"author": "Andras Schmelczer", "author": "Andras Schmelczer",
"license": "GPL-3.0-or-later", "license": "GPL-3.0-or-later",
"bugs": { "bugs": {
"url": "https://github.com/schmelczer/schmelczer.github.io/issues" "url": "https://github.com/schmelczer/reconcile/issues"
}, },
"browserslist": [ "browserslist": [
"defaults" "defaults"
], ],
"homepage": "https://github.com/schmelczer/schmelczer.github.io#readme", "homepage": "https://github.com/schmelczer/reconcile#readme",
"devDependencies": { "devDependencies": {
"reconcile": "file:../../reconcile-js", "reconcile": "file:../../reconcile-js",
"css-loader": "^7.1.2", "css-loader": "^7.1.2",
"html-webpack-plugin": "^5.6.3", "html-webpack-plugin": "^5.6.3",
"inline-source-webpack-plugin": "^3.0.1",
"mini-css-extract-plugin": "^2.9.2", "mini-css-extract-plugin": "^2.9.2",
"inline-source-webpack-plugin": "^3.0.1",
"prettier": "^3.6.2", "prettier": "^3.6.2",
"resolve-url-loader": "^5.0.0", "resolve-url-loader": "^5.0.0",
"sass": "^1.89.2", "sass": "^1.89.2",

10
examples/website/style.scss
View file

@ -56,6 +56,16 @@ p * {
user-select: text; user-select: text;
} }
code {
background: #61769a;
color: #e2e8f0;
padding: 2px 6px;
border-radius: 4px;
font-family: 'SF Mono', 'Monaco', 'Inconsolata', 'Roboto Mono', 'Consolas', monospace;
font-size: 0.875em;
font-weight: 500;
}
header > p { header > p {
color: #5a6272; color: #5a6272;
font-size: 1.1rem; font-size: 1.1rem;

4
examples/website/tsconfig.json
View file

@ -13,7 +13,5 @@
"skipLibCheck": true, "skipLibCheck": true,
"inlineSourceMap": true "inlineSourceMap": true
}, },
"exclude": [ "exclude": ["./dist"]
"./dist"
]
} }

17
reconcile-js/package-lock.json generated
View file

@ -10,6 +10,7 @@
"devDependencies": { "devDependencies": {
"@types/jest": "^30.0.0", "@types/jest": "^30.0.0",
"jest": "^30.0.4", "jest": "^30.0.4",
"prettier": "^3.6.2",
"reconcile": "file:../pkg", "reconcile": "file:../pkg",
"ts-jest": "^29.4.0", "ts-jest": "^29.4.0",
"ts-loader": "^9.5.2", "ts-loader": "^9.5.2",
@ -4449,6 +4450,22 @@
"node": ">=8" "node": ">=8"
} }
}, },
"node_modules/prettier": {
"version": "3.6.2",
"resolved": "https://registry.npmjs.org/prettier/-/prettier-3.6.2.tgz",
"integrity": "sha512-I7AIg5boAr5R0FFtJ6rCfD+LFsWHp81dolrFD8S79U9tb8Az2nGrJncnMSnys+bpQJfRUzqs9hnA81OAA3hCuQ==",
"dev": true,
"license": "MIT",
"bin": {
"prettier": "bin/prettier.cjs"
},
"engines": {
"node": ">=14"
},
"funding": {
"url": "https://github.com/prettier/prettier?sponsor=1"
}
},
"node_modules/pretty-format": { "node_modules/pretty-format": {
"version": "30.0.2", "version": "30.0.2",
"resolved": "https://registry.npmjs.org/pretty-format/-/pretty-format-30.0.2.tgz", "resolved": "https://registry.npmjs.org/pretty-format/-/pretty-format-30.0.2.tgz",

4
reconcile-js/package.json
View file

@ -8,6 +8,7 @@
], ],
"scripts": { "scripts": {
"build": "webpack --mode production", "build": "webpack --mode production",
"format": "prettier --write \"./**/*.(ts|scss|json|html)\"",
"test": "NODE_OPTIONS=\"$NODE_OPTIONS --experimental-vm-modules\" jest" "test": "NODE_OPTIONS=\"$NODE_OPTIONS --experimental-vm-modules\" jest"
}, },
"devDependencies": { "devDependencies": {
@ -19,6 +20,7 @@
"tslib": "2.8.1", "tslib": "2.8.1",
"typescript": "5.8.3", "typescript": "5.8.3",
"webpack": "^5.99.9", "webpack": "^5.99.9",
"webpack-cli": "^6.0.1" "webpack-cli": "^6.0.1",
"prettier": "^3.6.2"
} }
} }

40
reconcile-js/src/index.test.ts
View file

@ -1,32 +1,28 @@
import { init, reconcile, reconcileWithHistory } from "./index"; import { init, reconcile, reconcileWithHistory } from './index';
import * as fs from "fs"; import * as fs from 'fs';
describe("reconcile", () => { describe('reconcile', () => {
it("tries calling functions without init", () => { it('tries calling functions without init', () => {
expect(() => reconcile("Hello", "Hello world", "Hi world")).toThrow( expect(() => reconcile('Hello', 'Hello world', 'Hi world')).toThrow(/call init()/);
expect(() => reconcileWithHistory('Hello', 'Hello world', 'Hi world')).toThrow(
/call init()/ /call init()/
); );
expect(() =>
reconcileWithHistory("Hello", "Hello world", "Hi world")
).toThrow(/call init()/);
}); });
it("call reconcile without cursors", async () => { it('call reconcile without cursors', async () => {
await initWasm(); await initWasm();
expect(reconcile("Hello", "Hello world", "Hi world").text).toEqual( expect(reconcile('Hello', 'Hello world', 'Hi world').text).toEqual('Hi world');
"Hi world"
);
}); });
it("call reconcile with cursors", async () => { it('call reconcile with cursors', async () => {
await initWasm(); await initWasm();
const result = reconcile( const result = reconcile(
"Hello", 'Hello',
{ {
text: "Hello world", text: 'Hello world',
cursors: [ cursors: [
{ {
id: 3, id: 3,
@ -35,7 +31,7 @@ describe("reconcile", () => {
], ],
}, },
{ {
text: "Hi world", text: 'Hi world',
cursors: [ cursors: [
{ {
id: 4, id: 4,
@ -46,7 +42,7 @@ describe("reconcile", () => {
} }
); );
expect(result.text).toEqual("Hi world"); expect(result.text).toEqual('Hi world');
expect(result.cursors).toEqual([ expect(result.cursors).toEqual([
{ id: 3, position: 0 }, { id: 3, position: 0 },
{ id: 4, position: 0 }, { id: 4, position: 0 },
@ -54,17 +50,17 @@ describe("reconcile", () => {
]); ]);
}); });
it("call reconcileWithHistory", async () => { it('call reconcileWithHistory', async () => {
await initWasm(); await initWasm();
const result = reconcileWithHistory("Hello", "Hello world", "Hi world"); const result = reconcileWithHistory('Hello', 'Hello world', 'Hi world');
expect(result.text).toEqual("Hi world"); expect(result.text).toEqual('Hi world');
expect(result.history.length).toBeGreaterThan(0); expect(result.history.length).toBeGreaterThan(0);
}); });
}); });
async function initWasm() { async function initWasm() {
const wasmBin = fs.readFileSync("../pkg/reconcile_bg.wasm"); const wasmBin = fs.readFileSync('../pkg/reconcile_bg.wasm');
await init({ module_or_path: wasmBin }); await init({ module_or_path: wasmBin });
} }

47
reconcile-js/src/index.ts
View file

@ -7,7 +7,7 @@ import wasmInit, {
reconcileWithHistory as wasmReconcileWithHistory, reconcileWithHistory as wasmReconcileWithHistory,
History, History,
InitInput, InitInput,
} from "reconcile"; } from 'reconcile';
export interface TextWithCursors { export interface TextWithCursors {
/** The document's entire content */ /** The document's entire content */
@ -39,16 +39,16 @@ export interface SpanWithHistory {
history: History; history: History;
} }
export type Tokenizer = "Line" | "Word" | "Character"; export type Tokenizer = 'Line' | 'Word' | 'Character';
const TOKENIZERS = ["Line", "Word", "Character"]; const TOKENIZERS = ['Line', 'Word', 'Character'];
let isInitialised = false; let isInitialised = false;
const UNINITIALISED_MODULE_ERROR = const UNINITIALISED_MODULE_ERROR =
"Reconcile module has not been initialized. Please call init() before using any other functions."; 'Reconcile module has not been initialized. Please call init() before using any other functions.';
const UNSUPPORTED_TOKENIZER_ERROR = `Unsupported tokenizer. Only ${TOKENIZERS.join( const UNSUPPORTED_TOKENIZER_ERROR = `Unsupported tokenizer. Only ${TOKENIZERS.join(
", " ', '
)} are supported.`; )} are supported.`;
/** /**
@ -83,7 +83,7 @@ export function reconcile(
original: string, original: string,
left: string | TextWithCursors, left: string | TextWithCursors,
right: string | TextWithCursors, right: string | TextWithCursors,
tokenizer: BuiltinTokenizer = "Word" tokenizer: BuiltinTokenizer = 'Word'
): TextWithCursors { ): TextWithCursors {
if (!isInitialised) { if (!isInitialised) {
throw new Error(UNINITIALISED_MODULE_ERROR); throw new Error(UNINITIALISED_MODULE_ERROR);
@ -122,7 +122,7 @@ export function reconcileWithHistory(
original: string, original: string,
left: string | TextWithCursors, left: string | TextWithCursors,
right: string | TextWithCursors, right: string | TextWithCursors,
tokenizer: BuiltinTokenizer = "Word" tokenizer: BuiltinTokenizer = 'Word'
): TextWithCursorsAndHistory { ): TextWithCursorsAndHistory {
if (!isInitialised) { if (!isInitialised) {
throw new Error(UNINITIALISED_MODULE_ERROR); throw new Error(UNINITIALISED_MODULE_ERROR);
@ -135,12 +135,7 @@ export function reconcileWithHistory(
const leftCursor = toWasmTextWithCursors(left); const leftCursor = toWasmTextWithCursors(left);
const rightCursor = toWasmTextWithCursors(right); const rightCursor = toWasmTextWithCursors(right);
const result = wasmReconcileWithHistory( const result = wasmReconcileWithHistory(original, leftCursor, rightCursor, tokenizer);
original,
leftCursor,
rightCursor,
tokenizer
);
leftCursor.free(); leftCursor.free();
rightCursor.free(); rightCursor.free();
@ -155,29 +150,19 @@ export function reconcileWithHistory(
}; };
} }
function toWasmTextWithCursors( function toWasmTextWithCursors(text: string | TextWithCursors): wasmTextWithCursors {
text: string | TextWithCursors const isInputString = typeof text == 'string';
): wasmTextWithCursors {
const isInputString = typeof text == "string";
const leftText = isInputString ? text : text.text; const leftText = isInputString ? text : text.text;
const leftCursors = isInputString ? [] : text.cursors ?? []; const leftCursors = isInputString ? [] : (text.cursors ?? []);
return new wasmTextWithCursors( return new wasmTextWithCursors(leftText, leftCursors.map(toWasmCursorPosition));
leftText,
leftCursors.map(toWasmCursorPosition)
);
} }
function toWasmCursorPosition({ function toWasmCursorPosition({ id, position }: CursorPosition): wasmCursorPosition {
id,
position,
}: CursorPosition): wasmCursorPosition {
return new wasmCursorPosition(id, position); return new wasmCursorPosition(id, position);
} }
function toTextWithCursors( function toTextWithCursors(textWithCursor: wasmTextWithCursors): TextWithCursors {
textWithCursor: wasmTextWithCursors
): TextWithCursors {
return { return {
text: textWithCursor.text(), text: textWithCursor.text(),
cursors: textWithCursor.cursors().map(toCursorPosition), cursors: textWithCursor.cursors().map(toCursorPosition),
@ -191,9 +176,7 @@ function toCursorPosition(cursor: wasmCursorPosition): CursorPosition {
}; };
} }
function toSpanWithHistory( function toSpanWithHistory(textWithHistory: wasmSpanWithHistory): SpanWithHistory {
textWithHistory: wasmSpanWithHistory
): SpanWithHistory {
return { return {
text: textWithHistory.text(), text: textWithHistory.text(),
history: textWithHistory.history(), history: textWithHistory.history(),

4
reconcile-js/tsconfig.json
View file

@ -13,7 +13,5 @@
"skipLibCheck": true, "skipLibCheck": true,
"inlineSourceMap": true "inlineSourceMap": true
}, },
"exclude": [ "exclude": ["./dist"]
"./dist"
]
} }

6
scripts/lint.sh
View file

@ -5,4 +5,10 @@ set -e
cargo clippy --all-targets --all-features --fix --allow-dirty --allow-staged cargo clippy --all-targets --all-features --fix --allow-dirty --allow-staged
cargo fmt --all cargo fmt --all
cd reconcile-js
npm run format
cd ../examples/website
npm run format
echo "Success!" echo "Success!"

88
src/lib.rs
View file

@ -1,80 +1,86 @@
//! # Reconcile //! # Reconcile: conflict-free 3-way text merging
//! //!
//! [`diff3`](https://www.gnu.org/software/diffutils/manual/html_node/Invoking-diff3.html) (or `git merge`) //! Think [`diff3`](https://www.gnu.org/software/diffutils/manual/html_node/Invoking-diff3.html) or `git merge`,
//! but with automatic conflict resolution. //! but with intelligent conflict resolution.
//! //!
//! Reconcile is a Rust and JavaScript (through WebAssembly) library for merging //! Reconcile is a Rust and JavaScript (via WebAssembly) library that merges
//! text without user intervention. It automatically resolves conflicts that //! conflicting text edits without requiring manual intervention. Where
//! would typically require user action in traditional 3-way merge tools. //! traditional 3-way merge tools would leave you with conflict markers to
//! resolve by hand, Reconcile automatically weaves changes together using
//! sophisticated algorithms inspired by Operational Transformation.
//! //!
//! Try out the [interactive demo](https://schmelczer.dev/reconcile)! //! ✨ **[Try the interactive demo](https://schmelczer.dev/reconcile)** to see it in action!
//! //!
//! ``` //! ```
//! use reconcile::{reconcile, BuiltinTokenizer}; //! use reconcile::{reconcile, BuiltinTokenizer};
//! //!
//! // Start with original text
//! let parent = "Merging text is hard!"; //! let parent = "Merging text is hard!";
//! let left = "Merging text is easy!"; //! // Two people edit simultaneously
//! let right = "With reconcile, merging documents is hard!"; //! let left = "Merging text is easy!"; // Changed "hard" to "easy"
//! let right = "With reconcile, merging documents is hard!"; // Added prefix and changed word
//! //!
//! let deconflicted = reconcile(parent, &left.into(), &right.into(), &*BuiltinTokenizer::Word); //! // Reconcile combines both changes intelligently
//! assert_eq!(deconflicted.apply().text(), "With reconcile, merging documents is easy!"); //! let result = reconcile(parent, &left.into(), &right.into(), &*BuiltinTokenizer::Word);
//! assert_eq!(result.apply().text(), "With reconcile, merging documents is easy!");
//! ``` //! ```
//! > You can also try out an interactive demo at [schmelczer.dev/reconcile](https://schmelczer.dev/reconcile).
//! //!
//! ## Tokenizing //! ## Tokenisation strategies
//! //!
//! Merging is done on the token level, the granularity of which is //! Merging happens at the token level, where you control the granularity.
//! configurable. By default, words are the atoms for merging and thus words //! By default, words serve as the atomic units for merging, ensuring words
//! can't get jumbled up at the end of reconciling. //! remain intact during the reconciliation process.
//! //!
//! ### Built-in tokenizers //! ### Built-in tokenisers
//! //!
//! ``` //! ```
//! use reconcile::{reconcile, BuiltinTokenizer}; //! use reconcile::{reconcile, BuiltinTokenizer};
//! //!
//! let parent = "The quick brown fox\n"; //! let parent = "The quick brown fox\n";
//! let left = "The very quick brown fox\n"; //! let left = "The very quick brown fox\n"; // Added "very"
//! let right = "The quick red fox\n"; //! let right = "The quick red fox\n"; // Changed "brown" to "red"
//! //!
//! // Using line-based tokenisation
//! let result = reconcile(parent, &left.into(), &right.into(), &*BuiltinTokenizer::Line); //! let result = reconcile(parent, &left.into(), &right.into(), &*BuiltinTokenizer::Line);
//! assert_eq!(result.apply().text(), "The quick red foxThe very quick brown fox\n"); //! assert_eq!(result.apply().text(), "The quick red foxThe very quick brown fox\n");
//! ``` //! ```
//! //!
//! ### Custom tokenization //! ### Custom tokenisation
//! //!
//! If something custom is needed, for instance, to better support structured //! For specialised use cases—such as structured text like Markdown or HTML—
//! text such as Markdown or HTML, a custom tokenizer can be implemented: //! you can implement custom tokenisation logic:
//! //!
//! ``` //! ```
//! use reconcile::{reconcile, Token, BuiltinTokenizer}; //! use reconcile::{reconcile, Token, BuiltinTokenizer};
//! //!
//! // Example with custom tokenizer - split by sentences //! // Example: custom sentence-based tokeniser
//! let sentence_tokenizer = |text: &str| { //! let sentence_tokeniser = |text: &str| {
//! text.split(". ") //! text.split(". ")
//! .map(|sentence| Token::new( //! .map(|sentence| Token::new(
//! sentence.to_string(), //! sentence.to_string(),
//! sentence.to_string(), //! sentence.to_string(),
//! false, // don't allow joining token with the preceding one //! false, // don't allow joining with the preceding token
//! false, // don't allow joining token with the following one //! false, // don't allow joining with the following token
//! )) //! ))
//! .collect::<Vec<_>>() //! .collect::<Vec<_>>()
//! }; //! };
//! //!
//! let parent = "Hello world. This is a test."; //! let parent = "Hello world. This is a test.";
//! let left = "Hello beautiful world. This is a test."; //! let left = "Hello beautiful world. This is a test."; // Added "beautiful"
//! let right = "Hello world. This is a great test."; //! let right = "Hello world. This is a great test."; // Changed "a" to "great"
//! //!
//! // Using built-in tokenizer is usually sufficient //! // For most cases, the built-in word tokeniser works perfectly
//! let result = reconcile(parent, &left.into(), &right.into(), &*BuiltinTokenizer::Word); //! let result = reconcile(parent, &left.into(), &right.into(), &*BuiltinTokenizer::Word);
//! assert_eq!(result.apply().text(), "Hello beautiful world. This is a great test."); //! assert_eq!(result.apply().text(), "Hello beautiful world. This is a great test.");
//! ``` //! ```
//! > By setting the joinability to `false`, longer runs of inserts will be //! > **Tip**: Setting joinability to `false` causes longer runs of insertions
//! > interleaved like LRLRLR instead of LLLRRR. //! > to interleave (LRLRLR) rather than group together (LLLRRR), which can
//! > produce more natural-looking merged text.
//! //!
//! ## Cursors and selection ranges //! ## Cursor tracking
//! //!
//! The library supports updating cursor and selection ranges during the merging //! Perfect for collaborative editors—the library automatically repositions
//! for interactive workflows: //! cursors and selection ranges during merging:
//! //!
//! ``` //! ```
//! use reconcile::{reconcile, BuiltinTokenizer, TextWithCursors, CursorPosition}; //! use reconcile::{reconcile, BuiltinTokenizer, TextWithCursors, CursorPosition};
@ -86,21 +92,21 @@
//! ); //! );
//! let right = TextWithCursors::new( //! let right = TextWithCursors::new(
//! "Hi world".to_string(), //! "Hi world".to_string(),
//! vec![CursorPosition { id: 2, char_index: 0 }] // At beginning //! vec![CursorPosition { id: 2, char_index: 0 }] // At the beginning
//! ); //! );
//! //!
//! let result = reconcile(parent, &left, &right, &*BuiltinTokenizer::Word); //! let result = reconcile(parent, &left, &right, &*BuiltinTokenizer::Word);
//! let merged = result.apply(); //! let merged = result.apply();
//! //!
//! assert_eq!(merged.text(), "Hi beautiful world"); //! assert_eq!(merged.text(), "Hi beautiful world");
//! // Cursors are automatically repositioned //! // Cursors are automatically repositioned in the merged text
//! assert_eq!(merged.cursors().len(), 2); //! assert_eq!(merged.cursors().len(), 2);
//! ``` //! ```
//! //!
//! ## The algorithm //! ## How it works
//! //!
//! For a discussion of the algorithm and architecture, see the //! For a detailed explanation of the algorithm and architecture, see the
//! [README](README.md#algorithm) page. //! [README](README.md#how-it-works).
mod operation_transformation; mod operation_transformation;
mod raw_operation; mod raw_operation;
@ -108,8 +114,8 @@ mod tokenizer;
mod types; mod types;
mod utils; mod utils;
pub use operation_transformation::{EditedText, reconcile}; pub use operation_transformation::{reconcile, EditedText};
pub use tokenizer::{BuiltinTokenizer, Tokenizer, token::Token}; pub use tokenizer::{token::Token, BuiltinTokenizer, Tokenizer};
pub use types::{ pub use types::{
cursor_position::CursorPosition, history::History, side::Side, cursor_position::CursorPosition, history::History, side::Side,
span_with_history::SpanWithHistory, text_with_cursors::TextWithCursors, span_with_history::SpanWithHistory, text_with_cursors::TextWithCursors,

47
src/tokenizer/snapshots/reconcile__tokenizer__character_tokenizer__tests__with_snapshots-2.snap
View file

@ -1,142 +1,143 @@
--- ---
source: src/tokenizer/character_tokenizer.rs source: src/tokenizer/character_tokenizer.rs
expression: "character_tokenizer(\" hello, \\nwhere are you?\")" expression: "character_tokenizer(\" hello, \\nwhere are you?\")"
snapshot_kind: text
--- ---
[ [
Token { Token {
normalised: " ", normalized: " ",
original: " ", original: " ",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "h", normalized: "h",
original: "h", original: "h",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "e", normalized: "e",
original: "e", original: "e",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "l", normalized: "l",
original: "l", original: "l",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "l", normalized: "l",
original: "l", original: "l",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "o", normalized: "o",
original: "o", original: "o",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: ",", normalized: ",",
original: ",", original: ",",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: " ", normalized: " ",
original: " ", original: " ",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "\n", normalized: "\n",
original: "\n", original: "\n",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "w", normalized: "w",
original: "w", original: "w",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "h", normalized: "h",
original: "h", original: "h",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "e", normalized: "e",
original: "e", original: "e",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "r", normalized: "r",
original: "r", original: "r",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "e", normalized: "e",
original: "e", original: "e",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: " ", normalized: " ",
original: " ", original: " ",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "a", normalized: "a",
original: "a", original: "a",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "r", normalized: "r",
original: "r", original: "r",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "e", normalized: "e",
original: "e", original: "e",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: " ", normalized: " ",
original: " ", original: " ",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "y", normalized: "y",
original: "y", original: "y",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "o", normalized: "o",
original: "o", original: "o",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "u", normalized: "u",
original: "u", original: "u",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "?", normalized: "?",
original: "?", original: "?",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

2
src/tokenizer/snapshots/reconcile__tokenizer__line_tokenizer__tests__with_snapshots-2.snap
View file

@ -5,7 +5,7 @@ snapshot_kind: text
--- ---
[ [
Token { Token {
normalised: "Hello", normalized: "Hello",
original: "Hello", original: "Hello",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

6
src/tokenizer/snapshots/reconcile__tokenizer__line_tokenizer__tests__with_snapshots-3.snap
View file

@ -5,19 +5,19 @@ snapshot_kind: text
--- ---
[ [
Token { Token {
normalised: "Hello", normalized: "Hello",
original: "Hello", original: "Hello",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "\n", normalized: "\n",
original: "\n", original: "\n",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "World", normalized: "World",
original: "World", original: "World",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

8
src/tokenizer/snapshots/reconcile__tokenizer__line_tokenizer__tests__with_snapshots-4.snap
View file

@ -5,25 +5,25 @@ snapshot_kind: text
--- ---
[ [
Token { Token {
normalised: "Hello", normalized: "Hello",
original: "Hello", original: "Hello",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "\n", normalized: "\n",
original: "\n", original: "\n",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "World", normalized: "World",
original: "World", original: "World",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "\n", normalized: "\n",
original: "\n", original: "\n",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

6
src/tokenizer/snapshots/reconcile__tokenizer__line_tokenizer__tests__with_snapshots-5.snap
View file

@ -5,19 +5,19 @@ snapshot_kind: text
--- ---
[ [
Token { Token {
normalised: "Line 1", normalized: "Line 1",
original: "Line 1", original: "Line 1",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "\r\n", normalized: "\r\n",
original: "\r\n", original: "\r\n",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "Line 2", normalized: "Line 2",
original: "Line 2", original: "Line 2",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

14
src/tokenizer/snapshots/reconcile__tokenizer__line_tokenizer__tests__with_snapshots-6.snap
View file

@ -5,43 +5,43 @@ snapshot_kind: text
--- ---
[ [
Token { Token {
normalised: "Multi", normalized: "Multi",
original: "Multi", original: "Multi",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "\n", normalized: "\n",
original: "\n", original: "\n",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "Line", normalized: "Line",
original: "Line", original: "Line",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "\n", normalized: "\n",
original: "\n", original: "\n",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "Text", normalized: "Text",
original: "Text", original: "Text",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "\n", normalized: "\n",
original: "\n", original: "\n",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "Here", normalized: "Here",
original: "Here", original: "Here",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

2
src/tokenizer/snapshots/reconcile__tokenizer__line_tokenizer__tests__with_snapshots-7.snap
View file

@ -5,7 +5,7 @@ snapshot_kind: text
--- ---
[ [
Token { Token {
normalised: "\n", normalized: "\n",
original: "\n", original: "\n",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

4
src/tokenizer/snapshots/reconcile__tokenizer__line_tokenizer__tests__with_snapshots-8.snap
View file

@ -5,13 +5,13 @@ snapshot_kind: text
--- ---
[ [
Token { Token {
normalised: "\n", normalized: "\n",
original: "\n", original: "\n",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "\n", normalized: "\n",
original: "\n", original: "\n",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

8
src/tokenizer/snapshots/reconcile__tokenizer__line_tokenizer__tests__with_snapshots-9.snap
View file

@ -5,25 +5,25 @@ snapshot_kind: text
--- ---
[ [
Token { Token {
normalised: "Start", normalized: "Start",
original: "Start", original: "Start",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "\n", normalized: "\n",
original: "\n", original: "\n",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "\n", normalized: "\n",
original: "\n", original: "\n",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "End", normalized: "End",
original: "End", original: "End",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

8
src/tokenizer/snapshots/reconcile__tokenizer__word_tokenizer__tests__with_snapshots-3.snap
View file

@ -1,23 +1,23 @@
--- ---
source: reconcile/src/tokenizer/word_tokenizer.rs source: src/tokenizer/word_tokenizer.rs
expression: "word_tokenizer(\" what? \")" expression: "word_tokenizer(\" what? \")"
snapshot_kind: text snapshot_kind: text
--- ---
[ [
Token { Token {
normalised: " what?", normalized: " what?",
original: " ", original: " ",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "what?", normalized: "what?",
original: "what?", original: "what?",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: " ", normalized: " ",
original: " ", original: " ",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

18
src/tokenizer/snapshots/reconcile__tokenizer__word_tokenizer__tests__with_snapshots-4.snap
View file

@ -1,53 +1,53 @@
--- ---
source: reconcile/src/tokenizer/word_tokenizer.rs source: src/tokenizer/word_tokenizer.rs
expression: "word_tokenizer(\" hello, \\nwhere are you?\")" expression: "word_tokenizer(\" hello, \\nwhere are you?\")"
snapshot_kind: text snapshot_kind: text
--- ---
[ [
Token { Token {
normalised: " hello,", normalized: " hello,",
original: " ", original: " ",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "hello,", normalized: "hello,",
original: "hello,", original: "hello,",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: " \nwhere", normalized: " \nwhere",
original: " \n", original: " \n",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "where", normalized: "where",
original: "where", original: "where",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: " are", normalized: " are",
original: " ", original: " ",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "are", normalized: "are",
original: "are", original: "are",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: " you?", normalized: " you?",
original: " ", original: " ",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "you?", normalized: "you?",
original: "you?", original: "you?",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

8
src/tokenizer/snapshots/reconcile__tokenizer__word_tokenizer__tests__with_snapshots.snap
View file

@ -1,23 +1,23 @@
--- ---
source: reconcile/src/tokenizer/word_tokenizer.rs source: src/tokenizer/word_tokenizer.rs
expression: "word_tokenizer(\"Hi there!\")" expression: "word_tokenizer(\"Hi there!\")"
snapshot_kind: text snapshot_kind: text
--- ---
[ [
Token { Token {
normalised: "Hi", normalized: "Hi",
original: "Hi", original: "Hi",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: " there!", normalized: " there!",
original: " ", original: " ",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
}, },
Token { Token {
normalised: "there!", normalized: "there!",
original: "there!", original: "there!",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

13
src/utils/snapshots/reconcile__utils__myers_diff__tests__complex_diff.snap
View file

@ -1,12 +1,13 @@
--- ---
source: src/utils/myers_diff.rs source: src/utils/myers_diff.rs
expression: result expression: result
snapshot_kind: text
--- ---
[ [
Equal( Equal(
[ [
Token { Token {
normalised: "a", normalized: "a",
original: "a", original: "a",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
@ -16,7 +17,7 @@ expression: result
Insert( Insert(
[ [
Token { Token {
normalised: "x", normalized: "x",
original: "x", original: "x",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
@ -26,7 +27,7 @@ expression: result
Delete( Delete(
[ [
Token { Token {
normalised: "b", normalized: "b",
original: "b", original: "b",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
@ -36,7 +37,7 @@ expression: result
Equal( Equal(
[ [
Token { Token {
normalised: "c", normalized: "c",
original: "c", original: "c",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
@ -46,7 +47,7 @@ expression: result
Insert( Insert(
[ [
Token { Token {
normalised: "y", normalized: "y",
original: "y", original: "y",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
@ -56,7 +57,7 @@ expression: result
Delete( Delete(
[ [
Token { Token {
normalised: "d", normalized: "d",
original: "d", original: "d",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

5
src/utils/snapshots/reconcile__utils__myers_diff__tests__delete_only.snap
View file

@ -1,12 +1,13 @@
--- ---
source: src/utils/myers_diff.rs source: src/utils/myers_diff.rs
expression: result expression: result
snapshot_kind: text
--- ---
[ [
Delete( Delete(
[ [
Token { Token {
normalised: "a", normalized: "a",
original: "a", original: "a",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
@ -16,7 +17,7 @@ expression: result
Delete( Delete(
[ [
Token { Token {
normalised: "b", normalized: "b",
original: "b", original: "b",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

7
src/utils/snapshots/reconcile__utils__myers_diff__tests__identical_content.snap
View file

@ -1,12 +1,13 @@
--- ---
source: src/utils/myers_diff.rs source: src/utils/myers_diff.rs
expression: result expression: result
snapshot_kind: text
--- ---
[ [
Equal( Equal(
[ [
Token { Token {
normalised: "a", normalized: "a",
original: "a", original: "a",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
@ -16,7 +17,7 @@ expression: result
Equal( Equal(
[ [
Token { Token {
normalised: "b", normalized: "b",
original: "b", original: "b",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
@ -26,7 +27,7 @@ expression: result
Equal( Equal(
[ [
Token { Token {
normalised: "c", normalized: "c",
original: "c", original: "c",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

5
src/utils/snapshots/reconcile__utils__myers_diff__tests__insert_only.snap
View file

@ -1,12 +1,13 @@
--- ---
source: src/utils/myers_diff.rs source: src/utils/myers_diff.rs
expression: result expression: result
snapshot_kind: text
--- ---
[ [
Insert( Insert(
[ [
Token { Token {
normalised: "a", normalized: "a",
original: "a", original: "a",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
@ -16,7 +17,7 @@ expression: result
Insert( Insert(
[ [
Token { Token {
normalised: "b", normalized: "b",
original: "b", original: "b",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,

11
src/utils/snapshots/reconcile__utils__myers_diff__tests__prefix_and_suffix.snap
View file

@ -1,12 +1,13 @@
--- ---
source: src/utils/myers_diff.rs source: src/utils/myers_diff.rs
expression: result expression: result
snapshot_kind: text
--- ---
[ [
Equal( Equal(
[ [
Token { Token {
normalised: "a", normalized: "a",
original: "a", original: "a",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
@ -16,7 +17,7 @@ expression: result
Delete( Delete(
[ [
Token { Token {
normalised: "b", normalized: "b",
original: "b", original: "b",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
@ -26,7 +27,7 @@ expression: result
Delete( Delete(
[ [
Token { Token {
normalised: "c", normalized: "c",
original: "c", original: "c",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
@ -36,7 +37,7 @@ expression: result
Insert( Insert(
[ [
Token { Token {
normalised: "x", normalized: "x",
original: "x", original: "x",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,
@ -46,7 +47,7 @@ expression: result
Equal( Equal(
[ [
Token { Token {
normalised: "d", normalized: "d",
original: "d", original: "d",
is_left_joinable: true, is_left_joinable: true,
is_right_joinable: true, is_right_joinable: true,