Add Markdown tokeniser #48

Merged
schmelczer merged 5 commits from asch/md-tokeniser into main 2026-03-11 20:47:44 +00:00
schmelczer commented 2026-03-11 20:47:32 +00:00 (Migrated from github.com)

Word-based merging would often result in mangled markdown. We can now tokenise the text in markdown-aware, smarter way.

Word-based merging would often result in mangled markdown. We can now tokenise the text in markdown-aware, smarter way.
Sign in to join this conversation.
No description provided.