Add Markdown tokeniser #48

Merged

schmelczer merged 5 commits from asch/md-tokeniser into main 2026-03-11 20:47:44 +00:00

Conversation 0 Commits 5 Files changed 29 +1539 -145

5 commits

Author	SHA1	Message	Date
Andras Schmelczer	09b5c606ea	Update tests	2026-03-11 20:43:41 +00:00
Andras Schmelczer	5978f73c97	Minimise allocations	2026-03-11 20:43:34 +00:00
Andras Schmelczer	bbe3b7573a	Update website	2026-03-11 20:39:36 +00:00
Andras Schmelczer	9da5bf6e3e	Add snapshots	2026-03-11 20:39:14 +00:00
Andras Schmelczer	446bbdfe5d	Implement makrdown tokeniser	2026-03-11 20:39:04 +00:00