Incremental Markdown parser that emits streams of semantic events, plus tools to manipulate them — designed for real-time rendering of streamed LLM output.

Use cases

Markdown Parsing

val markdown = """
# Hello

A *streaming* parser.
"""

flowOf(markdown).parse().collect {
    println(it)
}

Will print:

{"type":"mark","name":"h1"}
{"type":"text","text":"Hello"}
{"type":"unmark","name":"h1"}
{"type":"mark","name":"p"}
{"type":"text","text":"A"}
{"type":"text","text":" "}
{"type":"mark","name":"em"}
{"type":"text","text":"streaming"}
{"type":"unmark","name":"em"}
{"type":"text","text":" "}
{"type":"text","text":"parser."}
{"type":"unmark","name":"p"}

The stream is append-only: each event is emitted as soon as the parser commits to it, so <h1> opens before Hello arrives and <em> opens the moment the * resolves.

Rendering Markdown as HTML

println(flowOf(markdown).parse().render())

Will print:

<h1>
  Hello
</h1>
<p>
  A <em>streaming</em> parser.
</p>

Rendering Markdown as DOM (Kotlin JS)

val markdownFlow = flowOf(markdown)
document.body!!.appendSemanticEvents(
    markdownFlow.parse()
)

Renders equivalent HTML into the browser's DOM tree.

Note: Typically markdownFlow: Flow<String> represents a Markdown text stream, for example from LLM inference.

Transforming the event stream

val emphasizeToStrong = Transformer {
    match("em") {
        "strong" {
            children()
        }
    }
}

println(
    flowOf(markdown)
        .parse()
        .transform(emphasizeToStrong)
        .render()
)

Will print:

<h1>
  Hello
</h1>
<p>
  A <strong>streaming</strong> parser.
</p>

The Transformer DSL rewrites the event stream on the fly — match marks by name (or expression), emit replacement marks, wrap or unwrap nested content, or rewrite text. Transformations compose, so the same pipeline can normalize HTML to Markdown, redact spans, or route <thinking> blocks to a separate sink.

If you have transformed XML trees with XSLT, the model should feel familiar — match { ... } plays the role of an XSLT template, children() mirrors <xsl:apply-templates/>, and emitted marks/text replace <xsl:element> / <xsl:text>. The difference: it operates on a streaming event flow rather than a buffered document tree, and the source is Markdown (or mixed Markdown + HTML) rather than XML.

Supported Markdown features

GFM is the dialect LLMs were trained on, which is why we take it as the baseline — not as a conformance target. The parser diverges where spec-correct behaviour would force buffering past the next emitted event (which would break streaming, the whole point of the library), and those divergent shapes are ones LLMs effectively never emit. The long-term aim is a separate spec, anchored on this parser, defined to fit the streaming model rather than the document model.

GFM baseline

Extensions beyond GFM

Namespaced tags let you embed arbitrary custom markup in a Markdown document. Any <namespace:tagname …> / </namespace:tagname> pair passes through as Mark/Unmark events with isTagged = true and parsed attributes — your renderer handles them however it wants. This covers use cases like custom card components, alert boxes, or any domain-specific block type without requiring new parser syntax.

GFM features not supported

See markanywhere-parse/README.md for a full list of divergences and their rationale.

Modules

You can depend only on markanywhere-parse and consume the Flow<SemanticEvent> with your own renderer — the API surface is a single three-variant sealed class. The markanywhere-transform module additionally lets you intercept and rewrite events before they reach any renderer.

Elaborate rationale

We use language to convey meaning, and we use text to express language. The document-whether scroll, codex, or book-established a paradigm for how text is preserved as a packaged unit. Documents also introduced formatting: visual and structural conventions that signal the intent behind particular fragments of text within a larger context.

When we built machines to process text, we formalized this into "document formats". These formats naturally inherited the hierarchical structure of books-parts, chapters, sections, paragraphs-and the software we built assumed that documents exist as complete artifacts to be parsed, transformed, and rendered.

But something new has emerged. We started texting each other, and text became a stream of information: received, comprehended, and often discarded in the moment of reception. This is also the communication paradigm between humans and LLMs. The text is not a document to be opened and read-it is an unfolding stream, with alternating modalities, comprehended while being generated.

Structured documents are not the right abstraction here. What we need instead is an ontology of expressive meaning as a stream of events: each event signaling either an incremental fragment of text or a transition between modalities of linguistic expression (from prose to code, from paragraph to heading, from plain text to emphasis). markanywhere inverts the traditional document processing flow. Rather than consuming complete documents and producing structure, it consumes streaming tokens and emits semantic events in real-time. These events can then be transformed-also as a stream-into various output formats: HTML, Markdown, XML, or whatever the receiving context requires.

The ontology of a meaningful stream of text

The SemanticEvent can be a:

• Text: a chunk of characters
• Mark (e.g. <em> tag, with optional attributes)
• Unmark (e.g. </div>, indicating that previously opened mark is closed)

Mark and Unmark carry an isTagged flag distinguishing the origin of the event: true when it comes from an actual HTML/XML tag in the source, false when it is derived from Markdown syntax (e.g. *text* yields an em mark with isTagged = false, while <em>text</em> yields isTagged = true). The same SemanticEvent stream can therefore represent pure Markdown, pure HTML/XML (everything isTagged = true), or HTML embedded in Markdown — with the distinction preserved end-to-end so downstream renderers can treat each origin appropriately.

See the SemanticEvent definition.

Usage

In build.gradle.kts add:

dependencies {
    implementation("com.xemantic.markanywhere:markanywhere:0.1.3")
}