Can I build a modified Markdown reader using Lua?

[dominikue]

I’d like to extend Pandoc’s reading step so that some substrings are parsed into AST directly (via a custom lua reader), while the remaining text is parsed by an existing reader (Markdown). Is there a supported way to (a) splice “pre-parsed” AST nodes into the document while (b) delegating all other text to a built-in reader?

.md → Own Reader → Markdown Reader → AST → Filter → AST → latex

Principle is similar to Example: modified Markdown writer.

Background / Motivation
I have documents that are mostly Markdown, but include specific scientific syntax (e.g., figures + description syntax, subfigures, etc.). I’d prefer not to preprocess into Markdown with placeholders and then repair via filters. Instead, I’m looking for a cleaner approach:

• detect my custom segments and turn them straight into Pandoc AST nodes,
• feed all other text to the normal Markdown reader,
• result into one Pandoc AST.

Would this be possible or do I need to use here lua filters? I want to avoid implement/extend the reader in Haskell.

Thanks in advance!
Dominik

[jgm]

Yes, you can create a custom Lua reader that divides the text into chunks and generates AST nodes directly for some chunks while passing others through pandoc's markdown reader to create AST nodes. The Lua function you'd use is pandoc.read. There are some limitations to this, because each chunk you pass to the markdown reader will be parsed in isolation. Thus, anything that depends on the global context may not come out right -- e.g. if you have reference links or footnotes defined elsewhere in the document, or if you use automatically numbered example lists. But it should largely work.

[dominikue]

Thanks @jgm! I will try this!

[fiapps]

Another approach is to use panluna: this is an extensible Markdown reader for pandoc. This would allow you to parse your custom syntax as part of a complete document.