XML namespaces

[steven-r]

I recently started using the XML output in my project. I've created a publishing pipeline getting word documents, translating them with pandoc and a lot of lua and python scripts into several target formats (pdf over LaTeX, epub, html, ...) For several reasons I use intermediate formats to accomplish a couple of tasks (like word count, spelling via language tool, ...)

In former releases I used the json format but started to use XML as this format seems more suitable to process (combined texts, good mix between processing capability and readability).

As one of my first problems I reconized that the XML generated does not include a namespace. This is fine as long as standard (pandoc) tags are used. I failed when using epub-special tags like epub:type.

While processing this XML, any parser used complains about a missing namespace (epub:) (see the pandoc manual about the epub:type semantics)

Is there any chance to have namespaces available or at least to have a command line parameter to add namespaces?

pandoc --from docx test.doc --to=xml test.xml --xml-namespace=epub:http//...

in addition, it might make sense to have a default pandoc namespace available to allow parsers to recognize a document as a valid pandoc XML document.

[jgm]

Sorry, I can't really understand what you're trying to do from this description.

[massifrg]

In blocks and inlines that provide an Attr you can insert an attribute starting with xmlns::

pandoc -f markdown -t xml 
# Title {xmlns:epub="xyz"} 

(CTRL+D) and you get:

<?xml version='1.0' ?>
<Pandoc api-version="1,23,1">
<meta />
<blocks>
<Header id="title" xmlns:epub="xyz" level="1">Title</Header>
</blocks>
</Pandoc>

A command line option could be useful to add a namespace to the root <Pandoc> element.
But, while the former use of arbitrary attributes in Attr is fine for the xml reader and Pandoc, the latter would add an attribute that is not expected in Pandoc AST.

The outermost element you can use to declare a namespace is likely a Div.
If you don't need namespaces in metadata, you can declare a namespace embedding all the blocks of your document in a Div with a xmlns:epub attribute.

[jgm]

@massifrg do you have any thoughts on this?

[steven-r]

Actually, the point is different. Take the following as an example:

Consider the following input (as recommended in the manual):

# Chapter {epub:type=prologue}

A long time ago ...

Converting this to XML results in:

<?xml version='1.0' ?>
<Pandoc api-version="1,23,1">
  <meta />
    <blocks>
      <Header id="chapter" epub:type="prologue" level="1">Chapter</Header>
      <Para>A long time ago …</Para>
   </blocks>
</Pandoc>

looks legit, but running xmllint on the output file results in

sreindl@LAPTOP-1SPATK57:~/src/test$ xmllint test.xml
test.xml:5: namespace error : Namespace prefix epub for type on Header is not defined
<Header id="chapter" epub:type="prologue" level="1">Chapter</Header>

Enhancing the <Pandoc> tag by

<Pandoc api-version="1,23,1" xmlns:epub="http://www.idpf.org/2007/ops">

results in valid XML and ease in post-processing. Anyhow, for completeness, there should be a global namespace on top:

<Pandoc api-version="1,23,1" xmlns:epub="http://www.idpf.org/2007/ops" xmlns="http://pandoc.org/2025/xml.html">

but the latter is just a recommendation. The first one (epub) is a real issue (at least for me)

[massifrg]

As a workaround, you could do this:

::: {xmlns:epub="http://www.idpf.org/2007/ops"}
# Chapter {epub:type=prologue}

A long time ago ...
:::

in xml:

<?xml version='1.0' ?>
<Pandoc api-version="1,23,1">
<meta />
<blocks>
<Div xmlns:epub="http://www.idpf.org/2007/ops">
<Header id="chapter" epub:type="prologue" level="1">Chapter</Header>
<Para>A long time ago …</Para>
</Div>
</blocks>
</Pandoc>

I tested it with xmllint and it does not complain.

You may also write a simple lua filter to surround all the blocks of a Pandoc document with such a Div.

I have nothing against the --xml-namespace=epub:http://www.idpf.org/2007/ops option in the xml writer to add such an attribute to the <Pandoc> element.
Currently I would need some time to figure out how to code it in haskell.

The problem I see is on the xml reader side, because it should ignore that attribute, since the Pandoc item has metadata and blocks, but no attributes.
Alternatively, the namespace(s) could be stored in metadata, but it does not look such a clean solution to me, at least as default behavior (a --xmlns-as-metadata option?)

[steven-r]

You can even write

# Chapter {epub:type=prologue xmlns:epub="http://www.idpf.org/2007/ops"}

A long time ago ...

at least xmllint doesn't complain here.

Long story short. As I'm coming from docx/odt, I will use the filter option for now (at the same position I patch the epub:type into the flow).

BTW: I like the metadata approach as this can then be use inside metadata files.

[jgm]

Another possible approach would be to munge the attribute names, removing the colon, e.g. represent epub:type as epub.type in the XML representation. This is a bit ugly and could lead to some ambiguities, but it would remove the linting problem.

At the moment, though, I don't think we should do anything. This does not strike me as a big problem, and there is a way to work around it.