Dual column text layout lost

[piertdewitt-glitch]

Hi,

Please forgive me if I write something that doesn't make sense, I'm new to this and still learning.

Trying to feed an LLM with single invoices (through n8n) so I can ask questions about it. For clarity, I'm using n8n's AI nodes which does not support uploading pdf files and ask questions about it.
The issue I run into is that some invoices have multiple two-column tables on them. For example:
Invoice number:__123456 ____________Customer: some company
Invoice date:_____01-01-2025 ________Customer number: 654321
Due date:________31-01-2025 ________Customer tax ID: 321654

Using docling-serve's UI to convert the pdf will correctly label each text elemen'ts position in docling-json (verifyable through the docling render tab). Also, the doctags format seems to correctly identify each text position within loc_xxx tags. However, in markdown, it looks something like this:
Invoice number:
Some company
01-01-2025
123456
Customer number
Due date:
321654

If I use the docling-json or doctags in a prompt the LLM isn't able to correctly comprehend the format and it's not able to answer much questions about it. I think the markdown would work, except for invoices with columns (which are quite a few). I have been watching a lot of youtube videos and reading a lot of articles, but I can't find/understand how to feed the text from invoices, including each element's location, to an LLM. It needs to understand the locations of the text elements to understand that 123456 is the invoice number.
I tried almost all possible options in the web-ui (different ocr, fast and accurate tables, the vlm pipeline etc etc).
Maybe my assumptions are wrong?
Assumption: an LLM can only work with markdown if you want it to 'see' the document like we see it.
Only tables that are detected, are represented in markdown as objects where elements have a specific location.
Markdown ignores location tags for text elements.
There is a conspiracy against me to never mention how to get documents with element location data to an LLM (directly or through a RAG). This is only discussed in secret forums which are hidden from me and google search.

Assuming you just can't use the sjon or doctag location tags with LLM's, is there a workaround to this? Maybe there's a way to convert those location tags from the json to tables using an intermediary step and output the result to markdown? It's just invoices, I don't need the images or anything, I just need all the invoice's text and locations in such a way that the AI can understand the invoice like we see it. It does work if I upload the pdf to chatgpt or gemini for example, so there must be a way right? :-)

[dosubot]

Hi @piertdewitt-glitch! I'm Dosu and I’m helping the docling-serve team.

You're right—when Docling exports to markdown, it only preserves table structure for content that was explicitly detected as a table. Any other text, even if it has location tags or bounding boxes in the JSON or doctags, gets flattened into a single column of text in markdown, so dual-column layouts are lost. This is a limitation of markdown export and not something you can fix just by toggling options in the UI or pipeline [source].

Location tags and bounding boxes are preserved in docling-json and doctags, but markdown serialization ignores them unless the content is recognized as a table. If the table detection step misses a two-column region, it won't show up as a table in markdown [source].

There’s no built-in feature to automatically convert arbitrary location-tagged text blocks (like label-value pairs that aren’t detected as tables) into markdown tables. To get around this, you’d need to postprocess the docling-json or doctags: write a script that groups text elements by their bounding box coordinates to infer columns, then output those as markdown tables. This is how you can reconstruct the structure for LLM input [source].

Alternatively, exporting to HTML preserves layout better than markdown, or you can work directly with the Docling JSON to reconstruct tables programmatically. For LLMs, markdown is convenient but limited for spatial layouts; if you want the LLM to "see" the document as you do, you need to ensure the structure is present in the text you feed it—either by improving table detection, using HTML, or custom postprocessing [source].

If you want to automate this, look into grouping text elements by their x/y coordinates in the JSON, then outputting them as markdown tables. This is a common workaround for invoices and forms with columnar layouts that aren’t always detected as tables.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[piertdewitt-glitch]

@dosu your reference to docling-project/docling#964 looks interesting. In that post, user vagenas said:
"capturing the layout in the chunking itself, by using native Docling chunkers (which do document-based chunking)"
While no further details are provided, could you please verify:
The native Docling chunkers will correctly preserve the locations of text that is not enclosed in table like tags (so my two column text example, where the location is lost after converting to markdown, will correctly be preserved when using native Docling chunkers).
So if this would be the case, if I'd convert the pdf to docling(json) and use the native Docling chunkers to feed a RAG, any AI would then be able to tell me what the invoice number is?

[dosubot]

Yes, native Docling chunkers do preserve the location metadata (like bounding boxes and coordinates) for all text elements—including non-table, multi-column layouts—in the docling-json output. This means that if you convert a PDF to docling-json, the spatial information for each text segment is retained, regardless of whether it's part of a detected table or just a two-column block.

This preserved layout data can be used downstream in a RAG or AI workflow to reconstruct the original structure and accurately extract fields like invoice numbers, even if markdown export loses this structure. The key is to process the docling-json directly and leverage the location metadata to group or align elements as needed for your use case. The chunkers themselves do not discard or flatten this information—layout is only lost if you later export to markdown without custom postprocessing. So, your approach is correct: working with docling-json and native chunkers enables precise extraction from complex layouts like invoices, as long as your downstream logic or model can utilize the coordinates appropriately for reconstruction or extraction [source].

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[piertdewitt-glitch]

@dosu considering only invoices are to be stored in a RAG, and these are usually small files, which native docling chunker would work best to preserve the document layout and makes sure the LLM will understand the layout of the document when asking questions about a specific invoice?

[dosubot]

For invoices, the HierarchicalChunker is the best native Docling chunker to preserve document layout and structure for RAG and LLM use. It segments content based on logical and visual structure—like headings, tables, and grouped fields—while retaining metadata about each chunk, so spatial and contextual relationships are maintained as long as the extraction phase correctly models the invoice structure. If you need to control chunk size for LLM input limits, use HybridChunker, which adds token-based splitting on top of the same structure-aware logic. The key is that both chunkers preserve layout and metadata in each chunk, making them suitable for structured documents like invoices—provided the initial extraction captures the layout accurately. PageChunker, by contrast, only splits by page and is less granular for layout preservation in small files like invoices.
HierarchicalChunker source | HybridChunker source

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[piertdewitt-glitch]

@dosu i used the docling-serve/v1/chunk/hybrid/source endpoint to chunk a pdf file. I used the hybrid chunker and set the 'to_formats' to json. However, in the resulting chunk output, i do not see any layout information other than "headings". Should it not provide the bounding box and table cell information in its output?

[dosubot]

The chunked JSON output from /v1/chunk/hybrid/source typically only includes high-level metadata like headings and references to document items, not detailed layout info such as bounding boxes or table cell data. Even though recent updates in docling-core added richer table cell serialization (including bounding boxes and spans) to the underlying DoclingDocument format and serializers, the chunking API in docling-serve does not expose this layout metadata by default in its chunk output—only what is modeled in the chunk result (like headings, text, and doc_items references) is included. This is why you only see headings and not bounding boxes or table cell info in your output, even with 'to_formats' set to json [source].

If you need bounding box or table cell metadata, you’ll need to extract it directly from the raw DoclingDocument JSON (not the chunked output), or postprocess the doc_items references in the chunk output to look up their full metadata in the original document. Recent improvements in docling-core (RichTableCell, group handling) mean the data is there, but the API response model doesn’t surface it yet [source].

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}