docling Improve backend resolution logic

Requested feature

Document conversion currently contains a logic for "guessing" / resolving the backend to use for a given input (ref).

This logic has some limitations, e.g. when working with streams, it relies on the first 8KB to detect the backend to use — which may or may not be enough for a correct detection (e.g. deciding info could only appear at the end of a 10KB stream).

Consider ways to remove these limitations.

One possible high-level approach to examine could be to:

remove the current layer of "guessing" a backend a priori and then committing to that guess, and
instead, keep for each format, e.g. XML, a list of backends to try one after another, until one successfully parses (can have a default list, parametrizable by the user).

Jan 24 '25 13:01 vagenas

It turns out also the filetype library is loading only 8K bytes ref, so this happens also in file inputs.

Jan 29 '25 14:01 dolfim-ibm

As discovered in #542, some MS Office XML archives have the meta file [Content_Types].xml at the end, which is not captured by the 8K bytes signature.

One way of improving the logic could be:

Detect if the file is a zip archive (here filetype should work)
List all the files in there and check if [Content_Types].xml is present
In case, read it and infer the proper file type from it. Since zip archives allow random access, this could be more efficient than reading the whole file.

Jan 29 '25 14:01 dolfim-ibm

Another sample of a word document not detected as such is seen in issue https://github.com/DS4SD/docling/issues/476.

Jan 31 '25 09:01 cau-git

I'm seeing this issue for pptx files where [Content_Types].xml is present at the top, for example, this slide deck, which I've ran zipinfo on to display that [Content_Types].xml does indeed sit at the top as expected, but I've truncated the rest of the zipinfo output to clean this post up. Below that is [Content_Types].xml for the file, and I've also included a [Content_Types].xml of a similar pptx file that does process properly. Happy to help test any fixes.

Archive: name_scrubbed.pptx Zip file size: 4291099 bytes, number of entries: 115 -rw---- 1.0 fat 8628 b- defS 80-Jan-01 00:00 [Content_Types].xml ... 115 files, 4964777 bytes uncompressed, 4274329 bytes compressed: 13.9%

And the subsequent [Content_Types].xml

Finally, I hope this is helpful, here is a [Content_Types].xml for a similar powerpoint that does process properly

Feb 03 '25 15:02 dward4

@dolfim-ibm @cau-git

As discovered in #542, some MS Office XML archives have the meta file [Content_Types].xml at the end, which is not captured by the 8K bytes signature.

One way of improving the logic could be:

Detect if the file is a zip archive (here filetype should work)

List all the files in there and check if [Content_Types].xml is present

In case, read it and infer the proper file type from it. Since zip archives allow random access, this could be more efficient than reading the whole file.

DocumentSteam() also has a name parameter. A more performance efficient algorithm would be to:

Open all documents that are supposed to be ZIP files as ZipFile(). If only one file format as allowlisted, use that back-end. Use the filename extension.
In case more than one ZIP-based file format was allowlisted, check the Central directory file header for an indication of the content type using, e.g., the occurrence of [Content_Types].xml in the return value of .name_list().

The larger issue is: why detect the content type? Besides going against the better to ask forgiveness than permission principle, it costs time every time for the corner case where someone would present a document ’guess what this is ...‘. When using Docling in a web application back-end, the content type (Internet media type) of data is supposed to be specified/detected already by the back-end. E.g., to check against ZIP bombs etc. No need for Docling to take on this responsibility.

Feb 13 '25 08:02 sanmai-NL

There is no doubt the logic has to be fixed and improved, maybe also simplified altogether.

The initial use case which was pretty relevant for us is iterating through a folder of files which were crawled. That is where we learned the file extension is very unreliable. You get .pdf which are just 429 html pages, you get .php which are actually PDF, etc

Feb 13 '25 11:02 dolfim-ibm

You could have a fallback type detection (configurable, default off).

Feb 13 '25 13:02 sanmai-NL

Any updates on this issue?

Apr 02 '25 10:04 yssAI

Facing this issue too with .pptx files

Apr 24 '25 15:04 anuar12

how about google/magika? Detect file content types with deep learning

May 09 '25 03:05 whisper-bye