DS4SD
Docling fails to convert docx document
Bug
Even when I have set OCR=False, it is failing asking to install VLM model
`def _initialize_converter(self) -> None: """Initialize the document converter with appropriate settings.""" if self.optimize_pdf: pdf_pipeline_options = PdfPipelineOptions(ocr_options=EasyOcrOptions()) pdf_pipeline_options.do_ocr = self.enable_ocr pdf_pipeline_options.do_table_structure = True pdf_pipeline_options.table_structure_options.do_cell_matching = True pdf_pipeline_options.do_picture_description = False pdf_pipeline_options.picture_description_options = None
# Use GPU only when OCR is enabled, otherwise use CPU
device = AcceleratorDevice.AUTO if self.enable_ocr else AcceleratorDevice.CPU
cuda_flash = self.enable_ocr
pdf_pipeline_options.accelerator_options = AcceleratorOptions(
num_threads=self.pdf_threads,
device=device,
cuda_use_flash_attention2=cuda_flash
)
else:
pdf_pipeline_options = PdfPipelineOptions()
pdf_pipeline_options.do_picture_description = False
pdf_pipeline_options.do_ocr = False
pdf_pipeline_options.do_table_structure = False
# Create converter instance with separate PDF and image options
self.converter = DoclingConverter(
allowed_formats=[
InputFormat.PDF,
InputFormat.IMAGE,
InputFormat.DOCX,
InputFormat.HTML,
InputFormat.PPTX,
InputFormat.ASCIIDOC,
InputFormat.CSV,
InputFormat.MD,
InputFormat.XLSX
],
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_cls=StandardPdfPipeline,
backend=PyPdfiumDocumentBackend,
pipeline_options=pdf_pipeline_options
),
InputFormat.DOCX: WordFormatOption(
pipeline_cls=SimplePipeline
),
InputFormat.IMAGE: ImageFormatOption(
pipeline_cls=StandardPdfPipeline,
backend=PyPdfiumDocumentBackend,
pipeline_options=pdf_pipeline_options
),
},
)
@file_operation(error_code=ErrorCodes.DOCUMENT_CONVERSION_ERROR)
def convert_to_markdown(self, input_path: Union[str, Path]) -> str:
"""
Convert a single file to markdown format.
Args:
input_path: Path to the input file or URL
Returns:
Markdown content as string
Raises:
ValueError: If the file doesn't exist, has an unsupported extension, or conversion fails
"""
# Handle .txt and .json files directly without using docling converter
if self._is_text_file(input_path):
content = self._read_text_file(input_path)
if content is None:
raise ValueError(f"Failed to read text file: {self._safe_log_path(input_path)}")
return content
# For all other file types, use the docling converter
conv_result = self.converter.convert(input_path)
# Get markdown content
markdown_content = conv_result.document.export_to_markdown()
if not markdown_content:
raise ValueError(f"Conversion returned empty content for {self._safe_log_path(input_path)}")
return markdown_content`
...
Steps to reproduce
File "/app/src/objects_core/utils.py", line 34, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/app/src/vector_store_processor/core/docling_converter.py", line 142, in convert_to_markdown
conv_result = self.converter.convert(input_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pydantic/_internal/_validate_call.py", line 39, in wrapper_function
return wrapper(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pydantic/_internal/_validate_call.py", line 136, in call
res = self.__pydantic_validator__.validate_python(pydantic_core.ArgsKwargs(args, kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/docling/document_converter.py", line 237, in convert
return next(all_res)
^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/docling/document_converter.py", line 260, in convert_all
for conv_res in conv_res_iter:
File "/usr/local/lib/python3.11/site-packages/docling/document_converter.py", line 332, in _convert
for item in map(
File "/usr/local/lib/python3.11/site-packages/docling/document_converter.py", line 379, in _process_document
conv_res = self._execute_pipeline(in_doc, raises_on_error=raises_on_error)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/docling/document_converter.py", line 400, in _execute_pipeline
pipeline = self._get_pipeline(in_doc.format)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/docling/document_converter.py", line 362, in _get_pipeline
self.initialized_pipelines[cache_key] = pipeline_class(
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/docling/pipeline/simple_pipeline.py", line 24, in init
super().__init__(pipeline_options)
File "/usr/local/lib/python3.11/site-packages/docling/pipeline/base_pipeline.py", line 144, in init
picture_description_model := self._get_picture_description_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/docling/pipeline/base_pipeline.py", line 170, in _get_picture_description_model
return factory.create_instance(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/docling/models/factories/base_factory.py", line 59, in create_instance
raise RuntimeError(self._err_msg_on_class_not_found(options.kind))
RuntimeError: No class found with the name 'vlm', known classes are:
same error ...
Docling version
2.58.0 ...
Python version
3.11.9 ...
This is a bug in Docling's pipeline logic: even when picture description is disabled, the pipeline still tries to load the VLM model during DOCX conversion. This happens because the pipeline initialization always calls the picture description model factory, regardless of the do_picture_description flag. If the model isn't installed, you get the "No class found with the name 'vlm'" error.
To work around this, explicitly pass pipeline options with do_picture_description=False and a valid (non-None) picture_description_options object to the DOCX pipeline (WordFormatOption). If you omit pipeline_options, Docling may use defaults that inadvertently enable enrichment features, triggering the model loading bug.
This behavior is not intended—DOCX conversion should not require the VLM model when enrichment features are off. A code fix would be to update the pipeline so it only attempts to load the picture description model if do_picture_description is True.
For now, make sure you pass explicit pipeline options to the DOCX pipeline with all enrichment features disabled to avoid this error.
Reference
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other 
![dosubot[bot] avatar](https://avatars.githubusercontent.com/in/324583?v=4 "Published by a pub.dev verified publisher"){kind=small}
@dosu Can you write a small logic for the same because, this will happen for other formats as well like PPTx, HTMLX, etc
Here's a concise way to prevent VLM (or any enrichment model) from being loaded for DOCX, PPTX, HTML, and similar formats: create a ConvertPipelineOptions instance with all enrichment features disabled, and inject it into each relevant FormatOption. You can reuse the same options object for all formats using SimplePipeline.
from docling.datamodel.pipeline_options import ConvertPipelineOptions, PictureDescriptionApiOptions
from docling.document_converter import (
DocumentConverter, InputFormat,
WordFormatOption, PowerpointFormatOption, HTMLFormatOption,
# ... other FormatOptions as needed
)
# Explicitly disable enrichment features and use a safe picture_description_options
pipeline_options = ConvertPipelineOptions(
do_picture_classification=False,
do_picture_description=False,
enable_remote_services=False,
picture_description_options=PictureDescriptionApiOptions() # avoids VLM
)
converter = DocumentConverter(
allowed_formats=[
InputFormat.DOCX,
InputFormat.PPTX,
InputFormat.HTML,
# ... add other formats as needed
],
format_options={
InputFormat.DOCX: WordFormatOption(pipeline_options=pipeline_options),
InputFormat.PPTX: PowerpointFormatOption(pipeline_options=pipeline_options),
InputFormat.HTML: HTMLFormatOption(pipeline_options=pipeline_options),
# ... add other formats as needed
}
)
This pattern ensures that enrichment features are off and avoids any attempt to load the VLM model for these formats. You can extend this to other formats using SimplePipeline in the same way.
Reference
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
@dosu Now I am getting the below error:
` File "/app/src/objects_core/utils.py", line 34, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/app/src/vector_store_processor/core/docling_converter.py", line 151, in convert_to_markdown
conv_result = self.converter.convert(input_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pydantic/_internal/_validate_call.py", line 39, in wrapper_function
return wrapper(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pydantic/_internal/_validate_call.py", line 136, in call
res = self.__pydantic_validator__.validate_python(pydantic_core.ArgsKwargs(args, kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/docling/document_converter.py", line 237, in convert
return next(all_res)
^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/docling/document_converter.py", line 260, in convert_all
for conv_res in conv_res_iter:
File "/usr/local/lib/python3.11/site-packages/docling/document_converter.py", line 332, in _convert
for item in map(
File "/usr/local/lib/python3.11/site-packages/docling/document_converter.py", line 379, in _process_document
conv_res = self._execute_pipeline(in_doc, raises_on_error=raises_on_error)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/docling/document_converter.py", line 400, in _execute_pipeline
pipeline = self._get_pipeline(in_doc.format)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/docling/document_converter.py", line 362, in _get_pipeline
self.initialized_pipelines[cache_key] = pipeline_class(
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/docling/pipeline/simple_pipeline.py", line 24, in init
super().__init__(pipeline_options)
File "/usr/local/lib/python3.11/site-packages/docling/pipeline/base_pipeline.py", line 144, in init
picture_description_model := self._get_picture_description_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/docling/pipeline/base_pipeline.py", line 170, in _get_picture_description_model
return factory.create_instance(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/docling/models/factories/base_factory.py", line 59, in create_instance
raise RuntimeError(self._err_msg_on_class_not_found(options.kind))
RuntimeError: No class found with the name 'api', known classes are:`
This error happens because Docling always tries to instantiate a picture description model based on the kind in picture_description_options—even if do_picture_description is False. If you set picture_description_options to PictureDescriptionApiOptions(), but the 'api' model isn't registered in your environment, you'll get the "No class found with the name 'api'" error. If you leave it as default, it tries to load 'vlm', which also fails if the VLM model isn't installed.
There is currently no built-in "noop" or "none" kind to bypass this model loading step. The only safe workaround is to ensure that picture_description_options is set to a valid, registered kind for your environment, and do_picture_description is False. If neither 'vlm' nor 'api' is available, there is no out-of-the-box option to fully disable this step—this is a limitation in Docling's pipeline logic right now (source).
If you control your environment, you could register a dummy/no-op model for testing, but otherwise, you may need to patch the pipeline logic to skip model instantiation when do_picture_description is False. For now, there is no configuration-only solution that avoids both errors if neither model is installed or registered (source).
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
@dosu, can you provide me with a logic? I mean, every time you register a small model, like whenever there's an instance of pipeline, download a small model w.r.t. API or VLM, depending on the size
There’s no built-in Docling feature to auto-download and register a small model at pipeline instantiation, but you can add a check before creating your pipeline: if the minimal model (like "smolvlm" or "granitedocling") isn’t present, trigger a download and set the artifacts path so Docling can find it. Here’s a Python pattern you can use:
import os
import subprocess
from pathlib import Path
def ensure_minimal_vlm_model(artifacts_dir="models", model_name="granitedocling"):
# Check if the model folder exists (adjust for your model's expected subdir)
model_path = Path(artifacts_dir) / model_name
if not model_path.exists():
# Download the model using the CLI tool
subprocess.run([
"docling-tools", "models", "download", model_name, "-o", artifacts_dir
], check=True)
# Set the environment variable so Docling finds the model
os.environ["DOCLING_SERVE_ARTIFACTS_PATH"] = str(artifacts_dir)
# Call this before pipeline instantiation
ensure_minimal_vlm_model()
# Now create your pipeline as usual
# ... your Docling pipeline code ...
This ensures the smallest available model is always present and registered before any pipeline runs, avoiding runtime errors if the model is missing. You can swap "granitedocling" for "smolvlm" or another small model as needed. For robust deployments, pre-download models and mount them into your environment rather than relying on runtime downloads, especially in production or containerized setups [source].
If you want to get even fancier, you could inspect the available models and pick the smallest one programmatically, but the above pattern covers the core logic.
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
@dosu You can have this ticket for reference in your future deployment pipeline. Currently, I will downgrade the docling model version, where this is not implemented. Could you share the version which is suitable for the same.
Based on user reports and issue discussions, Docling version 2.37.0 is the last known stable release before the forced VLM/API model loading bug was introduced. Downgrading to 2.37.0 should allow DOCX, PPTX, and HTML conversions without requiring VLM or API models when enrichment features are disabled. The bug appears in versions 2.43.0 and later, so avoid those until a fix is released.
See this issue thread for confirmation.
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
Could you please provide the input document and Python code as attachment files for reproducing the issue?
