pymupdf
ImportError: libmupdf.so.24.4: cannot open shared object file: No such file or directory
Description of the bug
I am attempting to install PyMuPDF on a Docker container running on an Apple Silicon. The installation is successful, but it is impossible to import the library due to a missing shared object.
How to reproduce the bug
!pip install --upgrade --force-reinstall --no-deps --no-cache-dir PyMuPDF
Collecting PyMuPDF Downloading PyMuPDF-1.24.7-cp311-none-manylinux2014_aarch64.whl.metadata (3.4 kB) Downloading PyMuPDF-1.24.7-cp311-none-manylinux2014_aarch64.whl (3.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 4.6 MB/s eta 0:00:00a 0:00:01 Installing collected packages: PyMuPDF Attempting uninstall: PyMuPDF Found existing installation: PyMuPDF 1.24.7 Uninstalling PyMuPDF-1.24.7: Successfully uninstalled PyMuPDF-1.24.7 Successfully installed PyMuPDF-1.24.7
import fitz # PyMuPDF
ImportError Traceback (most recent call last) Cell In[12], line 4 ----> 4 import fitz # PyMuPDF
File /opt/conda/lib/python3.11/site-packages/fitz/init.py:2 1 # pylint: disable=wildcard-import,unused-import,unused-wildcard-import ----> 2 from pymupdf import * 3 from pymupdf import _as_fz_document 4 from pymupdf import _as_fz_page
File /opt/conda/lib/python3.11/site-packages/pymupdf/init.py:29
26 import weakref
27 import zipfile
---> 29 from . import extra
32 # Set up g_out_log and g_out_message from environment variables.
33 #
34 # PYMUPDF_MESSAGE controls the destination of user messages (the message()
(...)
43 # stdout.
44 #
46 def _set_stream(name, default):
File /opt/conda/lib/python3.11/site-packages/pymupdf/extra.py:10 8 # Import the low-level C/C++ module 9 if package or "." in name: ---> 10 from . import _extra 11 else: 12 import _extra
ImportError: libmupdf.so.24.4: cannot open shared object file: No such file or directory
The machine is an Apple Silicon M2 (Ventura 13.5). This error happens when using JupyterLab 4.0.11 running inside Docker 24.0.7.
PyMuPDF version
1.24.7
Operating system
MacOS
Python version
3.11
Please retry without --no-deps - PyMuPDF requires a second wheel called PyMuPDFb.
I'm seeing the same thing on my Apple Silicon Macbook Pro with a fairly simple Dockerfile. Seems like the dependency is not available or not installed when running on an arm64 image?
Dockerfile:
FROM python:slim
ENV APP_HOME=/home/app
ENV PIP_DISABLE_VERSION_CHECK 1
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
WORKDIR ${APP_HOME}
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY ./ .
...
Error:
+ python /home/app/django_troop/manage.py collectstatic --noinput
145 static files copied to '/home/app/django_troop/staticfiles', 415 post-processed.
+ python /home/app/django_troop/manage.py migrate
Traceback (most recent call last):
File "/home/app/django_troop/manage.py", line 22, in <module>
main()
File "/home/app/django_troop/manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.12/site-packages/django/core/management/base.py", line 413, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.12/site-packages/django/core/management/base.py", line 459, in execute
output = self.handle(*args, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/core/management/base.py", line 107, in wrapper
res = handle_func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/core/management/commands/migrate.py", line 100, in handle
self.check(databases=[database])
File "/usr/local/lib/python3.12/site-packages/django/core/management/base.py", line 486, in check
all_issues = checks.run_checks(
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/core/checks/registry.py", line 88, in run_checks
new_errors = check(app_configs=app_configs, databases=databases)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/core/checks/urls.py", line 14, in check_url_config
return check_resolver(resolver)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/core/checks/urls.py", line 24, in check_resolver
return check_method()
^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/urls/resolvers.py", line 519, in check
for pattern in self.url_patterns:
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/utils/functional.py", line 47, in __get__
res = instance.__dict__[self.name] = self.func(instance)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/urls/resolvers.py", line 738, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/utils/functional.py", line 47, in __get__
res = instance.__dict__[self.name] = self.func(instance)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/urls/resolvers.py", line 731, in urlconf_module
return import_module(self.urlconf_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 995, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/home/app/django_troop/django_troop/urls.py", line 26, in <module>
path("imports/", include("imports.urls")),
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/urls/conf.py", line 39, in include
urlconf_module = import_module(urlconf_module)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 995, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/home/app/django_troop/imports/urls.py", line 3, in <module>
from . import views
File "/home/app/django_troop/imports/views.py", line 10, in <module>
from imports.tasks import import_scouts_scoutbook, import_scouts_troopmaster
File "/home/app/django_troop/imports/tasks.py", line 6, in <module>
from scouts.load_scouts import (
File "/home/app/django_troop/scouts/load_scouts.py", line 13, in <module>
from signoffs.load_signoffs import start_a_badge
File "/home/app/django_troop/signoffs/load_signoffs.py", line 13, in <module>
from tm_parser import Parser
File "/usr/local/lib/python3.12/site-packages/tm_parser/__init__.py", line 3, in <module>
from .parser import Parser, dump_string
File "/usr/local/lib/python3.12/site-packages/tm_parser/parser.py", line 3, in <module>
from .pdf import parse_file
File "/usr/local/lib/python3.12/site-packages/tm_parser/pdf.py", line 12, in <module>
import fitz
File "/usr/local/lib/python3.12/site-packages/fitz/__init__.py", line 2, in <module>
from pymupdf import *
File "/usr/local/lib/python3.12/site-packages/pymupdf/__init__.py", line 29, in <module>
from . import extra
File "/usr/local/lib/python3.12/site-packages/pymupdf/extra.py", line 10, in <module>
from . import _extra
ImportError: libmupdf.so.24.4: cannot open shared object file: No such file or directory
I still have the same error without
--no-deps
Could you paste the full output from the new pip install command?
I'm seeing the same thing on my Apple Silicon Macbook Pro with a fairly simple Dockerfile. Seems like the dependency is not available or not installed when running on an arm64 image?
File "/usr/local/lib/python3.12/site-packages/tm_parser/pdf.py", line 12, in
import fitz File "/usr/local/lib/python3.12/site-packages/fitz/init.py", line 2, in from pymupdf import * File "/usr/local/lib/python3.12/site-packages/pymupdf/init.py", line 29, in from . import extra File "/usr/local/lib/python3.12/site-packages/pymupdf/extra.py", line 10, in from . import _extra ImportError: libmupdf.so.24.4: cannot open shared object file: No such file or directory
Could you show the contents of the /usr/local/lib/python3.12/site-packages/ directory, e.g. with ls -l /usr/local/lib/python3.12/site-packages/?
Collecting PyMuPDF
Downloading PyMuPDF-1.24.7-cp311-none-manylinux2014_aarch64.whl.metadata (3.4 kB)
Collecting PyMuPDFb==1.24.6 (from PyMuPDF)
Downloading PyMuPDFb-1.24.6.tar.gz (37 kB)
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Downloading PyMuPDF-1.24.7-cp311-none-manylinux2014_aarch64.whl (3.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 2.4 MB/s eta 0:00:0000:0100:01
Building wheels for collected packages: PyMuPDFb
Building wheel for PyMuPDFb (pyproject.toml) ... done
Created wheel for PyMuPDFb: filename=PyMuPDFb-1.24.6-py3-none-linux_aarch64.whl size=1030 sha256=5280f0e0e0bf661125c18711705f7fd65db8cb88d48720cd8c4a5175760a9744
Stored in directory: /tmp/pip-ephem-wheel-cache-rvhqhrix/wheels/da/ea/6d/38ad8fe6eca7b00a9bdebcfcd40354ca1b2afa903f5f923312
Successfully built PyMuPDFb
Installing collected packages: PyMuPDFb, PyMuPDF
Successfully installed PyMuPDF-1.24.7 PyMuPDFb-1.24.6
The libraries are in the directory : /opt/conda/lib/python3.11/site-packages
(base) jovyan@e7de1aac2531:/opt/conda/lib/python3.11/site-packages$ ls -al *pdf*
total 15680
drwxr-sr-x 3 jovyan users 4096 Jul 1 14:56 .
drwsrwsr-x 1 jovyan users 4096 Jul 1 14:56 ..
-rw-r--r-- 1 jovyan users 7568 Jul 1 14:56 _apply_pages.py
-rwxr-xr-x 1 jovyan users 249368 Jul 1 14:56 _extra.cpython-311-aarch64-linux-gnu.so
-rw-r--r-- 1 jovyan users 6408 Jul 1 14:56 extra.py
-rw-r--r-- 1 jovyan users 795852 Jul 1 14:56 __init__.py
-rw-r--r-- 1 jovyan users 41110 Jul 1 14:56 __main__.py
-rw-r--r-- 1 jovyan users 2218494 Jul 1 14:56 mupdf.py
-rwxr-xr-x 1 jovyan users 12426152 Jul 1 14:56 _mupdf.so
drwxr-sr-x 2 jovyan users 4096 Jul 1 14:56 __pycache__
-rw-r--r-- 1 jovyan users 64 Jul 1 14:56 pymupdf.py
-rw-r--r-- 1 jovyan users 81413 Jul 1 14:56 table.py
-rw-r--r-- 1 jovyan users 194265 Jul 1 14:56 utils.py
I filtered the list because the list is very long and probably not linked to this issue
Thanks for the extra information.
It looks like the PyMuPDFb-1.24.6 linux/aarch64 wheel is missing on pypi.org, and unfortunately pip's attempt to build from sdist deliberately produces no files. [This is because building PyMuPDF from sdist does not require a PyMuPDFb wheel, because PyMuPDFb wheels are used only to reduce storage requirements on pypi.org.]
I have the original PyMuPDFb-1.24.6 linux/aarch64 wheel, which i've uploaded to:
http://ghostscript.com/~julian/PyMuPDFb-1.24.6-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl
Could you try installing this with pip before installing PyMuPFD? for example:
wget http://ghostscript.com/~julian/PyMuPDFb-1.24.6-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl
pip install ./PyMuPDFb-1.24.6-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl
pip install pymupdf
I was experiencing a similar issue, but it is now resolved as mentioned above. Your help was invaluable. Thank you very much.
I was experiencing a similar issue, but it is now resolved as mentioned above. Your help was invaluable. Thank you very much.
Thanks for reporting this. I've just uploaded the wheel to pypi.org, so pip install PyMuPDF should now work on Linux/aarch64.
If someone could check it's working, that would be appreciated.
I still have an issue but different. Thanks
Collecting PyMuPDF
Downloading PyMuPDF-1.24.7-cp311-none-manylinux2014_aarch64.whl.metadata (3.4 kB)
Collecting PyMuPDFb==1.24.6 (from PyMuPDF)
Downloading PyMuPDFb-1.24.6-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl.metadata (1.4 kB)
Downloading PyMuPDF-1.24.7-cp311-none-manylinux2014_aarch64.whl (3.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 2.0 MB/s eta 0:00:0000:0100:01
Downloading PyMuPDFb-1.24.6-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl (15.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.5/15.5 MB 4.4 MB/s eta 0:00:0000:0100:01
Installing collected packages: PyMuPDFb, PyMuPDF
Attempting uninstall: PyMuPDFb
Found existing installation: PyMuPDFb 1.24.6
Uninstalling PyMuPDFb-1.24.6:
Successfully uninstalled PyMuPDFb-1.24.6
Successfully installed PyMuPDF-1.24.7 PyMuPDFb-1.24.6
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
Cell In[22], line 4
2 import pprint
3 import json
----> 4 import fitz # PyMuPDF
5 import base64
6 import openai
File /opt/conda/lib/python3.11/site-packages/fitz/__init__.py:2
1 # pylint: disable=wildcard-import,unused-import,unused-wildcard-import
----> 2 from pymupdf import *
3 from pymupdf import _as_fz_document
4 from pymupdf import _as_fz_page
File /opt/conda/lib/python3.11/site-packages/pymupdf/__init__.py:29
26 import weakref
27 import zipfile
---> 29 from . import extra
32 # Set up g_out_log and g_out_message from environment variables.
33 #
34 # PYMUPDF_MESSAGE controls the destination of user messages (the `message()`
(...)
43 # stdout.
44 #
46 def _set_stream(name, default):
File /opt/conda/lib/python3.11/site-packages/pymupdf/extra.py:10
8 # Import the low-level C/C++ module
9 if __package__ or "." in __name__:
---> 10 from . import _extra
11 else:
12 import _extra
ImportError: /opt/conda/lib/python3.11/site-packages/pymupdf/libmupdf.so.24.4: undefined symbol: fz_pclm_write_options_usage
I'm using Apple Silicon (M2) and adding the following environment variable to my .env file works:
DOCKER_DEFAULT_PLATFORM=linux/amd64
See this Stack Overflow post for more: Forcing docker to use linux/amd64 platform by default on macOS
However, my Docker image's performance is significantly slower due to emulating amd64 instructions in an ARM architecture.
I still have an issue but different. Thanks
ImportError: /opt/conda/lib/python3.11/site-packages/pymupdf/libmupdf.so.24.4: undefined symbol: fz_pclm_write_options_usage
Just in case conda is effecting things, or some old files are somehow interfering, could you try installing PyMuPDF into a clean virtual environment using Python's built-in venv using the following sequence of commands?
rm -r pymupdf-test-venv || true
python3 -m venv pymupdf-test-venv
. pymupdf-test-venv/bin/activate
pip install PyMuPDF
python -c 'import pymupdf; print("ok")'
So it works in venv but not in Jupyter labs on Apple Silicon
(base) u@
server ~ % docker exec -it e7de1aac2531 bash
(base) jovyan@e7de1aac2531:~$ rm -r pymupdf-test-venv || true
rm: cannot remove 'pymupdf-test-venv': No such file or directory
(base) jovyan@e7de1aac2531:~$ python3 -m venv pymupdf-test-venv
(base) jovyan@e7de1aac2531:~$ . pymupdf-test-venv/bin/activate
(pymupdf-test-venv) (base) jovyan@e7de1aac2531:~$ pip install PyMuPDF
Collecting PyMuPDF
Obtaining dependency information for PyMuPDF from https://files.pythonhosted.org/packages/76/a9/8e57a3355cad2256d4eb56420e8cd0e59655cc7d7a35553afab8a4f7d75a/PyMuPDF-1.24.7-cp311-none-manylinux2014_aarch64.whl.metadata
Downloading PyMuPDF-1.24.7-cp311-none-manylinux2014_aarch64.whl.metadata (3.4 kB)
Collecting PyMuPDFb==1.24.6 (from PyMuPDF)
Obtaining dependency information for PyMuPDFb==1.24.6 from https://files.pythonhosted.org/packages/4c/77/942d30c00796966f24e0bc7437820135c0a2168939d987079ef82f3fb362/PyMuPDFb-1.24.6-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl.metadata
Downloading PyMuPDFb-1.24.6-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl.metadata (1.4 kB)
Downloading PyMuPDF-1.24.7-cp311-none-manylinux2014_aarch64.whl (3.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 6.0 MB/s eta 0:00:00
Downloading PyMuPDFb-1.24.6-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl (15.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.5/15.5 MB 11.2 MB/s eta 0:00:00
Installing collected packages: PyMuPDFb, PyMuPDF
Successfully installed PyMuPDF-1.24.7 PyMuPDFb-1.24.6
[notice] A new release of pip is available: 23.2.1 -> 24.1.1
[notice] To update, run: pip install --upgrade pip
(pymupdf-test-venv) (base) jovyan@e7de1aac2531:~$ python -c 'import pymupdf; print("ok")'
ok
I would like to avoid having to use work around like : DOCKER_DEFAULT_PLATFORM=linux/amd64 I confirm it's working using linux/amd64 plateform
Thanks for this information.
So it looks like the problem is with with Jupyter labs on Apple Silicon, not PyMuPDF.
I don't use Jupyter labs so the best thing now is probably for you to ask the Jupyter labs people about the problem.
I'll close this issue soon.