opencv
ModuleNotFoundError: No module named 'distutils' in latest release
Expected behaviour
distutils is deprecated and should not be used in the module.
Actual behaviour
Installing latest version of opencv_python runs into an exception because of the distutils reference.
Call Stack
ERROR: Exception:
Traceback (most recent call last):
File "...Python312\Lib\site-packages\pip_internal\cli\base_command.py", line 180, in exc_logging_wrapper
status = run_func(*args)
^^^^^^^^^^^^^^^
File "...Python312\Lib\site-packages\pip_internal\cli\req_command.py", line 245, in wrapper
return func(self, options, args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "...Python312\Lib\site-packages\pip_internal\commands\install.py", line 377, in run
requirement_set = resolver.resolve(
^^^^^^^^^^^^^^^^^
File "...Python312\Lib\site-packages\pip_internal\resolution\resolvelib\resolver.py", line 95, in resolve
result = self._result = resolver.resolve(
^^^^^^^^^^^^^^^^^
File "...Python312\Lib\site-packages\pip_vendor\resolvelib\resolvers.py", line 546, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "...Python312\Lib\site-packages\pip_vendor\resolvelib\resolvers.py", line 397, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "...Python312\Lib\site-packages\pip_vendor\resolvelib\resolvers.py", line 173, in _add_to_criteria
if not criterion.candidates:
^^^^^^^^^^^^^^^^^^^^
File "...Python312\Lib\site-packages\pip_vendor\resolvelib\structs.py", line 156, in bool
return bool(self._sequence)
^^^^^^^^^^^^^^^^^^^^
File "...Python312\Lib\site-packages\pip_internal\resolution\resolvelib\found_candidates.py", line 155, in bool
return any(self)
^^^^^^^^^
File "...Python312\Lib\site-packages\pip_internal\resolution\resolvelib\found_candidates.py", line 143, in
Same issue in our pipelines: https://github.com/microsoft/torchgeo/actions/runs/9356814161/job/25755818665?pr=2103
distutils was removed in Python 3.12
- https://docs.python.org/3/whatsnew/3.12.html#summary-release-highlights
Would also be good to get the source build working in Python 3.12+ even if macOS arm64 wheels are available. May be as simple as removing the pin to older setuptools versions?
MacOS ARM wheels are now available, but as mentioned the sdist should probably still be fixed
- https://github.com/opencv/opencv-python/issues/994#issuecomment-2147363545
Related to this issue, the python:alpine image includes the updated setuptools which resolves the missing distutils module but installation of opencv-python still fails.
For context, trying to build my own package which includes opencv-python as a dependency, but I am having issues installing opencv-python using pip in alpine 3.20 (python3.12).
Steps to reproduce using simplified dockerfile and no source code
FROM python:alpine
RUN pip3 install opencv-python
then:
$ docker build -t opencv-python .
[+] Building 10.8s (6/6) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 131B 0.0s
=> [internal] load metadata for docker.io/library/python:alpine 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [1/3] FROM docker.io/library/python:alpine 0.0s
=> CACHED [2/3] RUN pip3 install --upgrade setuptools pip 0.0s
=> ERROR [3/3] RUN pip3 install opencv-python 10.8s
------
> [3/3] RUN pip3 install opencv-python:
0.669 Collecting opencv-python
0.939 Downloading opencv-python-4.10.0.84.tar.gz (95.1 MB)
3.129 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 95.1/95.1 MB 27.7 MB/s eta 0:00:00
5.320 Installing build dependencies: started
10.43 Installing build dependencies: finished with status 'done'
10.44 Getting requirements to build wheel: started
10.51 Getting requirements to build wheel: finished with status 'error'
10.51 error: subprocess-exited-with-error
10.51
10.51 × Getting requirements to build wheel did not run successfully.
10.51 │ exit code: 1
10.51 ╰─> [33 lines of output]
10.51 Traceback (most recent call last):
10.51 File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
10.51 main()
10.51 File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
10.51 json_out['return_val'] = hook(**hook_input['kwargs'])
10.51 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
10.51 File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 112, in get_requires_for_build_wheel
10.51 backend = _build_backend()
10.51 ^^^^^^^^^^^^^^^^
10.51 File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
10.51 obj = import_module(mod_path)
10.51 ^^^^^^^^^^^^^^^^^^^^^^^
10.51 File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
10.51 return _bootstrap._gcd_import(name[level:], package, level)
10.51 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
10.51 File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
10.51 File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
10.51 File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
10.51 File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
10.51 File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
10.51 File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
10.51 File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
10.51 File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
10.51 File "<frozen importlib._bootstrap_external>", line 995, in exec_module
10.51 File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
10.51 File "/tmp/pip-build-env-rzx3tyxe/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 16, in <module>
10.51 import setuptools.version
10.51 File "/tmp/pip-build-env-rzx3tyxe/overlay/lib/python3.12/site-packages/setuptools/version.py", line 1, in <module>
10.51 import pkg_resources
10.51 File "/tmp/pip-build-env-rzx3tyxe/overlay/lib/python3.12/site-packages/pkg_resources/__init__.py", line 2172, in <module>
10.51 register_finder(pkgutil.ImpImporter, find_on_path)
10.51 ^^^^^^^^^^^^^^^^^^^
10.51 AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
10.51 [end of output]
10.51
10.51 note: This error originates from a subprocess, and is likely not a problem with pip.
10.51 error: subprocess-exited-with-error
10.51
10.51 × Getting requirements to build wheel did not run successfully.
10.51 │ exit code: 1
10.51 ╰─> See above for output.
10.51
10.51 note: This error originates from a subprocess, and is likely not a problem with pip.
------
Dockerfile:4
--------------------
2 |
3 | RUN pip3 install --upgrade setuptools pip
4 | >>> RUN pip3 install opencv-python
5 |
6 |
--------------------
ERROR: failed to solve: process "/bin/sh -c pip3 install opencv-python" did not complete successfully: exit code: 1
A possible workaround is to install opencv-python from the alpine repository
FROM python:alpine
RUN apk add py3-opencv
but the package is not available to the runtime python distribution (not included in the output of pip list).
$ docker run --rm -it opencv-python pip list
Package Version
---------- -------
pip 24.0
setuptools 70.3.0
wheel 0.43.0
@rickgamez
File "...Python312\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 77, in build_backend [...] File "...Python312\Lib\importlib_init.py", line 90, in import_module [...] File "...overlay\Lib\site-packages\setuptools_init.py", line 10, in import distutils.core ModuleNotFoundError: No module named 'distutils'
Have you tried updating pip, it used to vendor setuptools, which is of course an outdated vendor.
Same issue in our pipelines: microsoft/torchgeo/actions/runs/9356814161/job/25755818665?pr=2103
setuptools/init.py", line 10, in
import distutils.core ModuleNotFoundError: No module named 'distutils'
@adamjstewart
Have you tried with the environment variable SETUPTOOLS_USE_DISTUTILS NOT set to stdlib (the other intended valid value is local, but the implementation details work in a way that any value that's not stdlib should cause to prefer using the vendored version)
Have you tried updating setuptools ? I think it should fallback to its vendored version even with SETUPTOOLS_USE_DISTUTILS=stdlib
In your own code, make sure you always import setuptools before distutils.
@1riggs Lastly, this PR should also be merged: https://github.com/opencv/opencv-python/pull/1012 , see https://github.com/pypa/setuptools/issues/3935#issuecomment-2074086030
~Supporting newer setuptools is great, but is there a reason we don't just remove all references to distutils? The setuptools copy of distutils is also planned to be deprecated, so we're just kicking the can down the road instead of solving the root problem.~
Oh wait, opencv doesn't use distutils, setuptools does. So yes, I think supporting newer setuptools really is the solution here. Thanks for the fix @bryankaplan and the pointer @Avasam!
I've also encountered this issue today when trying to install opencv on a Windows arm64 machine. So either the build process should be fixed or we could also supply wheels for Windows arm64 architecture.
@CanePlayz I opened #1012 almost four months ago, and it's just sitting there unmerged. Someone reported that it didn't work under MacOS. Can you please test whether it works on your Windows arm64 machine?
Sure, I'll check tomorrow. Since I'm new to this, just to make sure: I'll just have to run python setup.py bdist_wheel to build the package locally?