connextdds-py
connextdds-py copied to clipboard
rticommunity
Building on Debian 12 x86_64 with Python 3.11.2 fails
There seems to an issue with pybind11 and/or PyFrameObject.
Following the instructions clone (main branch) and python3 configure.py x64Linux4gcc7.3.0 execute successfully.
But when executing
pip wheel .
the result is the following: (maybe it could be related to (https://github.com/sirfz/tesserocr/issues/298)[Python3.11 support #298]
Processing /home/tkrueger/projects/connextdds-py
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: rti.connext
Building wheel for rti.connext (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for rti.connext (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [152 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-311
creating build/lib.linux-x86_64-cpython-311/rti
copying rti_pkg/__init__.py -> build/lib.linux-x86_64-cpython-311/rti
copying rti_pkg/idl.py -> build/lib.linux-x86_64-cpython-311/rti
copying rti_pkg/asyncio.py -> build/lib.linux-x86_64-cpython-311/rti
creating build/lib.linux-x86_64-cpython-311/rti/logging
copying rti_pkg/logging/__init__.py -> build/lib.linux-x86_64-cpython-311/rti/logging
copying rti_pkg/logging/handler.py -> build/lib.linux-x86_64-cpython-311/rti/logging
creating build/lib.linux-x86_64-cpython-311/rti/request
copying rti_pkg/request/_util_async.py -> build/lib.linux-x86_64-cpython-311/rti/request
copying rti_pkg/request/_async.py -> build/lib.linux-x86_64-cpython-311/rti/request
copying rti_pkg/request/__init__.py -> build/lib.linux-x86_64-cpython-311/rti/request
copying rti_pkg/request/_util.py -> build/lib.linux-x86_64-cpython-311/rti/request
copying rti_pkg/request/_basic.py -> build/lib.linux-x86_64-cpython-311/rti/request
creating build/lib.linux-x86_64-cpython-311/rti/idl_impl
copying rti_pkg/idl_impl/unions.py -> build/lib.linux-x86_64-cpython-311/rti/idl_impl
copying rti_pkg/idl_impl/__init__.py -> build/lib.linux-x86_64-cpython-311/rti/idl_impl
copying rti_pkg/idl_impl/decorators.py -> build/lib.linux-x86_64-cpython-311/rti/idl_impl
copying rti_pkg/idl_impl/csequence.py -> build/lib.linux-x86_64-cpython-311/rti/idl_impl
copying rti_pkg/idl_impl/type_plugin.py -> build/lib.linux-x86_64-cpython-311/rti/idl_impl
copying rti_pkg/idl_impl/type_hints.py -> build/lib.linux-x86_64-cpython-311/rti/idl_impl
copying rti_pkg/idl_impl/sample_interpreter.py -> build/lib.linux-x86_64-cpython-311/rti/idl_impl
copying rti_pkg/idl_impl/type_utils.py -> build/lib.linux-x86_64-cpython-311/rti/idl_impl
copying rti_pkg/idl_impl/annotations.py -> build/lib.linux-x86_64-cpython-311/rti/idl_impl
copying rti_pkg/idl_impl/test_utils.py -> build/lib.linux-x86_64-cpython-311/rti/idl_impl
copying rti_pkg/idl_impl/reflection_utils.py -> build/lib.linux-x86_64-cpython-311/rti/idl_impl
creating build/lib.linux-x86_64-cpython-311/rti/types
copying rti_pkg/types/builtin.py -> build/lib.linux-x86_64-cpython-311/rti/types
copying rti_pkg/types/__init__.py -> build/lib.linux-x86_64-cpython-311/rti/types
copying rti_pkg/types/type_registry.py -> build/lib.linux-x86_64-cpython-311/rti/types
running egg_info
writing rti.connext.egg-info/PKG-INFO
writing dependency_links to rti.connext.egg-info/dependency_links.txt
writing requirements to rti.connext.egg-info/requires.txt
writing top-level names to rti.connext.egg-info/top_level.txt
reading manifest file 'rti.connext.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'rti.connext.egg-info/SOURCES.txt'
running build_ext
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test HAVE_FLAG_STD_CXX17
-- Performing Test HAVE_FLAG_STD_CXX17 - Success
-- Performing Test HAVE_FLAG_STD_CXX14
-- Performing Test HAVE_FLAG_STD_CXX14 - Success
-- RTI Connext DDS installation directory: /opt/rti_connext_dds
-- RTI Connext DDS architecture: x64Linux4gcc7.3.0
-- Found RTIConnextDDS: /opt/rti_connext_dds (found suitable version "7.1.0.0", minimum required is "7.1.0") found components: core
-- Found Python3: /usr/bin/python3.11 (found suitable exact version "3.11.2") found components: Interpreter Development.Module
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Found pybind11: /tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/pybind11/include (found version "2.9.0")
-- RTI Connext DDS installation directory: /opt/rti_connext_dds
-- RTI Connext DDS architecture: x64Linux4gcc7.3.0
-- Found RTIConnextDDS: /opt/rti_connext_dds (found suitable version "7.1.0.0", minimum required is "7.1.0") found components: core distributed_logger
-- Found Python3: /usr/bin/python3.11 (found suitable exact version "3.11.2") found components: Interpreter Development.Module
-- Found pybind11: /tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/pybind11/include (found version "2.9.0")
-- RTI Connext DDS installation directory: /opt/rti_connext_dds
-- RTI Connext DDS architecture: x64Linux4gcc7.3.0
-- Found RTIConnextDDS: /opt/rti_connext_dds (found suitable version "7.1.0.0", minimum required is "7.1.0") found components: core messaging_api
-- Found pybind11: /tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/pybind11/include (found version "2.9.0")
-- Configuring done (5.7s)
-- Generating done (0.1s)
-- Build files have been written to: /home/tkrueger/projects/connextdds-py/build/temp.linux-x86_64-cpython-311/connext-py
[ 1%] Building CXX object connextdds/CMakeFiles/connextdds.dir/src/misc/Constants.cpp.o
In file included from /tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/pybind11/include/pybind11/cast.h:16,
from /tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/pybind11/include/pybind11/attr.h:13,
from /tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/pybind11/include/pybind11/pybind11.h:13,
from /home/tkrueger/projects/connextdds-py/modules/connextdds/src/misc/Constants.cpp:12:
/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/pybind11/include/pybind11/detail/type_caster_base.h: In function ‘std::string pybind11::detail::error_string()’:
/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/pybind11/include/pybind11/detail/type_caster_base.h:482:26: error: invalid use of incomplete type ‘PyFrameObject’ {aka ‘struct _frame’}
482 | frame = frame->f_back;
| ^~
In file included from /usr/include/python3.11/Python.h:42,
from /tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/pybind11/include/pybind11/detail/common.h:209,
from /tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/pybind11/include/pybind11/pytypes.h:12,
from /tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/pybind11/include/pybind11/cast.h:13:
/usr/include/python3.11/pytypedefs.h:22:16: note: forward declaration of ‘PyFrameObject’ {aka ‘struct _frame’}
22 | typedef struct _frame PyFrameObject;
| ^~~~~~
gmake[2]: *** [connextdds/CMakeFiles/connextdds.dir/build.make:76: connextdds/CMakeFiles/connextdds.dir/src/misc/Constants.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:134: connextdds/CMakeFiles/connextdds.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/usr/lib/python3/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/build_meta.py", line 416, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
self.run_setup()
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/build_meta.py", line 488, in run_setup
self).run_setup(setup_script=setup_script)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/build_meta.py", line 338, in run_setup
exec(code, locals())
File "<string>", line 260, in <module>
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/__init__.py", line 107, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/dist.py", line 1234, in run_command
super().run_command(command)
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/wheel/bdist_wheel.py", line 343, in run
self.run_command("build")
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/dist.py", line 1234, in run_command
super().run_command(command)
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/command/build.py", line 131, in run
self.run_command(cmd_name)
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/dist.py", line 1234, in run_command
super().run_command(command)
File "/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "<string>", line 225, in run
File "/usr/lib/python3.11/subprocess.py", line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/cmake/data/bin/cmake', '--build', '.', '--config', 'Release', '--parallel', '1']' returned non-zero exit status 2.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for rti.connext
Failed to build rti.connext
ERROR: Failed to build one or more wheels
Hi @kruegerrobotics,
The build / installation guide states that Python 3.6 or newer is required to build the Connext DDS Python API. So I'm not sure this would be accepted as an 'Issue'.
Hope this helps.
Cheers, Thijs
Thanks, maybe just to cross check. Before on Debian 11 with Python 3.9 all was good. Now Debian 12 with almost the newest Python 3.11.2 fails. My guess it has more to do with the change of the API of pybind11 and thus the compilation fail.
I hope this helps. I basically upgraded connextdds to 7.1.0 and Debian from 11 to 12 and thus Python 3.9 to 3.11.
Kind regards,
Oh, sorry, I somehow thought 3.11 was smaller than 3.6... Time for the weekend I guess....
I think I have an idea and get it to build
Python 3.11 release notes under PyFrameObject states that functions have been changed.
When executing pip wheel . I see that CMake is getting pybind (strangely from my /tmp dir, not sure why)
-- Found RTIConnextDDS: /opt/rti_connext_dds (found suitable version "7.1.0.0", minimum required is "7.1.0") found components: core distributed_logger
-- Found Python3: /usr/bin/python3.11 (found suitable exact version "3.11.2") found components: Interpreter Development.Module
-- Found pybind11: /tmp/pip-build-env-verwekpf/overlay/local/lib/python3.11/dist-packages/pybind11/include (found version "2.9.0")
As a quick fix:
- I build in a virtual environment (this alone does not fix it, CMake still finds and places the magically the too-old pybind 2.9.0 in my /tmp folder)
- install pybind11 with pip in that virtial environment 3.10.2 in my case
- I change in the the CMakeLists.txt connextdds-py/modules/connextdds/CMakeLists.txt
from:
find_package(
pybind11
REQUIRED
)
to
find_package(
pybind11 2.10
REQUIRED
)
I think more checks are required and not ready for a pull request since this might break backward compatibility. Maybe a conditional Python version mechanism for the pybind version - or Why does this create a pybind 2.9.0 in the temp folder?