b2
b2 copied to clipboard
bfgroup
Python gets detected correctly but the Header File pyconfig.h can not be found - MSYS2 MinGW-w64 10.2.0
Make sure you completed the following tasks
- [x] I searched the discussions
- [x] I searched the closed and open issues
- [x] I read the contribution guidelines
Environment and version details
-
Operating System+version: ** Windows 10, 20H2 (19042.906), x86_64 **
-
Compiler+version: ** GCC, MinGW-w64 on MSYS2, 10.2.0, x86_64 **
-
Shell: ** Bash, MSYS2, also tried to invoke the Script with sh - no difference **
-
B2 Version: Output of
b2 -vandb2 --version
./b2 -v
B2 Version 4.4. OS=NT.
Copyright 1993-2002 Christopher Seiwald and Perforce Software, Inc.
Copyright 2001 David Turner.
Copyright 2001-2004 David Abrahams.
Copyright 2002-2019 Rene Rivera.
Copyright 2003-2015 Vladimir Prus.
DEFAULTS: jobs = 32
./b2 --version
B2 4.4-git
- B2 Configuration: Output of
b2 --debug-configurationin your project.
> g++ -x c++ -std=c++11 -O2 -s -DNDEBUG builtins.cpp class.cpp command.cpp compile.cpp constants.cpp cwd.cpp debug.cpp debugger.cpp execcmd.cpp execnt.cpp execunix.cpp filesys.cpp filent.cpp fileunix.cpp frames.cpp function.cpp glob.cpp hash.cpp hcache.cpp hdrmacro.cpp headers.cpp jam_strings.cpp jam.cpp jamgram.cpp lists.cpp make.cpp make1.cpp md5.cpp mem.cpp modules.cpp native.cpp object.cpp option.cpp output.cpp parse.cpp pathnt.cpp pathsys.cpp pathunix.cpp regexp.cpp rules.cpp scan.cpp search.cpp startup.cpp subst.cpp sysinfo.cpp timestamp.cpp variable.cpp w32_getreg.cpp modules/order.cpp modules/path.cpp modules/property-set.cpp modules/regex.cpp modules/sequence.cpp modules/set.cpp -o b2
> cp b2 bjam
tools/build/src/engine/b2
Detecting Python version... 3.8
Detecting Python root... C:/Dev/msys64/mingw64
Unicode/ICU support for Boost.Regex?... C:/Dev/ICUx64MinGW/lib/
Generating B2 configuration in project-config.jam for gcc...
Bootstrapping is done. To build, run:
./b2
To generate header files, run:
./b2 headers
The configuration generated uses gcc to build by default. If that is
unintended either use the --with-toolset option or adjust configuration, by
editing 'project-config.jam'.
Further information:
- Command line help:
./b2 --help
- Getting started guide:
http://www.boost.org/more/getting_started/unix-variants.html
- B2 documentation:
http://www.boost.org/build/
Brief problem description
The Header File 'pyconfig.h' does not get found and compilation errors out:
In file included from ./boost/python/detail/prefix.hpp:13,
from ./boost/python/list.hpp:8,
from libs\python\src\list.cpp:5:
./boost/python/detail/wrap_python.hpp:57:11: fatal error: pyconfig.h: No such file or directory
57 | # include <pyconfig.h>
Steps to reproduce the issue
I run the following Script (Snippet) to build the Boost Libraries from a locally fresh cloned Git Repository (Master Branch, 1.76):
if [ ! -d ${SYSTEMDRIVE}/Dev/BoostSrc/ ]; then cp --force --recursive --verbose ${SYSTEMDRIVE}/Dev/GitRepos/boost/ ${SYSTEMDRIVE}/Dev/BoostSrc/
fi
cd ${SYSTEMDRIVE}/Dev/BoostSrc/
./bootstrap.sh --with-icu=${SYSTEMDRIVE}/Dev/ICUx64MinGW/lib/ --prefix=${SYSTEMDRIVE}/Dev/Sources/GTK4/boost/x64/debug/
./b2 install --debug-configuration release link=static --prefix=${SYSTEMDRIVE}/Dev/Sources/GTK4/boost/x64/debug/
Actual behavior summary
- The Python-Devel Package is installed
- Setting Flags like '--with-python' etc. do not change anything: Detection is okay, compilation errors out.
- I also applied the change from https://github.com/bfgroup/b2/commit/ca7d1729b160da2f07be5369349dc00b1b8313ef manually in the Boost Library Repository - no luck
Also random stuff like this from the Internet did not help:
export C_INCLUDE_PATH="$C_INCLUDE_PATH:C:/Dev/msys64/mingw64/include/python3.8/"
export CPLUS_INCLUDE_PATH="$CPLUS_INCLUDE_PATH:C:/Dev/msys64/mingw64/include/python3.8/"
echo "using python : 3.8 ;" >> ${SYSTEMDRIVE}/Dev/BoostSrc/tools/build/src/site-config.jam
If a project of yours is blocked due to this bug, please, mention it explicitly.
Here is the cut Output from './b2 install -d2 --debug-configuration release link=static --prefix=${SYSTEMDRIVE}/Dev/Sources/GTK4/boost/x64/release/ 1>aaa.txt 2>&1':
notice: found boost-build.jam at C:/Dev/BoostSrc/boost-build.jam
notice: loading B2 from C:/Dev/BoostSrc/tools/build/src/bootstrap.jam
notice: Searching 'C:\Users\Oliver' 'C:\Dev\msys64\home\Oliver' 'C:\Users\Oliver' 'C:/Dev/BoostSrc/tools/build/src' 'C:/Dev/BoostSrc/tools/build/src/kernel' 'C:/Dev/BoostSrc/tools/build/src/util' 'C:/Dev/BoostSrc/tools/build/src/build' 'C:/Dev/BoostSrc/tools/build/src/tools' 'C:/Dev/BoostSrc/tools/build/src/contrib' 'C:/Dev/BoostSrc/tools/build/src/.' for site-config configuration file 'site-config.jam'.
notice: Configuration file 'site-config.jam' not found in 'C:\Users\Oliver' 'C:\Dev\msys64\home\Oliver' 'C:\Users\Oliver' 'C:/Dev/BoostSrc/tools/build/src' 'C:/Dev/BoostSrc/tools/build/src/kernel' 'C:/Dev/BoostSrc/tools/build/src/util' 'C:/Dev/BoostSrc/tools/build/src/build' 'C:/Dev/BoostSrc/tools/build/src/tools' 'C:/Dev/BoostSrc/tools/build/src/contrib' 'C:/Dev/BoostSrc/tools/build/src/.'.
notice: Searching 'C:\Users\Oliver' 'C:\Dev\msys64\home\Oliver' 'C:\Users\Oliver' 'C:/Dev/BoostSrc/tools/build/src' 'C:/Dev/BoostSrc/tools/build/src/kernel' 'C:/Dev/BoostSrc/tools/build/src/util' 'C:/Dev/BoostSrc/tools/build/src/build' 'C:/Dev/BoostSrc/tools/build/src/tools' 'C:/Dev/BoostSrc/tools/build/src/contrib' 'C:/Dev/BoostSrc/tools/build/src/.' for user-config configuration file 'user-config.jam'.
notice: Configuration file 'user-config.jam' not found in 'C:\Users\Oliver' 'C:\Dev\msys64\home\Oliver' 'C:\Users\Oliver' 'C:/Dev/BoostSrc/tools/build/src' 'C:/Dev/BoostSrc/tools/build/src/kernel' 'C:/Dev/BoostSrc/tools/build/src/util' 'C:/Dev/BoostSrc/tools/build/src/build' 'C:/Dev/BoostSrc/tools/build/src/tools' 'C:/Dev/BoostSrc/tools/build/src/contrib' 'C:/Dev/BoostSrc/tools/build/src/.'.
notice: Searching '.' for project-config configuration file 'project-config.jam'.
notice: Loading project-config configuration file 'project-config.jam' from '.'.
notice: will use 'g++' for gcc, condition <toolset>gcc-10.2.0
notice: using gcc libraries :: <toolset>gcc-10.2.0 :: C:\Dev\msys64\mingw64/bin C:\Dev\msys64\mingw64/lib C:\Dev\msys64\mingw64/lib32 C:\Dev\msys64\mingw64/lib64
notice: using gcc archiver :: <toolset>gcc-10.2.0 :: C:/Dev/msys64/mingw64/x86_64-w64-mingw32/bin/ar.exe
notice: using gcc ranlib :: <toolset>gcc-10.2.0 :: C:/Dev/msys64/mingw64/x86_64-w64-mingw32/bin/ranlib.exe
notice: using rc compiler :: <toolset>gcc-10.2.0 :: C:\Dev\msys64\mingw64\bin\windres.exe
notice: [python-cfg] Configuring python...
notice: [python-cfg] user-specified version: "3.8"
notice: [python-cfg] user-specified cmd-or-prefix: "C:/Dev/msys64/mingw64"
notice: [python-cfg] Checking interpreter command "C:/Dev/msys64/mingw64\PCBuild8\python"...
notice: [python-cfg] running command 'C:/Dev/msys64/mingw64\PCBuild8\python -c "from sys import *; print('version=%d.%d\nplatform=%s\nprefix=%s\nexec_prefix=%s\nexecutable=%s' % (version_info[0],version_info[1],platform,prefix,exec_prefix,executable))" 2>&1'
notice: [python-cfg] ...does not invoke a working interpreter
notice: [python-cfg] Checking interpreter command "C:/Dev/msys64/mingw64\PCBuild\python"...
notice: [python-cfg] running command 'C:/Dev/msys64/mingw64\PCBuild\python -c "from sys import *; print('version=%d.%d\nplatform=%s\nprefix=%s\nexec_prefix=%s\nexecutable=%s' % (version_info[0],version_info[1],platform,prefix,exec_prefix,executable))" 2>&1'
notice: [python-cfg] ...does not invoke a working interpreter
notice: [python-cfg] Checking interpreter command "C:/Dev/msys64/mingw64\python"...
notice: [python-cfg] running command 'C:/Dev/msys64/mingw64\python -c "from sys import *; print('version=%d.%d\nplatform=%s\nprefix=%s\nexec_prefix=%s\nexecutable=%s' % (version_info[0],version_info[1],platform,prefix,exec_prefix,executable))" 2>&1'
notice: [python-cfg] ...does not invoke a working interpreter
notice: [python-cfg] Checking interpreter command "python"...
notice: [python-cfg] running command 'DIR /-C /A:S "C:\Dev\msys64\mingw64\bin\python.exe" 2>&1'
notice: [python-cfg] running command 'python -c "from sys import *; print('version=%d.%d\nplatform=%s\nprefix=%s\nexec_prefix=%s\nexecutable=%s' % (version_info[0],version_info[1],platform,prefix,exec_prefix,executable))" 2>&1'
notice: [python-cfg] ...requested configuration matched!
notice: [python-cfg] Details of this Python configuration:
notice: [python-cfg] interpreter command: "python"
notice: [python-cfg] include path: "C:/Dev/msys64/mingw64\Include"
notice: [python-cfg] library path: "C:/Dev/msys64/mingw64\libs"
notice: [python-cfg] DLL search path: "C:/Dev/msys64/mingw64"
notice: [python-cfg] Checking for NumPy...
notice: [python-cfg] running command 'python -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())"'
notice: [python-cfg] NumPy disabled. Reason:
notice: [python-cfg] python -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())" aborted with
notice: [python-cfg] Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
notice: [zlib] Using pre-installed library
notice: [zlib] Condition
notice: [bzip2] Using pre-installed library
notice: [bzip2] Condition
notice: [lzma] Using pre-installed library
notice: [lzma] Condition
notice: [zstd] Using pre-installed library
notice: [zstd] Condition
Performing configuration checks
- default address-model : 64-bit [1]
- default architecture : x86 [1]
- symlinks supported : yes
- compiler supports SSE2 : yes [2]
- compiler supports SSE4.1 : yes [2]
- has synchronization.lib : no [2]
- C++11 mutex : yes [2]
- has BCrypt API : yes [2]
- has stat::st_mtim : no [2]
- has stat::st_mtimensec : no [2]
- has stat::st_mtimespec : no [2]
- has stat::st_birthtim : no [2]
- has stat::st_birthtimensec : no [2]
- has stat::st_birthtimespec : no [2]
- has statx : no [2]
- has statx syscall : no [2]
- cxx11_auto_declarations : yes [2]
- cxx11_constexpr : yes [2]
- cxx11_defaulted_functions : yes [2]
- cxx11_final : yes [2]
- cxx11_hdr_mutex : yes [2]
- cxx11_hdr_tuple : yes [2]
- cxx11_lambdas : yes [2]
- cxx11_noexcept : yes [2]
- cxx11_nullptr : yes [2]
- cxx11_rvalue_references : yes [2]
- cxx11_template_aliases : yes [2]
- cxx11_thread_local : no [2]
- cxx11_variadic_templates : yes [2]
- cxx11_auto_declarations : yes [3]
- cxx11_constexpr : yes [3]
- cxx11_defaulted_functions : yes [3]
- cxx11_final : yes [3]
- cxx11_hdr_mutex : yes [3]
- cxx11_hdr_tuple : yes [3]
- cxx11_lambdas : yes [3]
- cxx11_noexcept : yes [3]
- cxx11_nullptr : yes [3]
- cxx11_rvalue_references : yes [3]
- cxx11_template_aliases : yes [3]
- cxx11_thread_local : no [3]
- cxx11_variadic_templates : yes [3]
- has_icu builds : no [2]
warning: Graph library does not contain MPI-based parallel components.
note: to enable them, add "using mpi ;" to your user-config.jam.
note: to suppress this message, pass "--without-graph_parallel" to bjam.
- zlib : no [4]
- bzip2 : no [4]
- lzma : no [4]
- zstd : no [4]
- has_lzma_cputhreads builds : no [2]
- cxx11_decltype : yes [2]
- cxx11_basic_alignas : yes [2]
- iconv (libc) : no [2]
- iconv (separate) : no [2]
- icu : no [2]
- icu (lib64) : no [2]
- native atomic int32 supported : yes [2]
- has message compiler : no [2]
- native syslog supported : no [2]
- pthread supports robust mutexes : no [2]
- compiler supports SSSE3 : yes [2]
- compiler supports AVX2 : yes [2]
- gcc visibility : yes [2]
- sfinae_expr : yes [2]
- cxx11_unified_initialization_syntax : yes [2]
- cxx11_hdr_initializer_list : yes [2]
- cxx11_hdr_chrono : yes [2]
- cxx11_numeric_limits : yes [2]
- cxx11_hdr_array : yes [2]
- cxx11_hdr_atomic : yes [2]
- cxx11_hdr_type_traits : yes [2]
- cxx11_allocator : yes [2]
- cxx11_explicit_conversion_operators : yes [2]
- gcc visibility : yes [5]
- cxx11_noexcept : yes [5]
- cxx11_rvalue_references : yes [5]
- sfinae_expr : yes [5]
- cxx11_auto_declarations : yes [5]
- cxx11_lambdas : yes [5]
- cxx11_unified_initialization_syntax : yes [5]
- cxx11_hdr_tuple : yes [5]
- cxx11_hdr_initializer_list : yes [5]
- cxx11_hdr_chrono : yes [5]
- cxx11_thread_local : no [5]
- cxx11_constexpr : yes [5]
- cxx11_nullptr : yes [5]
- cxx11_numeric_limits : yes [5]
- cxx11_decltype : yes [5]
- cxx11_hdr_array : yes [5]
- cxx11_hdr_atomic : yes [5]
- cxx11_hdr_type_traits : yes [5]
- cxx11_allocator : yes [5]
- cxx11_explicit_conversion_operators : yes [5]
warning: skipping optional Message Passing Interface (MPI) library.
note: to enable MPI support, add "using mpi ;" to user-config.jam.
note: to suppress this message, pass "--without-mpi" to bjam.
note: otherwise, you can safely ignore this message.
- cxx11_static_assert : yes [2]
- std::fstream is moveable and swappable : yes [2]
- Has Large File Support : no [2]
- libbacktrace builds : no [2]
- libbacktrace builds : no [3]
- addr2line builds : no [2]
- addr2line builds : no [3]
- WinDbg builds : no [2]
- WinDbg builds : no [3]
- WinDbgCached builds : no [2]
- WinDbgCached builds : no [3]
- BOOST_COMP_GNUC >= 4.3.0 : yes [2]
- compiler supports SSE2 : yes [6]
- compiler supports SSE4.1 : yes [6]
- has synchronization.lib : yes [6]
- C++11 mutex : yes [6]
- has BCrypt API : yes [6]
- has stat::st_mtim : no [6]
- has stat::st_mtimensec : no [6]
- has stat::st_mtimespec : no [6]
- has stat::st_birthtim : no [6]
- has stat::st_birthtimensec : no [6]
- has stat::st_birthtimespec : no [6]
- has statx : no [6]
- has statx syscall : no [6]
- cxx11_auto_declarations : yes [6]
- cxx11_constexpr : yes [6]
- cxx11_defaulted_functions : yes [6]
- cxx11_final : yes [6]
- cxx11_hdr_mutex : yes [6]
- cxx11_hdr_tuple : yes [6]
- cxx11_lambdas : yes [6]
- cxx11_noexcept : yes [6]
- cxx11_nullptr : yes [6]
- cxx11_rvalue_references : yes [6]
- cxx11_template_aliases : yes [6]
- cxx11_thread_local : yes [6]
- cxx11_variadic_templates : yes [6]
- cxx11_auto_declarations : yes [7]
- cxx11_constexpr : yes [7]
- cxx11_defaulted_functions : yes [7]
- cxx11_final : yes [7]
- cxx11_hdr_mutex : yes [7]
- cxx11_hdr_tuple : yes [7]
- cxx11_lambdas : yes [7]
- cxx11_noexcept : yes [7]
- cxx11_nullptr : yes [7]
- cxx11_rvalue_references : yes [7]
- cxx11_template_aliases : yes [7]
- cxx11_thread_local : yes [7]
- cxx11_variadic_templates : yes [7]
- has_icu builds : no [6]
- zlib : no (cached) [4]
- bzip2 : no (cached) [4]
- lzma : no (cached) [4]
- zstd : no (cached) [4]
- has_lzma_cputhreads builds : no [6]
- cxx11_decltype : yes [6]
- cxx11_basic_alignas : yes [6]
- iconv (libc) : no [6]
- iconv (separate) : yes [6]
- icu : no [6]
- icu (lib64) : no [6]
- native atomic int32 supported : yes [6]
- has message compiler : no [6]
- native syslog supported : no [6]
- pthread supports robust mutexes : no [6]
- compiler supports SSSE3 : yes [6]
- compiler supports AVX2 : yes [6]
- gcc visibility : yes [6]
- sfinae_expr : yes [6]
- cxx11_unified_initialization_syntax : yes [6]
- cxx11_hdr_initializer_list : yes [6]
- cxx11_hdr_chrono : yes [6]
- cxx11_numeric_limits : yes [6]
- cxx11_hdr_array : yes [6]
- cxx11_hdr_atomic : yes [6]
- cxx11_hdr_type_traits : yes [6]
- cxx11_allocator : yes [6]
- cxx11_explicit_conversion_operators : yes [6]
- gcc visibility : yes [8]
- cxx11_noexcept : yes [8]
- cxx11_rvalue_references : yes [8]
- sfinae_expr : yes [8]
- cxx11_auto_declarations : yes [8]
- cxx11_lambdas : yes [8]
- cxx11_unified_initialization_syntax : yes [8]
- cxx11_hdr_tuple : yes [8]
- cxx11_hdr_initializer_list : yes [8]
- cxx11_hdr_chrono : yes [8]
- cxx11_thread_local : yes [8]
- cxx11_constexpr : yes [8]
- cxx11_nullptr : yes [8]
- cxx11_numeric_limits : yes [8]
- cxx11_decltype : yes [8]
- cxx11_hdr_array : yes [8]
- cxx11_hdr_atomic : yes [8]
- cxx11_hdr_type_traits : yes [8]
- cxx11_allocator : yes [8]
- cxx11_explicit_conversion_operators : yes [8]
- long double support : yes [8]
- cxx11_static_assert : yes [6]
- std::fstream is moveable and swappable : yes [6]
- Has Large File Support : yes [6]
- libbacktrace builds : no [6]
- libbacktrace builds : no [7]
- addr2line builds : no [6]
- addr2line builds : no [7]
- WinDbg builds : yes [6]
- WinDbg builds : yes [7]
- WinDbgCached builds : yes [6]
- BOOST_COMP_GNUC >= 4.3.0 : yes [6]
[1] gcc-10.2.0
[2] gcc-10.2.0/release/address-model-32/link-static/python-3.8/threadapi-win32/threading-multi/visibility-hidden
[3] gcc-10.2.0/release/address-model-32/build-no/link-static/python-3.8/threadapi-win32/threading-multi/visibility-hidden
[4] link-static
[5] gcc-10.2.0/release/address-model-32/build-no/link-static/pch-off/python-3.8/threadapi-win32/threading-multi/visibility-hidden
[6] gcc-10.2.0/release/link-static/python-3.8/threadapi-win32/threading-multi/visibility-hidden
[7] gcc-10.2.0/release/build-no/link-static/python-3.8/threadapi-win32/threading-multi/visibility-hidden
[8] gcc-10.2.0/release/link-static/pch-off/python-3.8/threadapi-win32/threading-multi/visibility-hidden
Component configuration:
- atomic : building
- chrono : building
- container : building
- context : building
- contract : building
- coroutine : building
- date_time : building
- exception : building
- fiber : building
- filesystem : building
- graph : building
- graph_parallel : building
- headers : building
- iostreams : building
- json : building
- locale : building
- log : building
- math : building
- mpi : building
- nowide : building
- program_options : building
- python : building
- random : building
- regex : building
- serialization : building
- stacktrace : building
- system : building
- test : building
- thread : building
- timer : building
- type_erasure : building
- wave : building
...found 34574 targets...
...updating 17670 targets...
...patience...
[... shortened - the whole Log is too big ...]
If you want the whole Log I can upload it. Just let me know. It is 8.1 Megabyte of Text.
Expected behavior summary
The Header File 'pyconfig.h' should get detected correctly and the Boost Libraries should get compiled without further Issues.
Disclaimer: This is my first Day with Boost at all. I just need to build some Libs for some GTKMM4 Dependencies. And as I need to compile from Source anyway, to get Build Variants not provided by MSYS2, here I am reporting an 9 year old Issue.
This might be related to boostorg/build#481 where the Python installation is built with PyMalloc, so the header path has an m suffix (e.g. path/to/Python/3.8.x/include/python3.8m)
Wow. I totally forgot about that Bug Report. Thanks for the Hint adeadman!
A general FYI for People interested: Since I have reported this Bug, I am building Boost on Linux and Windows always with:
--without-python
as a work-around.
This does not change the Detection of Python as it is hard-coded in the configuration Files but compilation works fine.
I am not using the Boost Libs currently, so I am not aware of the Cons of not building with / against Python support.
Have a nice Day.
#StaySafeStayHealthyEveryone
I needed to build the Boost.Python libraries back in 2012 or so, and it was an absolute nightmare back then. Now, here in 2023, I'm having to do it again, and having the same b2 and jam headaches. Even ChatGPT can't get it to compile.... All we really need is better documentation. For example, I'm running into the age-old problem of the compiler complaining about not finding Python.h. Capital P in Python.h, at least on this Linux machine. I'm an extensive user of conda environments, and I'm trying to bootstrap and b2 off of one of my environments. I've burned through 3 full days now trying to get the config right, and there seems to be no hope. Of all the code I have written over the decades of writing code, nothing has ever been as completely frustrating as trying to build Boost.Python with bootstrap and b2. I'm hoping that by venting here, I go back to my build project and it magically compiles so I can get on with what I need to be doing.
For what it is worth, as is often the case, after ranting publicly, I quickly came up with a solution that had been confounding me for days. My notes, should anyone care are here: github.com/rlovell3/boost_python_compile_notes. Those notes are for building on Ubuntu and associated flavors, using a highly customized conda environment running Python 3.11. Thank you to all the Boost contributors. Boost is truly an amazing work. b2 is still a pain though, but I get it.