python-mapnik Fails to build with Mapnik 4.0.0

A few patches are required to build with the mapnik (4.0.0+ds-1) Debian package:

https://github.com/mapnik/python-mapnik/pull/274
https://github.com/mapnik/python-mapnik/pull/275

This worked with mapnik 4.0.0-rc1 and python-mapnik @ https://github.com/mapnik/python-mapnik/commit/5ab32f0209909cc98c26e1d86ce0c8ef29a9bf3d

The above two patches needed to be rebased to work with python-mapnik @ https://github.com/mapnik/python-mapnik/commit/248003c9da0de1ec7996cebd007c982f9875f2a3

Unfortunately the pybind11 port doesn't support all variants of mapnik::value_holder::value_holder():

c++ -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -Werror=implicit-function-declaration -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -g -O2 -Werror=implicit-function-declaration -ffile-prefix-map=/build/python-mapnik-0.0~20240614-248003c9d=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/lib/python3/dist-packages/pybind11/include -I/usr/include/python3.11 -c src/mapnik_datasource.cpp -o build/temp.linux-x86_64-cpython-311/src/mapnik_datasource.o -std=c++17 -fvisibility=hidden -I/usr/include/mapnik -I/usr/include/mapnik/agg -DMAPNIK_THREADSAFE -DBOOST_REGEX_HAS_ICU -DBIGINT -DMAPNIK_MEMORY_MAPPED_FILE -DHAVE_LIBXML2 -DHAVE_PNG -DHAVE_JPEG -DHAVE_TIFF -DHAVE_WEBP -DHAVE_CAIRO -DMAPNIK_USE_PROJ -DMAPNIK_PROJ_VERSION=90401 -DGRID_RENDERER -DSVG_RENDERER -DMAPNIK_HAS_DLCFN -I/usr/include/harfbuzz -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/sysprof-6 -pthread -I/usr/include/libxml2 -I/usr/include/x86_64-linux-gnu -I/usr/include/webp -I/usr/include/cairo -I/usr/include/pixman-1
cc1plus: warning: '-Werror=' argument '-Werror=implicit-function-declaration' is not valid for C++
cc1plus: warning: '-Werror=' argument '-Werror=implicit-function-declaration' is not valid for C++
In file included from /usr/include/mapnik/datasource.hpp:28,
                 from src/mapnik_datasource.cpp:26:
/usr/include/mapnik/params.hpp: In instantiation of 'mapnik::value_holder::value_holder(T&&) [with T = long long int]':
src/mapnik_value_converter.hpp:183:42:   required from here
/usr/include/mapnik/params.hpp:58:49: error: no matching function for call to 'mapbox::util::variant<mapnik::value_null, long int, double, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool>::variant(long long int)'
   58 |         : value_holder_base(std::forward<T>(obj))
      |                                                 ^
In file included from /usr/include/mapnik/util/variant.hpp:27,
                 from /usr/include/mapnik/params.hpp:29:
/usr/include/mapbox/variant.hpp:560:20: note: candidate: 'template<class T, class Traits, class Enable> mapbox::util::variant<Types>::variant(T&&) [with Traits = T; Enable = Traits; Types = {mapnik::value_null, long int, double, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool}]'
  560 |     VARIANT_INLINE variant(T&& val) noexcept(std::is_nothrow_constructible<typename Traits::target_type, T&&>::value)
      |                    ^~~~~~~
/usr/include/mapbox/variant.hpp:560:20: note:   template argument deduction/substitution failed:
/usr/include/mapbox/variant.hpp:559:15: error: no type named 'type' in 'struct std::enable_if<false, void>'
  559 |               typename Enable = typename std::enable_if<Traits::is_valid && !std::is_same<variant<Types...>, typename Traits::value_type>::value>::type >
      |               ^~~~~~~~
/usr/include/mapbox/variant.hpp:572:20: note: candidate: 'mapbox::util::variant<Types>::variant(mapbox::util::variant<Types>&&) [with Types = {mapnik::value_null, long int, double, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool}]'
  572 |     VARIANT_INLINE variant(variant<Types...>&& old)
      |                    ^~~~~~~
/usr/include/mapbox/variant.hpp:572:48: note:   no known conversion for argument 1 from 'long long int' to 'mapbox::util::variant<mapnik::value_null, long int, double, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool>&&'
  572 |     VARIANT_INLINE variant(variant<Types...>&& old)
      |                            ~~~~~~~~~~~~~~~~~~~~^~~
/usr/include/mapbox/variant.hpp:566:20: note: candidate: 'mapbox::util::variant<Types>::variant(const mapbox::util::variant<Types>&) [with Types = {mapnik::value_null, long int, double, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool}]'
  566 |     VARIANT_INLINE variant(variant<Types...> const& old)
      |                    ^~~~~~~
/usr/include/mapbox/variant.hpp:566:53: note:   no known conversion for argument 1 from 'long long int' to 'const mapbox::util::variant<mapnik::value_null, long int, double, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool>&'
  566 |     VARIANT_INLINE variant(variant<Types...> const& old)
      |                            ~~~~~~~~~~~~~~~~~~~~~~~~~^~~
/usr/include/mapbox/variant.hpp:554:20: note: candidate: 'mapbox::util::variant<Types>::variant(mapbox::util::no_init) [with Types = {mapnik::value_null, long int, double, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool}]'
  554 |     VARIANT_INLINE variant(no_init) noexcept
      |                    ^~~~~~~
/usr/include/mapbox/variant.hpp:554:28: note:   no known conversion for argument 1 from 'long long int' to 'mapbox::util::no_init'
  554 |     VARIANT_INLINE variant(no_init) noexcept
      |                            ^~~~~~~
/usr/include/mapbox/variant.hpp:547:20: note: candidate: 'mapbox::util::variant<Types>::variant() [with Types = {mapnik::value_null, long int, double, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool}]'
  547 |     VARIANT_INLINE variant() noexcept(std::is_nothrow_default_constructible<first_type>::value)
      |                    ^~~~~~~
/usr/include/mapbox/variant.hpp:547:20: note:   candidate expects 0 arguments, 1 provided
error: command '/usr/bin/c++' failed with exit code 1

This might be caused by 64-bit time_t (-D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64) being used in Debian.

Jun 17 '24 05:06 sebastic

Same problem here.

Sep 05 '24 06:09 claeyswo

Same problem here. Do you have an update?

@sebastic @artemp

Nov 08 '24 11:11 edoardoramalli

I have refreshed the patches in the Debian package, but haven't bothered to rebase my branches here as I haven't been able to get the current state of master to build successfully.

The patches from Fedora may help, but I haven't found the motivation try again to get a newer git snapshot to build. I'm leaning more towards removing python-mapnik (and nik4) from Debian as there are still no proper releases after almost ten years.

Nov 08 '24 12:11 sebastic

Updated patches to apply to master @ https://github.com/mapnik/python-mapnik/commit/10315a6d898ed341f5df5975395f3dc67814ebf6:

https://github.com/mapnik/python-mapnik/pull/267
https://github.com/mapnik/python-mapnik/pull/274
https://github.com/mapnik/python-mapnik/pull/275 This conflicts with the changes in #274, ideally the pkg-config changes are merged first, and then this PR is rebased on top of those changes.
https://github.com/mapnik/python-mapnik/pull/284

Unfortunately the pybind11 build fails without this patch from Fedora:

https://src.fedoraproject.org/rpms/python-mapnik/blob/rawhide/f/python-mapnik-mapnik4.patch

But that's not sufficient, the resulting build then fails on import:

__ ERROR collecting test/python_tests/agg_rasterizer_integer_overflow_test.py __
ImportError while importing test module '/build/python-mapnik-0.0~20241108-10315a6d8/.pybuild/cpython3_3.13_mapnik/build/test/python_tests/agg_rasterizer_integer_overflow_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test/python_tests/agg_rasterizer_integer_overflow_test.py:2: in <module>
    import mapnik
mapnik/__init__.py:66: in <module>
    from ._mapnik import *
E   ImportError: Internal error while parsing type signature (1)
_______________ ERROR collecting test/python_tests/box2d_test.py _______________
ImportError while importing test module '/build/python-mapnik-0.0~20241108-10315a6d8/.pybuild/cpython3_3.13_mapnik/build/test/python_tests/box2d_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test/python_tests/box2d_test.py:1: in <module>
    import mapnik
mapnik/__init__.py:66: in <module>
    from ._mapnik import *
E   ImportError: generic_type: type "Color" is already registered!

The latter ImportError is repeated many times.

Dec 07 '24 10:12 sebastic