colcon
Colcon extends LD_LIBARAY_PATH with cross compiled libraries
I am trying to cross compile rosbag2 for QNX 7.2 on Ubuntu Jammy. If I do so, cmake crashes for packages which depend on zstd_vendor with this error
/usr/bin/cmake: error while loading shared libraries: libc.so.5: cannot open shared object file: No such file or directory
The reason for that is that:
- QNX 7.2 ships with libc.so.5
- Ubuntu Jammy ships with libc.so.6
- QNX and Linux both use ELF as their file format for executable files
- zstd_vendor build libzstd.so
- CMake links agains libzstd.so
- Colcon adds libraries build in dependencies to LD_LIBARARY_PATH
- This results for packages which depend on zstd_vendor to in cmake trying to load libzstd.so from zstd_vendor. This will fail because this version of libzstd.so is build against libc.so.5 - which is not available on Ubuntu Jamme and therefore results in a failure.
I was able to circumvent this issue by reconfiguring zstd_vendor so that is creates static libraries in stead of shared libraries. However, this issue concernes me a bit. It should be possible to cross compile projects using Colcon without the potential risk of injecting newly build libraries into the build system.
At this point I'm not entirely sure how to fix this issue. My first idea was to have some kind of "cross compilation flag" which turns this "LD_LIBRARY_PATH" behavior off entirely during build time. However, as think more about it: in general it should not be possible that parts of the build system end up using just build libraries by accident - this seems just wrong to me.
These hooks certainly aren't designed for cross-compiled builds like this, so I'm not surprised that strange behaviors like this manifest.
The easiest and most supported way to block the LD_LIBRARY_PATH modifications would be to use the environment variable COLCON_EXTENSION_BLOCKLIST=colcon_core.environment.library_path during the build. You may also want to consider COLCON_EXTENSION_BLOCKLIST=colcon_core.environment.path to block automatic changes to PATH as well.
Docs for that variable are here: https://colcon.readthedocs.io/en/released/reference/global-arguments.html?highlight=COLCON_EXTENSION_BLOCKLIST#environment-variables
Thank you for your anwser! Using COLCON_EXTENSION_BLOCKLIST actually helps making the build work. But it also supresses the generation of the LD_LIBRARY_PATH hooks alltogether, which means that during runtime on the target no libraries can be found.
I suppose we would need something like that but which only blocks the usage of the hooks of a certain extensions during build time. The hooks itself are still valuable and required on the target machine but can cause harm during build.
We were now able to circumvent this issue
printf "#!/bin/bash\nLD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu /usr/bin/cmake "$@"" > /tmp/cmake_wrapper chmod +x /tmp/cmake_wrapper export CMAKE_COMMAND="/tmp/cmake_wrapper"
With this workaround it is possible to protect cmake from the modified LD_LIBRARY_PATH. But this feels like a workaround to me. At this point I wonder whether it is possible with reasonable effort to protect parts of the toolchain (cmake, ctest, etc) from the modified LD_LIBRARY_PATH?
