Cross-compiling Halide pipelines in CMake builds should be much easier

[alexreinking]

tl;dr the obvious way to use Halide via CMake (find_package(Halide)) does not lead to easy cross-compiling builds. However, fixing this is almost certainly a breaking change.

---

If you run find_package(Halide), it will try to locate libHalide.so and run an architecture check to see that CMake is building for the same target as libHalide.so was built for. Trying to cross-compile such projects leads to errors like the following:

Could not find a configuration file for package "Halide" that is compatible
  with requested version "".

  The following configuration files were considered but not accepted:

    /path/to/lib/cmake/Halide/HalideConfig.cmake, version: 15.0.0 (64bit)

This is because (in this case) the target is wasm, which is 32bit, but Halide was built for the x86-64 host. On some level, this makes sense, because CMake only allows one active target and toolchain at a time.

Some time ago, I factored our platform-independent parts into a package called HalideHelpers, which Halide imports, and which skips architecture checks. Thus, the most effective way to cross compile today is to:

1. find_package(HalideHelpers) (rather than Halide)
2. Use add_halide_generator and add_halide_library as usual
3. Let add_halide_generator call find_package(Halide) internally when it can't import pre-built generators in a cross-compiling scenario.

Indeed, HANNK does this with some success, and can cross-compile to wasm and Android: https://github.com/halide/Halide/blob/main/apps/hannk/CMakeLists.txt#L46

It does seem better to switch the roles up so that find_package(Halide) is what you're always supposed to call and then add_halide_generator internally calls find_package(HalideBinaries) (which can be platform-dependent). However, this is actually a tricky change to make and would require some communication with our users.

If you're not using the generator infrastructure, then you're likely linking to Halide::Halide, which is just libHalide.so, for use in JIT mode. Even if you are using the generator infrastructure, you might be linking to Halide::Generator, which is just Halide::Halide + GenGen.cpp. Switching the roles like this breaks users who aren't using the add_halide_generator helper.

---

We could say that, going forward, if you want to use generators, you must use our helper. Yet, that doesn't help JIT users. One way of addressing that problem might be to tell such users to include a JIT component in their call to find_package(Halide REQUIRED JIT), the effect of which would be to immediately load HalideBinaries.

[alexreinking]

Adapted from a conversation with @steven-johnson

[barnou-psee]

Hello @alexreinking, I'm very interested in that issue / subject you opened since I have similar environment. I want to start using Halide in a project that is cross compiled using CMake for Android. Following what you said and the hannk example, I tried doing that and actually the CMake is passing correctly, finding only HalideHelpers, getting the targets ready and I think CMake is identifying the targets as expected:

-- Halide 'host' platform triple:   x86-64-linux
-- Halide 'cmake' platform triple:  arm-64-android
-- Halide default AOT target:       arm-64-android

But I have a linking issue and this is where I would need some help to understand the Halide pipelines. To my understanding the generator is compiled for the host and is used to generate target specific pipelines. In my case the host is identified as x86-64-linux but the generator is compiled and linked using the android NDK which if I understand correctly is wrong.

Linking error

/opt/android-ndk-r25c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ 
  --target=aarch64-none-linux-android30 
  --sysroot=/opt/android-ndk-r25c/toolchains/llvm/prebuilt/linux-x86_64/sysroot 
  -g -DANDROID 
  -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes
   -D_FORTIFY_SOURCE=2 
  -Wformat -Werror=format-security   -O3 -DNDEBUG  -Wl,--build-id=sha1 -Wl,--fatal-warnings
  -Wl,--no-undefined -Qunused-arguments   -Wl,--gc-sections  
  CMakeFiles/lrgb_to_lab.generator.dir/lrgb_to_lab.cpp.o 
  CMakeFiles/lrgb_to_lab.generator.dir/home/barnou/Tools/halide/Halide-16.0.0-x86-64-linux/share/Halide/tools/GenGen.cpp.o
  -o ../../../../../bins/arm64-v8a/lrgb_to_lab.generator  
  /home/barnou/Tools/halide/Halide-16.0.0-x86-64-linux/lib/libHalide.so.16.0.0 -ldl -latomic -lm 
ld: error: /home/barnou/Tools/halide/Halide-16.0.0-x86-64-linux/lib/libHalide.so.16.0.0 is incompatible with aarch64linux

Is there something I'm forgetting to kinda force the generator to be compiled with the Host clang and not the Target one?

Apart from that, from the initial point is there an updated method to cross compile Halide pipelines with CMake?