Compile bug: Vulkan Cross compile for arm64

[andyt9527]

Git commit

dc39a5e7a84815a90fa0c515ed8927870cf858c9

Operating systems

Linux

GGML backends

Vulkan

Problem description & steps to reproduce

When I try to cross compile llama.cpp with vulkan backend support on x86-64 ubuntu, and the cmd as below: cmake -DGGML_OPENMP=OFF -DGGML_LLAMAFILE=OFF -DGGML_VULKAN=ON -DVulkan_LIBRARY=/home/andy/sysroot/usr/lib/aarch64-linux-gnu/libvulkan.so -DVulkan_INCLUDE_DIR=/home/andy/sysroot/usr/include -DVulkan_GLSLC_EXECUTABLE=/home/andy/andywork/sdk-android/Sdk/ndk/25.1.8937393/shader-tools/linux-x86_64/glslc -DBUILD_SHARED_LIBS=OFF -DCMAKE_SYSROOT=/home/andy/sysroot -DCMAKE_C_COMPILER=/home/andy/tools/gcc/arm-gnu-toolchain-12.3.rel1-x86_64-aarch64-none-linux-gnu/arm-gnu-toolchain-12.3.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gcc -DCMAKE_CXX_COMPILER=/home/andy/tools/gcc/arm-gnu-toolchain-12.3.rel1-x86_64-aarch64-none-linux-gnu/arm-gnu-toolchain-12.3.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-g++ -DGGML_NATIVE=OFF -DGGML_AVX=off -DGGML_AVX2=off -DGGML_AVX512=off -DGGML_FMA=off -DGGML_F16C=off -DGGML_CPU_ARM_ARCH=armv9-a+sve -DCMAKE_SYSTEM_PROCESSOR=armv9-a -DCMAKE_OSX_ARCHITECTURES=arm64 -DCMAKE_SYSTEM_NAME=Linux ..

But I think the glslc check logic is not solid in the cmd above, since the glslc is from x86，which cannot get the features for the taeget arm64 soc gpu env.

So if we can add a cross compile check and use some build args for this?

First Bad Commit

No response

Compile command

cmake -DGGML_OPENMP=OFF -DGGML_LLAMAFILE=OFF -DGGML_VULKAN=ON -DVulkan_LIBRARY=/home/andy/sysroot/usr/lib/aarch64-linux-gnu/libvulkan.so -DVulkan_INCLUDE_DIR=/home/andy/sysroot/usr/include -DVulkan_GLSLC_EXECUTABLE=/home/andy/andywork/sdk-android/Sdk/ndk/25.1.8937393/shader-tools/linux-x86_64/glslc -DBUILD_SHARED_LIBS=OFF -DCMAKE_SYSROOT=/home/andy/sysroot -DCMAKE_C_COMPILER=/home/andy/tools/gcc/arm-gnu-toolchain-12.3.rel1-x86_64-aarch64-none-linux-gnu/arm-gnu-toolchain-12.3.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gcc -DCMAKE_CXX_COMPILER=/home/andy/tools/gcc/arm-gnu-toolchain-12.3.rel1-x86_64-aarch64-none-linux-gnu/arm-gnu-toolchain-12.3.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-g++ -DGGML_NATIVE=OFF -DGGML_AVX=off -DGGML_AVX2=off -DGGML_AVX512=off -DGGML_FMA=off -DGGML_F16C=off -DGGML_CPU_ARM_ARCH=armv9-a+sve -DCMAKE_SYSTEM_PROCESSOR=armv9-a -DCMAKE_OSX_ARCHITECTURES=arm64  -DCMAKE_SYSTEM_NAME=Linux ..

Relevant log output

-- Found Vulkan: /home/andy/cix-linux-master-debian/output/cix_evb/sysroot/usr/lib/aarch64-linux-gnu/libvulkan.so (found version "1.3.239") found components: glslc glslangValidator
-- Vulkan found
-- GL_KHR_cooperative_matrix not supported by glslc
-- GL_NV_cooperative_matrix2 not supported by glslc
-- GL_EXT_integer_dot_product not supported by glslc

[jeffbolznv]

But I think the glslc check logic is not solid in the cmd above, since the glslc is from x86，which cannot get the features for the taeget arm64 soc gpu env.

glslc doesn't get features from the target, regardless of which version you use. It's totally platform independent and it shouldn't matter which version you use. But you should try to use the newest version you can, to get the most features. The version in the Android NDK is very old and you'll be better off with something newer.

[mtmcp]

@andyt9527 Please take a look at the CI build logic here: https://github.com/ggml-org/llama.cpp/blob/dc39a5e7a84815a90fa0c515ed8927870cf858c9/.github/workflows/build-linux-cross.yml#L85

In particular, if you are running at least Ubuntu 24.04, the built-in GLSLC dependencies may be used. The CI build above uses the Ubuntu ports repository to pull in the libvulkan-dev:arm64, but since your command has its own sysroot and Vulkan libraries, you may just need to pull in glslc to get the system binaries (which won't need the ports).

          sudo apt-get install -y --no-install-recommends \
                  build-essential \
                  glslc \
                  crossbuild-essential-arm64 \
                  libvulkan-dev:arm64 \
                  libcurl4-openssl-dev:arm64

Here's the package on Ubuntu repository for convenience: https://packages.ubuntu.com/noble/glslc

[mtmcp]

So if we can add a cross compile check and use some build args for this?

That being said, @andyt9527 are you asking for flags to be able to forcibly turn off specific Vulkan features?

[andyt9527]

So if we can add a cross compile check and use some build args for this?

That being said, @andyt9527 are you asking for flags to be able to forcibly turn off specific Vulkan features?

Yes, I think this would be more flexible by adding some options in the CMake parameters to customize Vulkan features. Since we observe some vulkan feature are taken as enabled, but actually which is not available on the target arm64 mali gpu devices

[jeffbolznv]

There is a runtime check on the device to check for feature support. It's totally fine to build with the additional features enabled.

[mtmcp]

@andyt9527 Please verify the build works for you using the system-provided glslc as I mentioned above, or another glslc (as opposed to the one provided by Android NDK). It seems what you are attempting is a non-standard approach, and we would not like to have to add unnecessary features. Thank you!

[andyt9527]

@andyt9527 Please verify the build works for you using the system-provided glslc as I mentioned above, or another glslc (as opposed to the one provided by Android NDK). It seems what you are attempting is a non-standard approach, and we would not like to have to add unnecessary features. Thank you!

Yes, I got the point, and already updated the build process. But seems the vulkan output is not correct on mali gpu. Let me check further.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.