facebookexperimental
make bpftools build fails: `ld.lld: error: unable to find library -lLLVM` when linking `bpftrace-aotrt`
Steps to reproduce Execute standard steps:
# Build the Docker image
./scripts/build-docker-image.sh
# Run the environment
./scripts/run-docker-build-env.sh
# clean the project
make clean
# Create the artifact in out/archives/bpftools-arm64.tar.gz
make bpftools THREADS=$(nproc)
Description
Following the steps, the build stops at 96% during the bpftrace step with error "unable to find library -lLLVM" during the phase "Linking CXX executable bpftrace-aotrt". The same issue also occurs if I use make bpftools-min.
Can you fix this issue?
uname -r
5.14.0-503.40.1.el9_5.x86_64
podmav -v
podman version 5.2.2
Thank you for the awesome project and thanks in advance for for your reply.
** Logs**
make[3]: Leaving directory '/ExtendedAndroidTools/build/android/arm64/bpftrace'
[ 64%] Built target runtime
[ 87%] Built target ast
make[3]: Entering directory '/ExtendedAndroidTools/build/android/arm64/bpftrace'
make[3]: Entering directory '/ExtendedAndroidTools/build/android/arm64/bpftrace'
make[3]: Leaving directory '/ExtendedAndroidTools/build/android/arm64/bpftrace'
Consolidate compiler generated dependencies of target libbpftrace
make[3]: Entering directory '/ExtendedAndroidTools/build/android/arm64/bpftrace'
**[ 88%] Linking CXX executable bpftrace-aotrt
make[3]: Leaving directory '/ExtendedAndroidTools/build/android/arm64/bpftrace'
ld.lld: error: unable to find library -lLLVM
clang++: error: linker command failed with exit code 1 (use -v to see invocation)**
make[3]: *** [src/aot/CMakeFiles/bpftrace-aotrt.dir/build.make:112: src/aot/bpftrace-aotrt] Error 1
make[3]: Leaving directory '/ExtendedAndroidTools/build/android/arm64/bpftrace'
make[2]: *** [CMakeFiles/Makefile2:638: src/aot/CMakeFiles/bpftrace-aotrt.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....
make[3]: Entering directory '/ExtendedAndroidTools/build/android/arm64/bpftrace'
[ 89%] Building CXX object src/CMakeFiles/libbpftrace.dir/clang_parser.cpp.o
/ExtendedAndroidTools/projects/bpftrace/sources/src/clang_parser.cpp:387:25: warning: moving a temporary object prevents copy elision [-Wpessimizing-move]
387 | enum_name = std::move(name.str());
| ^
/ExtendedAndroidTools/projects/bpftrace/sources/src/clang_parser.cpp:387:25: note: remove std::move call here
387 | enum_name = std::move(name.str());
| ^~~~~~~~~~ ~
1 warning generated.
[ 91%] Linking CXX static library libbpftrace.a
make[3]: Leaving directory '/ExtendedAndroidTools/build/android/arm64/bpftrace'
[ 96%] Built target libbpftrace
make[2]: Leaving directory '/ExtendedAndroidTools/build/android/arm64/bpftrace'
make[1]: *** [Makefile:136: all] Error 2
make[1]: Leaving directory '/ExtendedAndroidTools/build/android/arm64/bpftrace'
make: *** [projects/bpftrace/build.mk:25: build/android/arm64/bpftrace.done] Error 2
I guess it's because you didn't use android-ndk-r27b to build. I had the same problem when using android-ndk-r23b or r25b.
Hi @4399erbai,
The scripts/download-ndk.sh script automatically downloads android-ndk-r27b, so the build should never fall back to r23b/r25b.
Quick check:
./scripts/run-docker-build-env.sh # start the container
ls -al /opt/ndk # inside the container
drwxr-xr-x. 11 root root 4096 Aug 29 2024 android-ndk-r27b
The repository has changed
The codebase has been updated since the issue was opened, and I now get a different error regardless of whether I set NDK_DIR.
Note: When I first opened the issue, I solved it using the procedure in the README 👉 https://github.com/alfonsoalongi/bpftools-builder/blob/main/README.md
Current build log
export NDK_DIR=/opt/ndk/android-ndk-r27b
make bpftools-min NDK_ARCH=arm64 THREADS=$(nproc)
First run
...
cd build/host/llvm && /ExtendedAndroidTools/out/host/bin/cmake /ExtendedAndroidTools/projects/llvm/sources/llvm \
-DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra" -DLLVM_TARGETS_TO_BUILD="AArch64;BPF" \
-DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=1 \
-DCMAKE_BUILD_TYPE=Release
CMake Error: The source directory "/ExtendedAndroidTools/projects/llvm/sources/llvm" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
make: *** [projects/llvm/build.mk:74: build/host/llvm] Error 1
Second run
...
cd build/host/llvm && make -j 20 clang-tblgen
make[1]: *** No rule to make target 'clang-tblgen'. Stop.
make[1]: Leaving directory '/ExtendedAndroidTools/build/host/llvm'
make: *** [projects/llvm/build.mk:68: out/host/bin/clang-tblgen] Error 2
It looks like the LLVM clone is being interrupted, leaving the build directory incomplete and causing the subsequent clang-tblgen failure.
I’m investigating this, as I don’t remember whether the same issue appeared in my CI workflow for releasing the artifacts (https://github.com/alfonsoalongi/bpftools-builder/blob/main/.github/workflows/bpftools-release.yml). However, the latest workflow currently works.
In any case, with the previous commit, the approach in the README still works (first step: build LLVM + Clang for the host). I haven’t tested it with the new version yet.