li-plus
[feature] Add AMD GPU support through ROCm
Add CMake flag in CMakeLists.txt refer to llama.cpp
Compile with args:
cmake -B build -DGGML_HIPBLAS=ON -DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang -DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++ && cmake --build build -j
Tested on Vega 56 and ChatGLM3 work well.
Need more test to verify?
⚠️ I later figured out the test was polluted by another build with
OPENBLASin the same environment. And I actually went into issues for ROCm build...I have solved the problems now... Except:
/workspace/chatglm.cpp.hip/third_party/ggml/src/ggml-cuda.cu:1577:38: error: 'x' is a protected member of '__half2' reinterpret_cast<half&>(y[ib].ds.x) = d;
Excuse me, did you compile under Windows or Linux? I'm failing in windows cmake, is there any result
lld-link: error: could not open 'm.lib': no such file or directory
I built with ROCm in Ubuntu with Docker image rocm/dev-ubuntu-18.04:4.2-complete
According to your description, I believe it is because of the environment path in the CMake file... I hardcoded the library path in the docker image.
Actually I find some problems with my CMake file. It will raise error say during building chatglm.cpp linker cannot find reference of functions in ggml.cpp. I have been confusing about it for few days.
ubuntu22.04，rocm5.7.1，6800xt cmake build提示： CMake Warning: Manually-specified variables were not used by the project: GGML_HIPBLAS
llama.cpp可以： -- hip::amdhip64 is SHARED_LIBRARY -- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS -- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS - Success -- hip::amdhip64 is SHARED_LIBRARY -- HIP and hipBLAS found AMD LLD 17.0.0 (compatible with GNU linkers) -- CMAKE_SYSTEM_PROCESSOR: x86_64 -- x86 detected -- Configuring done (0.9s) -- Generating done (0.0s) -- Build files have been written to:
ubuntu22.04，rocm5.7.1，6800xt cmake build提示： CMake Warning: Manually-specified variables were not used by the project: GGML_HIPBLAS
llama.cpp可以： -- hip::amdhip64 is SHARED_LIBRARY -- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS -- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS - Success -- hip::amdhip64 is SHARED_LIBRARY -- HIP and hipBLAS found AMD LLD 17.0.0 (compatible with GNU linkers) -- CMAKE_SYSTEM_PROCESSOR: x86_64 -- x86 detected -- Configuring done (0.9s) -- Generating done (0.0s) -- Build files have been written to:
Are you using CMakeList.txt from here?
I finally figured out what is wrong with my script in my side...
As the only error I encounter is:
...
/workspace/chatglm.cpp.hip/third_party/ggml/src/ggml-cuda.cu:12:10: fatal error: 'hipblas/hipblas.h' file not found
...
I tested everything I can do with the CMakeList.txt cannot help me out of this.
It end up to be a problem of my docker environment rocm/dev-ubuntu-20.04:4.2-complete the include path in hipblas do not have folder hipblas, i.e. I have to edit ggml-cuda.cu include hipblas using #include <hipblas.h> not #include <hipblas/hipblas.h>
@CellerX I have checked and patched the CMakeList.txt as much as I can. Can you verify if the problem still occur?