Kanishk Navale
Kanishk Navale
Use this to install: CMAKE_ARGS="-DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=80" pip install llama-cpp-python # additional things...
@SubatomicPlanets Agreed 💯
Hello @yutyan0119 @donguyen32 @abetlen Could you please share any progress on this? I think this is a super important PR!
I reviewed the PR now. However, I have been using that branch for re-ranking and it works fluently for me.