llama-cpp-python
llama-cpp-python copied to clipboard
abetlen
Python bindings for llama.cpp
I updated llama.cpp and compiled with vulkan=1 but I get an error about compiling with -fPIC enabled.
The existing dockerfile for cuda has a few flaws - It builds the pypi package instead of the local repository - Image requires too much space ~4GB This change rectifies...
# Prerequisites Please answer the following questions for yourself before submitting an issue. - [x] I am running the latest code. Development is very rapid so there are no tagged...
# Prerequisites Please answer the following questions for yourself before submitting an issue. - [x] I am running the latest code. Development is very rapid so there are no tagged...
Hi, I have built a rag app and I am loading a LLM with Llamacpp. However I have problems with making Streaming work for FastAPI or Langserve requests. Streaming is...
Hi Team, i tried all Dockerfile, but none is able to create image out of it. Is there something missing in dockerfile or any pre-requisite needed for dockerfile to run
llamacpp has recently added command-r suppport. Can we get it for llamacpp-python? https://github.com/ggerganov/llama.cpp/commit/12247f4c69a173b9482f68aaa174ec37fc909ccf https://huggingface.co/CohereForAI/c4ai-command-r-v01
@abetlen Thank you for the new efforts to start publishing wheels for CUDA, etc. I noticed that the METAL wheels only work for darwin platform, when using Docker in MacOS...
