llm
llm copied to clipboard
Explain differences from llama.cpp in README
llama.cpp now has a C interface, so we could theoretically switch to using it directly.
However, we don't want to do this for a few reasons:
- You still need a C++ compiler, which complicates deployment to other platforms
- Rust makes the code easier to work with
- We want to make ggml an optional backend in future (#31)
I don't understand why llama.cpp wasn't created using Rust in the first place (like a llama.rs), what advantage did the author see in using C++ instead of Rust?
He hasn't seen the light yet, that poor lost soul.
The real answer is probably because C++ is what he knows/is experienced with and also had already written stuff like GGML and Whisper in C++ already so he could just reuse a lot of it.