llm icon indicating copy to clipboard operation
llm copied to clipboard

Explain differences from llama.cpp in README

Open philpax opened this issue 3 years ago • 2 comments

llama.cpp now has a C interface, so we could theoretically switch to using it directly.

However, we don't want to do this for a few reasons:

  • You still need a C++ compiler, which complicates deployment to other platforms
  • Rust makes the code easier to work with
  • We want to make ggml an optional backend in future (#31)

philpax avatar Mar 23 '23 10:03 philpax

I don't understand why llama.cpp wasn't created using Rust in the first place (like a llama.rs), what advantage did the author see in using C++ instead of Rust?

hermesonbf avatar Mar 24 '23 17:03 hermesonbf

He hasn't seen the light yet, that poor lost soul.

The real answer is probably because C++ is what he knows/is experienced with and also had already written stuff like GGML and Whisper in C++ already so he could just reuse a lot of it.

KerfuffleV2 avatar Mar 24 '23 20:03 KerfuffleV2