llama_cpp.rb

llama_cpp.rb copied to clipboard

Error when trying to run the chat.rb example.

1

When trying to run the chat example I am getting an error. ruby chat.rb --model /playingwithai/models/llama-2-7b-chat.Q8_0.gguf llama_model_loader: loaded meta data with 19 key-value pairs and 291 tensors from /playingwithai/models/llama-2-7b-chat.Q8_0.gguf (version...

mybuddyandrew

Hello there! Thanks for providing this Gem! I'm new to llama.cpp and was wondering how I might use the following `llava-cli` command to use with the Ruby bindings: ``` ./llava-cli...

danielpuglisi

Chat.rb example: Crash when prompt exceeds context?

1

When the position n_past and the upcoming embedding exceeds the context size there is a bug in the code: ``` if n_past + embd.size > n_ctx ... embd.insert(0, last_n_tokens[(n_ctx -...

coezbek

How to ensure CUDA is working?

1

I installed the gem using `--cublas`, but don't see any difference when running. Is there a way to know that cublas is being used?

coezbek

Improving the quality of the results

2

Hi, I'm using this with `langchainrb`(https://github.com/andreibondarev/langchainrb). The purpose is to chain user added text as a "context" and create a response based on the "prompt. 1. First, I store user...

mldev94

Hi! Thanks for this library. By default it emits quite a lot of logs and I would like to disable them. How to achieve that? Thanks - V

vemv

Cannot build

3

Hello, I cannot build this library on my Mac (15.1.1 M2): ``` bundle exec rake lib/llama_cpp/llama_cpp.bundle -- --with-opt-dir=/opt/homebrew ``` (in /Users/kitaitimakoto/src/github.com/yoshoku/llama_cpp.rb) /usr/bin/make cd tmp/arm64-darwin24/llama_cpp/3.3.6 compiling ../../../../ext/llama_cpp/llama_cpp.cpp In file included from...

KitaitiMakoto

Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 6. Release notes Sourced from actions/checkout's releases. v6.0.0 What's Changed Update README to include Node.js 24 support details and requirements by @​salmanmkc in actions/checkout#2248 Persist...

dependabot[bot]

Build issue on ruby 3.4.5/Rocky Linux 10

1

Having issues installing the llama_cpp gem on ruby 3.4.5/RL-10. As you can see below, the prerequisites are indeed installed. thanks thornhill@abydos:~$ which ruby ~/local/bin/ruby thornhill@abydos:~$ ruby --version ruby 3.4.5 (2025-07-16...

mrsdavis5845