llama-cpp-swift icon indicating copy to clipboard operation
llama-cpp-swift copied to clipboard

Running crash by llama_tokenize

Open linxi5547371 opened this issue 11 months ago • 1 comments

Platform: MacOS13.6.1 Xcode 15.0

ScreenShot: Image

I download the model(https://hf-mirror.com/NikolayKozloff/DeepSeek-R1-Distill-Qwen-1.5B-Multilingual-Q8_0-GGUF) in local, then runnning as same as the simple.

let modelPath = "/Users/xxx/Documents/xxx/DeepSeek-R1-Distill-Qwen-1.5B-Multilingual-Q8_0-GGUF/deepseek-r1-distill-qwen-1.5b-multilingual-q8_0.gguf"

let model = try Model(modelPath: modelPath)
let llama = LLama(model: model)
let prompt = "what is the meaning of life?"

for try await token in await llama.infer(prompt: prompt, maxTokens: 1024) {
    print(token, terminator: "")
}

PS: Use Llama-cli can running successful!

linxi5547371 avatar Mar 18 '25 08:03 linxi5547371

I got the same problem.

orlandoxu avatar Jul 12 '25 02:07 orlandoxu