llama-cpp-swift
llama-cpp-swift copied to clipboard
Running crash by llama_tokenize
Platform: MacOS13.6.1 Xcode 15.0
ScreenShot:
I download the model(https://hf-mirror.com/NikolayKozloff/DeepSeek-R1-Distill-Qwen-1.5B-Multilingual-Q8_0-GGUF) in local, then runnning as same as the simple.
let modelPath = "/Users/xxx/Documents/xxx/DeepSeek-R1-Distill-Qwen-1.5B-Multilingual-Q8_0-GGUF/deepseek-r1-distill-qwen-1.5b-multilingual-q8_0.gguf"
let model = try Model(modelPath: modelPath)
let llama = LLama(model: model)
let prompt = "what is the meaning of life?"
for try await token in await llama.infer(prompt: prompt, maxTokens: 1024) {
print(token, terminator: "")
}
PS: Use Llama-cli can running successful!
I got the same problem.