Josh Leverette
Josh Leverette
While you're quantizing models, I also have noticed several times that the mini-4k model only has fp16 and q4_K_M. I guess it would be cool to have the rest of...
Getting the new Phi-3 models working is going to require a newer version of llama.cpp than ollama currently has. Some of the required changes were merged into llama.cpp only a...
And I see [this](https://github.com/ollama/ollama/releases/tag/v0.1.39-rc1) popped up a few seconds ago!
`phi3:14b-medium-4k-instruct-q8_0` seems to be working well in some quick testing!
I also see similar things… simply saying “Hello” does seem to cause problems for it here. I’m seeing both `**Instruction` and `### Instruction` just before it goes off the rails,...
This could also be related: https://github.com/ggerganov/llama.cpp/pull/7449
I'd like to see this as well. Blocked by this issue: https://github.com/ggerganov/llama.cpp/issues/7116
Support for the 20B and 34B models seems fine now in llama.cpp, but the latest release of ollama is still missing the necessary commits. I [have uploaded](https://github.com/ggerganov/llama.cpp/issues/7116#issuecomment-2120536190) a few GGUFs,...
Duplicate? https://github.com/ollama/ollama/issues/4026
@not-nullptr because of the Let’s Think Dot by Dot paper I linked? And in my previous reply, I was extremely clear that whitespace after the end of the valid JSON...