leikareipa
leikareipa
Adding results from Hugging Face Playground (should be the full BF16) for the fisheye prompt: | Q4 | Q8 | Q8 no GPU | HF Playground | | --- |...
The issue isn't whether there's some difference but that it appears to be categorical and fairly large, to the point of looking like a problem. It may also extend beyond...
I'm certain it's QwQ and there's only one revision of it on their Hugging Face. They did update their chat template the next day though, which my Q4 could have...
Writeup/condensement of the results, and some extra data: https://leikareipa.github.io/blog/comparing-quants-of-qwq-preview-in-ollama/
Odd that this was closed without testing, since as you say the behavior looks unusual.