honuvo comments

Repositories
Issues
Comments

Results 2 comments of


                                            honuvo

Bug: Qwen2-72B-Instruct (and finetunes) Q4_K_M, Q5_K_M generates random output with CuBLAS prompt processing

Hi, to add to this: My observations are the same. Random tokens after a few messages. I may have some new information. I don't have those problems when using a...

Bug: Qwen2-72B-Instruct (and finetunes) Q4_K_M, Q5_K_M generates random output with CuBLAS prompt processing

> Looks like Q5km works ok in latest koboldcpp if I use OpenBLAS instead of CuBLAS for prompt processing. It's slow, though. EDIT: CLBlast works too on GPU. Wow, indeed,...