SalomonKisters
SalomonKisters
So this can now be used to transcribe live from an audiostream like mic input? If yes, do you maybe know how?
Same issue here.
> @AlpinDale nope, same stack trace, Also checked with an awq non mixtral moe and works like a charm inclusive with the --quantization awq + load in 4 bits, I...
> @SalomonKisters in a 4090 I would say that the increase of speed it is noticeable to the naked eye (did not benchmarked yet) Okay, sounds nice. So you just...
> yes, but is not working for Moe's, for moes gptq is the best option now I think... Ah makes sense, but for GPTQs it doesnt work combined with --load-in-4bit,...
having the issue again, none of the above fixes worked for me
Any updates?