Don-Chad
Don-Chad
Guys what size GPU's are you running it at? My does hangs at.. 'Loading model...'; I have a 3090 24GB. Should be enough, I hope?
Yes please! Then one step further - how hard would it be to allow 4bit?
@Priestru would be amazing if you could share how you used the LCM model! Just replacing the standard does not work..
I am not ure it's possible yet via the api. Hopefully soon!
> I think category and name are required props! Appreciate the reply! I had those in also, but still *silence* unfortunately. Did it work for you? If anyone is able...
> Yesterday, I could make work, my steps bellow: > > 1. Upgrade 11labs to a paid plan > 2. Add category and name in the voice (if the voice...
Exactly! Did you get any further? do the other parts allow this?
From what I know AutoGPTQ does something smart with the data - working from the training prompts to do a high quality compression, to offer 32 bit quality at 4...
@haotian-liu Would you please want to share the settings for the AutoGPTQ quantisation - if any? I would like to see if I can get GPTQ-for-LLaMa working.
Yes this combination would be a perfect approach! I would be happy to do new training's and provide the GPU power for it. We could also have smaller models initially....