epinnock

> Thank you! That resolved the issue with the devices. > > However, now the following error appears: > > ``` > [/usr/local/lib/python3.8/dist-packages/peft/tuners/lora.py]) in forward(self, x) > 446 return F.linear(x,...

Hi @TimDettmers Is this something that would be worth looking into again, I'm out of my league here but willing to help in any way fp8 on the 4090 would...

How hard would it be to implement a really naive version of this with ctranslate2? I would like to pick this up if possible

Hi 👋 Wanted to follow up on this some more I'm currently attempting to finteune [TinyLlama/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T) Using LongLora I'm currently getting the following error. Per their Github [TinyLlama](https://github.com/jzhang38/TinyLlama) architecture should...

@Eric-mingjie Hi I had a question on this as well would it be possible to prune a 7b model to 50% and further quantize it to 4bit. If so what...