Blair Sadewitz
Blair Sadewitz
> Error solved by upgrading to gcc-11. Try that first. That's what I did, and the error resolved.
Why don't you use KoboldAI? (just curious)
Aah, thanks! It is obvious in retrospect, I'm amused I didn't think of that.
oh, BTW, FWIW, I got so annoyed at the fact that ./install_requirements.sh had to run the thing twice because of that bug I was like "Ok, that's _IT_. NEVER AGAIN....
Yeah, that's essentially the logic I'm talking about. I approached it with the assumption that you'd want to make it "experimental" at first, but, if not, I surely won't complain....
Being able to use a monkey patch would be cool, too, but I assume that's even more work.
What I am most interested in is being able to use models which use this: https://github.com/bhenrym14/qlora-airoboros-longcontext/blob/main/scaledllama/llama_rope_scaled_monkey_patch-16k.py Most of them are 8k. https://huggingface.co/TheBloke/airoboros-33B-gpt4-1-4-SuperHOT-8K-fp16/tree/main
Oh, OK, fair enough. Whenever you have a spare moment, would you kindly tell me where in the code the call is which loads a 16-bit llama-based model (you know,...
Hmm, yeah, I'm having some issues with it. :( Check this out, though: RoPE scaling got merged to transformers. Models don't have to be pretrained to use it, though apparently...
I just noticed everything you merged. Thanks! I'd been hopping between forks, and this makes my life a lot easier.