JP Lorandi

icon indicating a website link https://azumo.com/nearshore-software-development

icon company Azumo, LLC icon location Rosario, SF, Argentina icon location CTO @ Azumo https://azumo.com/nearshore-software-development

Results 2 issues of JP Lorandi

Support Peft Models (LoRA)

I'm running the HF model `curiousily/falcon-7b-qlora-chat-support-bot-faq` on top of `tiiuae/falcon-7b` and it runs quite fast, but unfortunately there's no way to load it in lmql. I created the PeftModel class,...

enhancement

Additional backends aren't supported with serve-model

1 comment

I implemented a PeftLLM backend (which I pasted into #152 ) but I cannot load it since there's no way to insert it into the registry via serve-model. I think...

enhancement