Ronan McGovern
Ronan McGovern
@FlorianJoncour could you share a sample of a fully formatted prompt containing tools? Say, for a Mistral model?
> @FlorianJoncour could you share a sample of a fully formatted prompt containing tools? Say, for a Mistral model? Is it possible to share a fully formatted prompt sample? That...
Any tips on how to reduce VRAM requirements? I'm training the 2.8B Mamba and I'm oom on 16k context on an A100 80GB. Batch size of 1. I guess the...
Thanks appreciate that, yeah I'll try that next time - reinstalling mamba-ssm On Mon, Mar 11, 2024 at 11:32 AM lqf0624 ***@***.***> wrote: > Any tips on how to reduce...
Thanks @danielhanchen , could you tag here when there is a PR so I can Subscribe to it?
> @RonanKMcGovern Yes! We had a chat on our Discord server about this! It looks very promising, and it removes lora_alpha (finally!!!) 1 less hyperparameter! > > Love how it...
I also tried quanting with mlx for the qwen1.5 4B model and there are no tokens generated when I run inference.
> > There are lots of models on HF which are only offered in either F16 of exl2 format > > Could you point to some ? > > Exl2...
any updates here?
Yes, +1 to this. As I understand, supporting Llama models would be straightforward @SunMarc @younesbelkada ?