DeepSpeed-MII icon indicating copy to clipboard operation
DeepSpeed-MII copied to clipboard
verified publisher icon microsoft

Mistral 8*7B Out of memory

Open byerose opened this issue 2 years ago • 1 comments

Does the framework support the mistral 87B model perfectly? I encountered an Out of Memory error during use. The machine is 8*A100 80G. image

byerose avatar Jan 11 '24 06:01 byerose

Hi @byerose, yes we do support Mixtral 8x7B model. Can you please share the script you are using? I have been able to run this model on as little as 2xA6000

mrwyattii avatar Jan 11 '24 17:01 mrwyattii