Support for Codellama Model in deepspeed-fastgen

Open LinKiling opened this issue 2 years ago • 2 comments

It seems that the current implementation does not include compatibility for Codellama. I would like to propose the inclusion of CodeLlama support in deepspeed-fastgen

Dec 01 '23 07:12 LinKiling

Hi @LinKiling it looks like we are able to load the Codellama models and run generation, but the output seems a bit off. I'll ask our kernel devs to take a look. Thanks!

Dec 05 '23 21:12 mrwyattii

Hi @LinKiling it looks like we are able to load the Codellama models and run generation, but the output seems a bit off. I'll ask our kernel devs to take a look. Thanks!

any updates on this? i also get wrong output on codellama

Jan 02 '24 02:01 Tmn07