llm Support for SalesForce CodeGen

https://github.com/salesforce/CodeGen https://huggingface.co/docs/transformers/v4.28.1/en/model_doc/codegen#transformers.CodeGenForCausalLM

Apparently one of the best for code-generation.

Apr 20 '23 00:04 philpax

Can you assign this to me

May 01 '23 16:05 hhamud

GPT-J is very similar to CodeGen, but not quite: https://gist.github.com/moyix/7896575befbe1b99162ccfec8d135566 (h/t @hhamud)

Once GPT-J's merged, we can look at either adding an option to GPT-J or setting up a separate CodeGen model that shares 99% of the logic.

May 03 '23 03:05 philpax

GPT-J is very similar to CodeGen, but not quite: https://gist.github.com/moyix/7896575befbe1b99162ccfec8d135566 (h/t @hhamud)

Once GPT-J's merged, we can look at either adding an option to GPT-J or setting up a separate CodeGen model that shares 99% of the logic.

I've noticed that others just use that script to convert codegen to be able to run on code that is based on GPT-J's model.

May 03 '23 15:05 hhamud

Are there any SFCG GGML models floating around? Are we the first ones to try this?

May 03 '23 17:05 philpax

I haven't found any that bothers with integrating it as it is into ggml rather than converting it into a gpt-j compatible implementation

May 18 '23 15:05 hhamud