esm icon indicating copy to clipboard operation
esm copied to clipboard
verified publisher icon evolutionaryscale

How to finetune ESMC

Open bj600800 opened this issue 11 months ago • 7 comments

I'm looking forward to fine - tuning ESMC by incorporating other specialized knowledge. Could you please tell me how to achieve that?

bj600800 avatar Feb 06 '25 03:02 bj600800

same question

peiyaoli avatar Feb 08 '25 06:02 peiyaoli

same too

lzygitk7 avatar Feb 08 '25 12:02 lzygitk7

same

chaofan520 avatar Feb 10 '25 13:02 chaofan520

Same，looking farward the answer.

tangmeiaoxue1 avatar Feb 18 '25 02:02 tangmeiaoxue1

download the model and unfreeze the weights then fine-tune

sermare avatar Feb 18 '25 22:02 sermare

Reply to sermare: The model did not integrated with the hugging face official package. Failed to use Automodel to load it.

nullland1027 avatar Mar 04 '25 13:03 nullland1027

Hi @bj600800 @peiyaoli @lzygitk7 @chaofan520 @tangmeiaoxue1 @sermare @nullland1027 ,

My group has put together an implementation of ESMC called ESM++ that is completely Huggingface compatible. It loads with AutoModel and can be trained with or without PEFT. Please reference our readme on Huggingface for details or check out our example fine-tuning script if you think it would be helpful. Happy to answer any questions, feel free to open up issues here. Best, Logan

lhallee avatar Mar 04 '25 21:03 lhallee