fairseq Is it possible to finetune the MMS model?

Is it possible to finetune the MMS model? I didn't find any doc or code supporting this feature.

May 30 '23 02:05 CopyNinja1999

We're working on making this very easy in transformers. You can check:

https://huggingface.co/docs/transformers/main/en/model_doc/mms
https://github.com/huggingface/transformers/pull/23813
https://huggingface.co/facebook/mms-1b-all

Stay tuned!

Jun 02 '23 11:06 patrickvonplaten

@patrickvonplaten Hi, could you please also upload the pretrained discriminator for mms-tts models? Also, I met this error when fine-tuning the Korean tts model using random discriminator checkpoint: packages/bitsandbytes/optim/optimizer.py", line 455, in update_step if state["state1"].dtype == torch.float: KeyError: 'state1'

Jun 10 '23 06:06 CopyNinja1999

https://github.com/huggingface/transformers/tree/main/examples/pytorch/speech-recognition#connectionist-temporal-classification-with-adapters

Jun 20 '23 13:06 allandclive

Also a more step-by-step blog post here: https://huggingface.co/blog/mms_adapters

Jun 21 '23 10:06 patrickvonplaten