haihuangcode
self.audio_semantic_decoder and self.Audio_decoder
https://github.com/haihuangcode/CMG/blob/2cbdad8f68d6000657ddf45ace97c855c022334d/code/src/model/main_model_2.py#L507C1-L515C60
Hi sir! Thanks for your great work! I have some questions I would like to ask you. I don't know if it's right to understand it this way: self.audio_semantic_decoder and self. Audio_decoder are used for classification and feature reconstruction, respectively. I also have a question about whether this work is using a transformer model? because I noticed a UniEncoder.py file
Looking forward to hearing from you!
Hello, thank you for your interest in our work.
The code only uses feature reconstruction in the loss function, classification is there but does not contribute to the loss. This is primarily because we are doing unsupervised pretrain, so we left that part in case we want to extend it later.
We did not actually use a transformer, the UniEncoder code was from an earlier attempt we tried but the results were not very satisfactory. So it was just left in the code but not really used.
Not all released code is useful, some are previous attempts or discarded code, and the core code is in
pretrain.py main_model_2.py CPC.py models.py CLUB.py.
