Meaning of joint training

[htn274]

First of all, thank you very much for your contributions. I have 2 questions now:

1. Does SupCon Model show better accuracy than traditional cross entropy if augmentation isn't used? I asked because I want to apply the model to a non-image dataset. At the present, I don't know how to augment this dataset. I think it is possible from your loss. However I am not sure. So I would like to ask for your opinions.
2. I would like to verify the meaning of jointly training for linear supervised model and SupCon model:

for each epoch:
    1. Train SupCon model 
    2. Freeze SupCon model: model.eval()
    4. Train linear supervised model
    5. Evaluate linear supervised model on validation set

Is the steps correct for jointly training?

Hope to receive your response soon. Thanks a lot!

[htn274]

2. I implemented jointly train version, as I tested. It runs correctly as it produced results same as seperate training. Anyone who has interest can check the code here

[htn274]

1. However when I tried to train the model without augmentation on CIFAR10 dataset, the classifier model becomes overfitting after 20 epochs.

Note: red line is train accuracy, green line is val accuracy

Is it normal to be overfitting in this case? Anyway it just my first step because I would like to apply the model into a custom dataset.

[Angelina1996]

2. I implemented joint train version, as I tested. It runs correctly as it produced results same as seperate training. Anyone who has interest can check the code here

Thanks so much, Nu Hoang, for providing the joint training code. I am wondering if you could guide how to execute supcon code for the Imagenet dataset, it says it has used the moco trick, however, I could not find any help regarding this. Cheers, Angelina