Spijkervet
exclude_from_weight_decay added
This modified LARS code was tested on CIFAR-10 with ResNet-50 backbone. Linear Evaluation Test Accuracy is ~85% after 100 epochs.
This modified LARS code was tested on CIFAR-10 with ResNet-50 backbone. Linear Evaluation Test Accuracy is ~85% after 100 epochs.
Hey, I am not able to achieve the linear evaluation accuracy on CIFAR10 with Resnet-50 reported in paper with this repo code. After modifying the LARS code, you are getting around 85%, so may I please know what hyper-parameters (like batchsize, LR, temperature) did you use? Also, 100 epochs is for Contrastive training or linear evaluation?
Thank you.
@Ektagavas Hi, I used a batch size of 64, LR 0.2, and temperature 0.5. 100 epochs for contrastive learning. Also, I used a pre-trained encoder. The results may differ if you train an encoder from scratch.
@Ektagavas Hi, I used a batch size of 64, LR 0.2, and temperature 0.5. 100 epochs for contrastive learning. Also, I used a pre-trained encoder. The results may differ if you train an encoder from scratch.
@sadimanna Thanks for your quick response and for sharing the details! Can you pls also share from where you got your pre-trained encoder?
Thank you.
@Ektagavas I used
torchvision.models.resnet50(pretrained = True)
Oh I see, the PyTorch's pre-trained model. Cool, thanks! :)