ALPRO Weight Decay

Hi, as stated in the issue, the ALPRO does use weight decay. But I did not find the process that passing the parameter "weight_decay" during the optimizer initialization.

optimizer = OptimCls(model.parameters(), lr=opts.learning_rate, betas=opts.betas)

Apr 21 '22 05:04 vateye

Thanks a lot for pointing this out. It seems the current repo indeed does not pass in the weight decay. This may be an issue during open-sourcing. We'll update the repo with required fix.

Apr 21 '22 15:04 dxli94

Thanks a lot for pointing this out. It seems the current repo indeed does not pass in the weight decay. This may be an issue during open-sourcing. We'll update the repo with required fix.

Hi, any following for this question?

May 27 '22 13:05 vateye

Hi @vateye, an easy fix would be to pass the weight_decay to the optimizer.

We will resolve this issue in future releases but would expect some delay.

Nice catch and thanks for your kind understanding.

May 30 '22 15:05 dxli94