BYOL-PyTorch icon indicating copy to clipboard operation
BYOL-PyTorch copied to clipboard
verified publisher icon yaox12

Will apex opt_level affect performance?

Open SunDoge opened this issue 5 years ago • 2 comments

I notice that you set the opt_level='O0', which is FP32 training instead of mixed-precision training. What would happen if using opt_level=O1 or higher opt_level?

SunDoge avatar Dec 25 '20 13:12 SunDoge

I have tried opt_level=O1 and O2, they gave very close results but didn't show much speedup in training time. I guess it is due to the communication bottleneck.

yaox12 avatar Dec 27 '20 06:12 yaox12

Thx.

SunDoge avatar Dec 27 '20 14:12 SunDoge