uda icon indicating copy to clipboard operation
uda copied to clipboard
verified publisher icon google-research

BatchNorm in WRN

Open bkj opened this issue 5 years ago • 1 comments

Hi --

I noticed that the last BatchNorm in WRN is always set to is_training: https://github.com/google-research/uda/blob/master/image/randaugment/wrn.py#L117

is_training changes for all of the other BNs. Is this intentional? Does it give some kind of performance advantage?

Thanks!

bkj avatar Sep 24 '20 20:09 bkj

Digging into this deeper -- seems like using batch statistics vs running statistics make a fair bit of difference in the convergence of the model. Do you have a good explanation for that? Seems interesting + surprising to me.

bkj avatar Sep 25 '20 03:09 bkj