CSRA icon indicating copy to clipboard operation
CSRA copied to clipboard
verified publisher icon Kevinz-code

Is 'normalized' classifier necessary?

Open dydxdt opened this issue 4 years ago • 3 comments

Thanks for your code. I just wonder the reason why we need to normalize the 'classifier' before the 'flatten' op. Does it perform bettern than that without normalizing? Thank you for your explanation.

dydxdt avatar Oct 31 '21 07:10 dydxdt

Now I use tensoflow and try to find how to get the 'self.head.weight' in tensorflow. Or I have misunderstood and make it complex? Thank you for your reply.

dydxdt avatar Oct 31 '21 07:10 dydxdt

Hi, thanks for reading.

  1. We use "Normalization" because it could lead to faster convergence (less training time) in our experiment, but it usually has no impact on the final performance (see Table 8 in our paper).

  2. Actually I'm not familiar with Tensorflow, you can find some instructions in its official website.

Best, Ke Zhu

Kevinz-code avatar Oct 31 '21 07:10 Kevinz-code

Ok, I see. Thank you very much. Why don't you just use the normalization like batch norm or group norm? I'm not sure why you use the normalization method in your code. Can you give some explanation~~~

dydxdt avatar Oct 31 '21 08:10 dydxdt