powernorm

powernorm copied to clipboard

Feature request: improved documentation

3

Hi! Thank you for releasing the paper code! I had some issues understanding the implementation that are solved by now. However, I expect that many of the people who decide...

Guitaricet

A few questions regarding fairseq/modules/norms/mask_powernorm.py

18

Hi, first of all thank you for your work. I've been spending some time trying to understand what is happening in this script fairseq/modules/norms/mask_powernorm.py but I've been having some trouble....

congwang093

The paper mentions that batch normalization can have large fluctuations in the batch statistics. This occurs in vanilla BN because it calculates the statistics over input of varying lengths padded...

enhuiz