PyTorch-progressive_growing_of_gans icon indicating copy to clipboard operation
PyTorch-progressive_growing_of_gans copied to clipboard

Possible bug in `he_init` function

Open ngoyal2707 opened this issue 7 years ago • 0 comments

It seems that in the current version of master in the he_init function, gain is being passed for argument a. But as per Pytorch code in both version 1.0 and 0.4.1, kaiming_normal_() expects a to be negative slope and has nonlinearity argument separately. So just following function call should do:

kaiming_normal(layer.weight, nonlinearity=nonlinearity, a=param) 

Pytorch source code link: https://github.com/pytorch/pytorch/blob/v0.4.1/torch/nn/init.py#L296

I have tested it on some layers, just above call gives the expected value to std() of layer.weight.

Your code might have been based on some different version of pytorch that did expect to pass gain separately but thought should you a heads up just in case.

ngoyal2707 avatar Dec 31 '18 00:12 ngoyal2707