how to mix sn and bn

[JamesKasperYu]

hello, Thank you for the great jobs and sharing it.I want to use encoder net with bn and decoder net with sn,but got the error "RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation".How can i deal with it.

[JiaminRen]

You can show more logs and your codes.

[JamesKasperYu]

the code is too long and i'll show part of my code which get the error is:

def __init__():
    self.encoder = vars(resnet)\['resnet50']()
    self.conv_a = nn.Sequential(
                        nn.Conv2d(512*4,256,1,1),
                        sn.SwitchNorm(256, using_moving_average=using_moving_average),
                        nn.ReLU(inplace=True))
def forward(x):
    x=self.encoder(x)
    x=self.comv_a(x) 

and got the error:

Traceback (most recent call last):
  File "train.py", line 352, in <module>
    main()
  File "train.py", line 324, in main
    optimizer, scheduler, epoch, train_writer)
  File "train.py", line 151, in train
    loss.backward()
  File "/home/yuxb/anaconda3/lib/python3.6/site-packages/torch/tensor.py", line 93, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/yuxb/anaconda3/lib/python3.6/site-packages/torch/autograd/__init__.py", line 89, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation.

Here encoder is resnet50 with bn.I take sn with nn.BatchNorm2d(256),it works well.