How can I add a new batchnorm layer?

[EyreEyre]

I want to add extra conv + bn+relu layers after dcr, but I donot konw how to init bn layer both in train and use in test, can you give me some examples to show how to do it? Thx.

[bowenc0221]

Weight initialization in training works like this:

1. load pre-trained weight and initialize new layers: https://github.com/bowenc0221/Decoupled-Classification-Refinement/blob/9c37d014212f2e9f335a7262e3b1c24cd6774e49/faster_rcnn_dcr/train_end2end.py#L103-L104
2. where the initialization of extra layers is defined in the symbol: https://github.com/bowenc0221/Decoupled-Classification-Refinement/blob/9c37d014212f2e9f335a7262e3b1c24cd6774e49/faster_rcnn_dcr/symbols/resnet_v1_101_rcnn_dcr_res2.py#L1882-L1917

For BN layers, it has 4 parameters to be initialized:

def init_bn_layer(self, cfg, arg_params, aux_params):
    arg_params['bn_weight'] = mx.random.normal(0, 0.01, shape=self.arg_shape_dict['bn_weight'])
    arg_params['bn_bias'] = mx.nd.zeros(shape=self.arg_shape_dict['bn_bias'])
    aux_params['bn_gamma'] = mx.nd.ones(shape=self.aux_shape_dict['bn_gamma'])
    aux_params['bn_beta'] = mx.nd.zeros(shape=self.aux_shape_dict['bn_beta'])

Note: weight and bias are in arg_params, gamma and beta are in aux_params.

[murari023]

@bowenc0221 We did initialize as mentioned above. But we are getting error : KeyError 'bn_weight'

[murari023]

@bowenc0221 We are preparing a new network from scratch and trying to train with faster r-cnn framework. We have initialized all the new layers along with the BatchNorm layers. We are getting this error even after initializing as mention above. Can you help? Thanks

[bowenc0221]

@murari023 replace 'bn' in 'bn_weight' with the name of your BatchNorm layers