This question stems from comparing the caffe way of batchnormalization layer and the pytorch way of the same. To provide a specific example, let us consider the ResNet50 architecture in caffe (prototxt link 10). We see “BatchNorm” layer followed by “scale” layers. While in the pytorch model of ResNet50 we see only “BatchNorm2d” layers (without any “scale” layer). If, in particular, I compare the first batchnorm layer in pytorch model and the first batchnorm+scale layer in caffe model we get the following.
Pytorch:-
Param Name				size
==========				====
bn1.weight 				torch.Size([64])
bn1.bias 				torch.Size([64])
bn1.running_mean		torch.Size([64])
bn1.running_var			torch.Size([64])
Caffe:
Param Name				size
==========				====
bn_conv1[0]				(64,)
bn_conv1[1]				(64,)
bn_conv1[2]				(1,)
scale_conv1[0]			(64,)
scale_conv1[1]			(64,)
My question is what is the correspondence between these parameters (basically which one in caffe is what in pytorch)?
I also have a second question. And it is regarding the ‘affine’ argument/parameter of the BatchNorm2d module in pytorch. Does setting it False mean \gamma=1, \beta=0?