Hi everyone,
I have a several layers of CNNs operating on the same size tensor. Can I use a single BatchNorm for all of them, or do I need to set a different BatchNorm per layer?
In other words, in my init() can I do:
self.batchNorm = nn.BatchNorm1d(128, affine=True)
self.conv1 = nn.Conv1d(in_channels = 128, out_channels = 128, kernel_size = 3, padding = 1)
self.conv2 = nn.Conv1d(in_channels = 128, out_channels = 128, kernel_size = 3, padding = 1)
self.conv3 = nn.Conv1d(in_channels = 128, out_channels = 128, kernel_size = 3, padding = 1)

and in my forward pass can I do:
x = self.conv1(x)
x = self.batchNorm(x)        
x = F.relu(x)

x = self.conv2(x)
x = self.batchNorm(x)        
x = F.relu(x)

x = self.conv3(x)
x = self.batchNorm(x)        
x = F.relu(x)

Or do I need to declare batchNorm1, 2, and 3 for each one of these conv layers?  I don’t know how it is implemented, so I don’t know if by reusing the same BatchNorm for different layers it will screw up the statistics it uses for computing the batch norm.
Thanks for your help!