Suppose we initialized batchnorm:
bn1=nn.BatchNorm2d(64,track_running_stats=False)
Can we use this bn1 after every convolution layer that outputs 64 channels?
or do we need to do:
bn1=nn.BatchNorm2d(64,track_running_stats=False)
bn2=nn.BatchNorm2d(64,track_running_stats=False)
for using after 2 conv layers with 64 layers output each?
Do batch norm track weights i.e. running mean and variance from previous states?