Solved by albanD in post #8 
              
              
                You will need to subclass batchnorm to make that happen. 
Here is an example for the 2d version: 
Class MyLearntBatchnorm(nn. BatchNorm2d):
  def __init__(self, *args, **kwargs):
    # Initialize the regular batchnorm
    super().__init__(*args, **kwargs)
    # Get the size of the runnning_* buffers…