Solved by ptrblck in post #2 
              
              
                You could just set the BatchNorm layer into eval mode, either by indexing it directly or recursively. 
Here is a small example for the latter approach: 
class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 3, 1, 1)
        sel…