Solved by ptrblck in post #2 
              
              
                They are saved and loaded as seen here: 
bn = nn.BatchNorm2d(3)
print(bn.running_mean)
> tensor([0., 0., 0.])
print(bn.running_var)
> tensor([1., 1., 1.])

out = bn(torch.randn(10, 3, 24, 24))
print(bn.running_mean)
> tensor([0.0009, 0.0018, 0.0004])
print(bn.running_var)
> tensor([1.0009, 1.0015, 1…