@rwightman, i’m no expert, but as @JIALI_MA suggested, the running averages of batch norm layer are buffers not parameters, hence they can’t be in model.parameters()
import torch
import torch.nn as nn
# With Learnable Parameters
m = nn.BatchNorm2d(5)

print(dict(m.named_parameters()))

output:
{'weight': Parameter containing:
 tensor([1., 1., 1., 1., 1.], requires_grad=True),
 'bias': Parameter containing:
 tensor([0., 0., 0., 0., 0.], requires_grad=True)}

print(dict(m.named_buffers()))

output:
{'running_mean': tensor([0., 0., 0., 0., 0.]),
 'running_var': tensor([1., 1., 1., 1., 1.]),
 'num_batches_tracked': tensor(0)}