Hi @Yozey
I think you’re right here by running_mean and running_var included in model.state_dict() rather than model.parameters().
My understanding is running_mean and running_var are just stat data extracted from a particular batch of data points, but during the model update phase i.e. using gradients calculated to update the model, those stat data won’t be updated. In this case, model.parameters() only contains those parameters which will be “trained” during the model training process.
Actually, by checking optimizer usage, you can get similar conclusion:
optimizer = optim.SGD(model.parameters(), lr=args.lr, momentum=args.momentum)

Hope this can help you a bit.