Brando_Miranda:

My understanding is that they should never have that value.


They should have these values after they are initialized. I would guess that your training might set the batchnorm layers or the entire model into .eval() mode so that the running stats are never updated and keep their initial values.
Check your code for .eval() calls (additionally also for self.training = False assignments) and see if that might be the issue.