Brando_Miranda:

are they zero even from a saved checkpoint?


The running stats are restored from the checkpoint. If they were updated previously and stored with these updated values, loading the state_dict would restore these updated values.
If they were never updated and the model was used in model.eval() during training, their initial values would be stored and loaded.