Interesting… Thanks for pointing this out. The issue that I’ve had with setting track_running_stats to True is that I get the “Unexpected key(s)” error when loading the state dictionary at a later time:
Unexpected key(s) in state_dict: “bn1.running_mean”, “bn1.running_var”, “bn1.num_batches_tracked”, “bn2.running_mean”, “bn2.running_var”, “bn2.num_batches_tracked”, “bn3.running_mean”, “bn3.running_var”, “bn3.num_batches_tracked”
I’ve seen reports of this before, and most people suggest to either remove the keys (which I can do, but then I seem to be in my current predicament again) or use DataParallel. I don’t believe I’m using DataParallel when training/saving, though… I merely call torch.save(state, file_name). Any suggestions on how to best save and load the model while turning on track_running_stats? I’ve tried converting my model to DataParallel when loading, but that didn’t seem to resolve the key errors.