Solved by tom in post #2 
              
              
                This is relatively rare and most likely considered a bug. In your case PyTorch master has a bugfix to help BatchNorm load older snapshots, don’t tell anyone, but you might try just copypasting from that. 
Best regards 
Thomas