Yes.
I was debugging and I realize the way to have the same results is by using torch.manual_seed(10) which is likely affecting the weight initializations.
But this doesn’t make sense considering I’m loading weights via network.load_state_dict()? Very bizarre.