The workaround seems to work (gives the same accuracy on the test set). I was wondering if there is a better way.
Would be great if PyTorch provided and option to torch.save/torch.load to save load either the whole model or the state_dict.