Thanks a lot. This is exactly what I am looking for.
In addition, do you know a good way to transfer the optimizer state to the new model with different names? The reason I am asking here is that the pre-trained model I have is constructed via nn.Sequential, but I want to insert some new blocks in between and retrain the new network. It will be easier for me if I can start with what they have.