I don’t know which model is the pretrained one, but in your current code you are storing the state_dict of model only, which would contain all properly registered parameters, buffers, and other submodules.