Nutsy:

Hence, the rephrased question would be: Assuming that I have already loaded the pretrained layer weights into the main model, how do I ensure that the weights for the pretrained layers stay frozen?


Set their requires_grad attribute to False after creating the model.
As given in my previous code snippet: the loading of a state_dict doesn’t change the requires_grad attributes. As long as you set them once to False and do not recreate the model or any layers, they will keep this value.