Suraj:

This looks fundamentally wrong to me now. Are the gradient of loss function by chain rule backpropagated to the pre-trained model parameters? I don’t think they are connected.


Why do you think the computation graph is not connected? Did you check the gradients in Model1 or do you see any other issues?
Your code looks alright and you can chain different models as you are also chaining different layers.
Since Model1 and Model2 are implemented as nn.Modules there would be no difference to “layers” such as nn.Conv2d.