How does one combine network parameters from two different networks?
Suppose I have two (could be more but let’s do two) distinct networks model1 and model2. I would like the optimizer to be aware of and optimize both models simultaneously.
I want to do something like this:
import torch.optim as optim
optimizer = optim.SGD(model1.parameters() + model2.parameters(), lr=0.001, momentum=0.9)
An example use case is where model1 is pretrained and potentially also model2.
Any ideas or suggestions?