Hello,
I have two models defined as:
model_1 = Net1()
model_2 = Net2()
opt1 = Adam(model1.parameters())
opt2 = Adam(model2.parameters())
When training, the model_1 the output of model_1 is input to model_2 and model_2 acts as a loss function for model_1.
So in this scenario, I wish to update the gradients of model_1 and not model_2 and backpropagate the output of model_2.
def train_model_1():
model_1.train()
model_2.eval()
output1 = model_1(data)
output2 = model_2(output1)
loss = Variable(output2, requires_grad=True)
and then,
loss.backward()
Similarly, for training model_2 I wish to update the gradients of model_2 and not model_1,
def train_model_2():
model_2.train()
model_1.eval()
output1 = model_1(data)
output2 = model_2(output1)
output2_real = get_target_value()
loss = l1_Loss(output2, output2_real)
loss.backward()
Main outer loop looks like this:
pretrain model_1 with some builtin loss function.
pretrain model_2 till its stable
for n epochs:
train_model_1()
train_model_2()