Hi everyone,
I have a Model consisting of 3 parts. Part 1 is an encoder, part 2 and 3 are classifiers, that both get the output of the encoder (part 1) as input. I have two optimizers, the first has the parameters of part 1 and 2 (named optimizer12), the second only has the parameters of part 3 (optimizer3). I am calculating a Loss for the output of part 2 (named loss2) and one for the output of part 3 (named loss3). Now I want to update part 3 and part 1&2 alternatingly. I used the code below in my training method:
loss2 = criterion2(output2, target2)
loss3 = criterion3(output3, target3)
loss3.backward(retain_graph=True)
optimizer3.step() 
loss12 = loss2 + some_value*loss3
loss12.backward()
optimizer12.step()

However, this gives me the following error:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [2, 2000]] is at version 2; expected version 1 instead.
I think the Variable mentioned in the Error Message is the output of part 3 (at least it hast the same dimensions). Regarding this answer 1, I assume that the error is caused by the multiplication of loss3 with some_value. However, I cannot get rid of this multiplication so I need to find a way around this.
Can you please help me?
Thank you!