Thanks for the update and yes, you are right: based on the description loss3 should be causing the issue not loss2.



 blueeagle:

Even if I calculate the loss3 again after the gradient step of optimizer3 (to get the loss dependent from the new parameters) I am getting this error when calling loss12.backward().


That is unexpected. Could you post an executable code snippet to reproduce this issue?