Roughly I understand that retain_graph will restore the memory which should have been released after calling backward().
I just wonder, if I have a multi-task computational graph as below, what’s the difference between (loss1+loss2).backward()
and loss1.backward(retain_graph=True); loss2.backward()?
Will they perform the same computation?