One more case:
def forward(self,x):
  x = self.root(x)
  out1 = self.branch_1(x)
  out2 = self.branch_2(x.detach())
  return out1, out2

loss = F.l2_loss(out1, target1) + F.l2_loss(out2, target2)
loss.backward()

I want the gradients for the branch1 to update the parameters of the root and branch1. But the gradients of the second branch should only update the branch_2 parameters. I guess the detach usage will make it happen, but I want to be sure?