I‘ve seen several similar problems.’https://discuss.pytorch.org/t/how-to-train-the-network-with-multiple-branches/2152 512. So I don’t think it is out of the box of pytorch, and I would like to double check whether it works.