Solved by Tony-Y in post #43 
              
              
                Please see my experiment using a linear model bellow. 
 [summary fig] 
MultiTaskLoss.ipynb · GitHub 
In this experiment, I used torch.stack instead of torch.Tensor to fix the reported bug of my original code as the following: 
total_loss = torch.stack(loss) * torch.exp(-self.eta) + self.eta

total_l…