I have a multi-output model in PyTorch when I train them using the same loss and then to backpropagate I combine the loss of both the output but when one output loss decreases others increase and so on. How can I fix the problem?
def forward(self, x): #neural network arch. forward pass x = dense1(x) x1 = dense2(x) x2 = dense2(x) x1 = F.log_softmax(x1) x2 = F.log_softmax(x2) return x1, x2 out1, out2 = model(data) loss1 = NLLL(out1, target1) loss2 = NLLL(out2, target2) loss = loss1 + loss2 loss.backward()
When loss1 decrease loss2 increase and when loss2 decrease loss1 increase how can i fix the issue.
Can any other operator other than ‘+’ be used to combine the loss or should I put weights to the different loss?