Hi there.
I am trying to implement the multi-task training to my own model (implementation: https://github.com/Hui-Li/multi-task-learning-example-PyTorch/blob/master/multi-task-learning-example-PyTorch.ipynb 2), and find some issues with combining the loss together.
I have three models to train, Model_1 and Model_2 are trained with nn.CrossEntropyLoss() and Model_3 is trained with nn.BCELoss().
how the training goes are like this:
optimizer.zero_grad()
loss = loss_1 + loss_2 + loss_3
loss = torch.mean(loss)
loss.backward()
optimizer.step()

But actually I found that Model_2 can not be trained properly, as loss_2 hardly decreases. However, when I try to train Model_2 separately with identical model and data, the loss can normally decrease and the model is learning quite well.
What I want to know is how loss.backward() specifically works in such situation? I also tried to manually give loss_2 a big weight value, or make loss_1 = loss_3 = 0, but these methods do not help.
Can someone please explain a little bit about that? Thanks a lot.  : )