Thanks for your hint. I also find this post. I try to compute a loss for each output and append them to a list as the following code,
loss_seq = []
        for o in output:
            cur_loss = criterion(o, target_var)
            loss_seq.append(cur_loss)

Then I print the losses, which seem quite correct.
Then I tried to do the backpropagation
 torch.autograd.backward(loss_seq)

However, an error occured:
 File "example/main.py", line 174, in train
    torch.autograd.backward(loss_seq)
TypeError: backward() takes at least 2 arguments (1 given)

Am I doing wrong? How can I do backpropagation with mutiple loss?