you have to give: torch.autograd.backward(loss_seq, grad_seq) where grad_seq is the gradients for each of the losses.
If you want you can give gradients to be all ones:
grad_seq = [loss_seq[0].new(1).fill_(1) for _ in range(len(loss_seq))]