Thanks a lot.
According your advice, I write the following code in PyTorch.
# calculating the gradients of two losses
grads_loss_1 = autograd.grad(loss_1, model.parameters(), retain_graph=True)
grads_loss_2 = autograd.grad(loss_2, model.parameters(), retain_graph=False)
# update generator parameters gradients
for idx, p in enumerate(gen.parameters()):
    grad_1 = grads_loss_1[idx]
    grad_2 = grads_loss_2[idx]
    if torch.norm(grad_1, p=2) > torch.norm(grad_2, p=2):
        grad_1 = grad_1 * torch.norm(grad_2, p=2) / torch.norm(grad_1, p=2)
    else:
        grad_2 = grad_2 * torch.norm(grad_1, p=2) / torch.norm(grad_2, p=2)
    p.grad = (kappa * grad_1 + kappa * grad_2).clone()

It works well in my work. Please point it out, if you find any errors.
Thanks again for your suggestions!