You need to specify parameters of each of loss function (say l1(w1), l2(w2)) in the corresponding optimizer (say optim1(w1), optim2(w2)) and then find the gradients of both loss functions with respect to each parameter (l1.backward(), and l2.backward()) and then update each parameter (optim1.step() and optim2.step()).