I have two networks: net1 and net2. They are different learning rate schedule but joint training.
net1_optimizer = torch.optim.Adam(net1.parameters()), lr=args.lr, betas=(0.5, 0.999))
net2_optimizer = torch.optim.Adam(net2.parameters(), lr = args.lr)
net1_lr_scheduler = torch.optim.lr_scheduler.StepLR(net1_optimizer, step_size=100, gamma=0.1)
net2_lr_scheduler = torch.optim.lr_scheduler.StepLR(net2_optimizer, step_size=100, gamma=0.1)

The net1 produces loss1 and net2 produces loss2. The total loss is
loss_total = loss1+loss2
loss_total.backward()

For step, should I call them separately for loss1 and loss2?
loss1_optimizer.step()
loss2_optimizer.step()