You can do that or create an optimizer that works with both models:
full_opt = torch.optim.SGD(list(Model1.parameters())+list(Model2.parameters()), other_args) and then you can just call full_opt.step() to update both models.