I’m not sure, how exactly your code works, but do you think you could play with the requires_grad attribute of your models?
E.g. in case you want to use loss1 to calculate the gradients for model1, but not model2, you could just disable gradients for model2:
for param in model2.parameters():
    param.requires_grad_(False)