In your current code snippet you don’t show how the loss is calculated, so it’s unclear how the loss.backward() pass would work.
In case you are calculating the loss using both outputs, Autograd will use these two computation graphs to calculate the gradients for all parameters. The actual backward operations depend on the used operations during the forward pass.
E.g. if you are adding both outputs together, the backward pass would call the backward of torch.add.