Semi-supervised learning is a relatively common technique to deal with situations in which labeled data isn’t abundant and might look like this.
PyTorch’s autograd it is much the same if the loss has 1, 2, or 10 terms - losses of minibatches typically are just sums or means of the individual items under the hood, too.
If you have very separate computations, you could also forward with the first, backward, forward for the second and backward again. PyTorch will accumulate the gradients of the two (which is why you need to zero gradients), and it might be much easier on GPU memory / you can afford larger batch sizes.
Best regards
Thomas