4 NN modules of which 2 share weights

In this case, you only have 3 NN modules, and one of them is simply reused.
If you have multiple objectives that you want to backprop, you can use:
autograd.backward http://pytorch.org/docs/autograd.html#torch.autograd.backward 137
You give it the list of losses and grads.
The optimization step is pretty standard, you give the all the modules’ parameters to a single optimizer.