So what do you want to do for the parameters that appears in both?
If you want to update them twice, you don’t really need to do anything except keeping the zero_grad-backward-step bits of each model disentangled. For the first, you probabably want retain_graph or so.
An alternative could be to update each part alternatingly on every second input batch.
That said, I’d probably try with your first approach. From my experience, what is important in such a case is some equilibration, i.e. you likely want the losses to be of the same order or in some relation to each other.
Best regards
Thomas