Using the pytorch framework.
Suppose you have 4 NN modules of which 2 share weights such that one objective relies on the computation of 3 NN modules (including the 2 that share weights) and the other objective relies on the computation of 2 NN modules of which only 1 belongs to the weight sharing pair, the other module is not used for the first objective.
What would the optimisation step in this scenario entail? With efficiency in mind.