I just thought of one possible solution (I don’t know whether it is feasible) where I would do the two backward() calls first which would accumulate the computed gradients on the overlapping parameters right?
And then I could call on of the optimizer’s step() function, which would apply the changes on every parameter on that branch (inlcuding the overlapping parameters) and then call zero_grad() so that the other optimizer’s step would not update the overlapping parameters again because they are 0 thanks to the previous zero_grad() call.
Is this a possibility or not in you opinion?