But this assumes the different loss functions are used to compute grads for the same parameters, right?  It doesn’t work in a GAN-like situation (although if the two loss functions are literally the negative of each other there might be some shortcut).