We apply the reduce_loss decorator singularly for every function, and from our experiments (but I’m talking about 2 years ago, so something might have changed in the TensorFlow behavior) there are no numerical stability problem.
E.g.
lambda1* l1 + lambda2 + l2 where lambda1,2 are scalars and l1 and l2 are loss functions decorated with the reduce_loss decorator