Oh, then above answer is likely correct:


If so, then I think the difference between both approaches could come from a different order of the reductions and thus accumulated small absolute erro


You’re either not looking beyond first 4 digits of loss, or differences only manifest in gradients.
Not sure how you’d “fix” that, and if you need to do adjustments to “mean” mode, you’d lose result identity anyway…