I see. So you want to calculate loss for whole training set and propagate same loss for each GPU.
Question is, do you really need this? With reduction=sum you can simply calculate partial losses per GPU, then sum across all of them. You can also do sum across batches within each GPU.