DDP averages gradients by dividing by world size. Is there any mechanism (current or planned) to run a user-defined function to scale gradients instead of the default DDP behavior?
In my case, I have variable and uneven batch sizes on each replica and need to compute the average by global batch size.