I want to accumulate results (in different processes) every n forward passes, aggregate, do something and repeat.
Let say I store the result in self.x. If I broadcast self.x at the end of a forward pass. In the second pass, when I broadcast again, there will be double counting.