Solved by rvarm1 in post #3 
              
              
                Exactly, regardless of the optimizer, before step() is called it is guaranteed that all grad communication has taken place and each replica has allreduced gradients computed in the parameter’s .grad fields.