I want to implement synchronize batch norm across multi-GPU. How can I do it? I think I should synchronize its mean and variance both forward and backward pass, so can I use the register_hook ? Can someone give me some advise? Thank you.