I am using distributed data parallel to train the model on multiple gpus. I meet one problem: I have used register_buffer to define one parameter. In addition, I need to manually update it. How could I achieve this? I tried to do the same update as the model is trained on one gpu, but the results are not correct. It seems that the value of this parameter is not synchronized over gpus. Thanks a lot