If this is a model parameter, any reason for using register_buffer instead of register_parameter?

In addition, I need to manually update it. How could I achieve this?

If it is a parameter (not a buffer) and if you don’t expect the autograd engine to compute gradients for you, you can set its .requires_grad field to False before passing the model to the DDP ctor. Then, DDP won’t sync it’s grads and optimizer won’t update the parameter value.

I tried to do the same update as the model is trained on one gpu, but the results are not correct. It seems that the value of this parameter is not synchronized over gpus.

I might miss something. Looks like you want to manually update a parameter, but still want DDP to help synchronize the parameter across the GPUs/processes? I don’t fully understand the use case, could you please elaborate this? If this parameter is manually updated, would it be possible to let all processes to set it to the same value?