Hi @mrshenli, you mentioned that DDP can skip the gradient communication for parameters whose requires_grad=False but the flag must be set before wrapping the model with DDP.
I have 2 following questions:


If I set requires_grad=False during training after the DDP ctor, will these parameters be updated anymore, if they still conduct communication?


Is there a way to dynamically freeze more parameters and skip their communication during DDP training after constructing the DDP model?


My use case is gradually freezing more layers during training/fine-tuning. Thanks!