Hey @aashaka, yep, we added _former_parameters after v1.5 to fix the regression caused on https://github.com/pytorch/pytorch/pull/33907.
If this has become very inconvenient for you, I would suggest switch to DistributedDataParallel. There are more discussions here: https://github.com/pytorch/pytorch/issues/36268 1