Solved by kumar.s in post #3 
              
              
                I figured it out. The answer is in the __setattr__ of the class Module (from which DDP inherits). So the Module class __setattr__ removes everything from __dict__ which is of type Parameter or Module and stores them in the self._parameters and self._modules OrderedDict. 
Here’s the corresponding sni…