Solved by ptrblck in post #2 
              
              
                No, DDP shares the state_dict in the first iteration, so you won’t need to seed the code for the parameter initialization, but might need it for other use cases. 
From the DDP Internal Design: 

The DDP constructor takes a reference to the local module, and broadcasts state_dict() from the process w…