Solved by tom in post #2 
              
              
                You have one object for each of the three classes 

m = ThePyTorchModel   (without DDP)
ddp_m = DistributedDataParallel(ThePyTorchModel)
anlp_m = Model(ThePyTorchModel) (AllenNLP’s model class)

ddp_m and anlp_m wrap (i.e. contain a reference to) the (same) instance m as .module and .model usually. …