valiantljk:

My question is what is the relationship between self.model and its wrapped version self._pytorch_model?


You have one object for each of the three classes

m = ThePyTorchModel   (without DDP)
ddp_m = DistributedDataParallel(ThePyTorchModel)
anlp_m = Model(ThePyTorchModel) (AllenNLP’s model class)

ddp_m and anlp_m wrap (i.e. contain a reference to) the (same) instance m as .module and .model usually.
Now AllenNLP doesn’t want to special case and write .model.module if isinstance(.model, DDP) else .model all the time, so it leaves .model to be the regular model m but stores ddp_m  as the PyTorch ._pytorch_model.
So you should have anlp_m._pytorch_model.module is anlp_m.model return True, they are indeed the very same object.



 valiantljk:

Do they share parameters and runtime state?


Yes in the above sense (that you have an additional hierarchy level when going through DDP).
Best regards
Thomas