Hey @OlivierCR,
both the SageMaker Distributed Data-Parallel and the Model-Parallel library are directly integrated into the Trainer API, which uses and initializes both libraries automatically.
For SMD:

The library is first imported with an alias for the default PyTorch DDP library here 5

and then wrapps the model here 3


P.S. The _wrap_model() 1 function also handles SMP