I already saw the example but the example is for data distributed parallel which is for distributed training on different machines but in my case,  I have multiple gpu’s on a single machine for which we usually use pytorch nn.DataParallel() as I used in my code. the problem is that the pre-trained MarianModel is not parallelized over 8 gpu’s as it’s shown in nvidia-smi output.