Solved by carloalbertobarbano in post #6 
              
              
                You would probably need to make your own custom data loader to handle data structures other than tensor, but the general scheme could be something like this? 
model = DDP(model, ...)
sampler = DistributedSampler(dataset)
dataloader = CustomDataLoader(dataset, shuffle=False, sampler=train_sampler, ba…