Are you using DistributedDataParallel or DataParallel for this? It seems like your question is more related to torchvision or the PyTorch dataset/dataloader (the ‘distributed’ tag is not appropriate for this). Maybe tag this with ‘vision’?