@FuriouslyCurious How are you applying your augmentation? If you implement it in the Dataset object, e.g. torchvision.datasets.ImageFolder, it gets spread unto separate worker processes in DataLoader, running in parallel with the main process.