Are you using a DataLoader torch.utils.data.dataloader — PyTorch 1.8.1 documentation to help with loading images? Usually that can provide a significant speedup because it allows for prefetching and parallelization across CPU cores.