Solved by ptrblck in post #4 
              
              
                Yes, it seems that the data loading time is large compared to the actual model training time, which would explain the peaks in the data loading time. 

There might be a potential improvement, if replacing: 
torch.tensor(self.datasets[index][1], dtype=torch.long)

which will trigger a copy with: 
to…