I am using pytorch version 1.1.0 and facing the same issue where the training time per step increases within the epoch. I tried removing all the lists/other data objects that could result in storing variables from previous iterations but that did not help. Any suggestions? Also, I am using multiple GPUs to train my model and using nn.DataParallel package