Hi smth, I’m having amount exactly the same issue. I’m using PyTorch 0.4.0, torchvision 0.2.1 and CUDA 9.0. Running bottleneck didn’t give me a lot of information, except that convolutions took the most time. I managed to the profiler instead, and found that almost every operation took significantly longer in those slow iterations, and the forward pass slowed down more than the backward pass.