Yeah, I agree with you. The possible reason of training time increase is that the computational graph is not freed correctly. But I think that only loss and output contain gradient tracks, other parts like labels is not traced by the graph, right? I try to delete all the intermediate tensors, but it doesn’t work either! And why the GPU memory will increase to a certain amount rather than keep growing and cause an OOM issue?