You are probably holding reference to some Variable from the previous iteration. This can cause the graph not being freed properly in some cases. See http://pytorch.org/docs/master/notes/faq.html#my-model-reports-cuda-runtime-error-2-out-of-memory 978