Hey thanks for the answer. Tried adding that line in the loop, but I still get out of memory after 3 iterations.
RuntimeError: cuda runtime error (2) : out of memory at /b/wheel/pytorch-src/torch/lib/THC/generic/THCStorage.cu:66

(I added the line after optimizer.step())