Are you seeing the out of memory error on the GPU or are you running out of system RAM?
In the former case could you check, if the memory usage is increasing in each iteration or epoch via nvidia-smi?
If so, make sure to not save tensors attached to the computation graph, e.g. by appending the loss to a list without calling loss.detach().