ptrblck:

Are you seeing the out of memory error on the GPU or are you running out of system RAM?
In the former case could you check, if the memory usage is increasing in each iteration or epoch via nvidia-smi ?
If so, make sure to not save tensors attached to the computation graph, e.g. by appending the loss to a list without calling loss.detach() .


thanks for your reply, actually it’s the error on the system RAM. How to solve this question? I pay attention to check the memory usage through ‘nvidia-smi’, but the GPU memory usage is very low, I have already put the device on Cuda.