Thought I’d share for anyone else having this issue:
I was getting the following error when I was trying to run a GAN architecture with adagrad optimizer on the generator.
THCudaCheck FAIL file=/pytorch/torch/lib/THC/generic/THCStorage.cu line=58 error=2 : out of memory
Traceback (most recent call last):
File “main.py”, line 29, in 
g_optim = torch.optim.Adagrad(generator.parameters(), lr=lr)
File “/imatge/lpanagiotis/ENV/local/lib/python2.7/site-packages/torch/optim/adagrad.py”, line 30, in init
state[‘sum’] = torch.zeros_like(p.data)
RuntimeError: cuda runtime error (2) : out of memory at /pytorch/torch/lib/THC/generic/THCStorage.cu:58
srun: error: c8: task 0: Exited with exit code 1
Switching to SGD instead fixed the error and the code run normally. I am bamboozled as to why would Adagrad give such an error, and I would be interested to hear others’ thoughts on it.