Tried your recommendation that made the process super fast but it prevented me from backpropagating because the resulting loss is not a variable type but a float type (no history).
I’m struggling to understand why it’s running out of memory with 12gb. @apaszke I’m thinking there’s a bug in PyTorch. When I run htop, it’s only taking up 2gb+. Somehow there’s something triggering the errors.
I was running the other CPU version with a larger dataset and this came out:
 File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/torch/optim/adam.py", line 52, in step
    state['exp_avg_sq'] = grad.new().resize_as_(grad).zero_()
RuntimeError: $ Torch: not enough memory: you tried to allocate 7GB. Buy new RAM! at /data/users/soumith/miniconda2/conda-bld/pytorch-cuda80-0.1.8_1486039719409/work/torch/lib/TH/THGeneral.c:270

This is weird considering how I’ve more than 60GB RAM. May I know if PyTorch is limiting the amount of RAM somehow?