I have a model that is about 41 million parameters.
When I try to call model.cuda(), it fails with cuda runtime error (2): out of memory.
  File "/home/anaconda3/lib/python3.6/site-packages/torch/_utils.py", line 69, in _cuda
    return new_type(self.size()).copy_(self, async)
RuntimeError: cuda runtime error (2) : out of memory at /opt/conda/conda-bld/pytorch_1512386481460/work/torch/lib/THC/generic/THCStorage.cu:58

The GPU has 12 GB and it not utilized. If I call another model (even bigger), it works fine.
>>> nn.Linear(100, 400).cuda()
Linear(in_features=100, out_features=400)
>>> nn.Linear(400, 400).cuda()
Linear(in_features=400, out_features=400)
>>> a=torchvision.models.resnet50().cuda()
(long output...)
>>> mymodel.cuda() #smaller than resnet50
--- errors ---

What could be the problem? How can a model make the .cuda() call fail with out of memory error?