CUDA error: invalid configuration argument when not using DataParallel
Good Evening,

We have a DGX2 system running the latest Nvidia pytorch docker container - 19.06.  When attempting to use the gpt2 or gpt2-medium models to extract out embeddings we are getting the following error, but only when not using dataparallel: (note we are using apex here but an optimization level of 0, this issue occurs without apex)

```
[...]                                                                                                                                           
  File "[removed]", line 167, in forward
    x1_emb, past = self.embedding(x1)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 494, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_transformers/modeling_gpt2.py", line 515, in forward
    outputs = block(hidden_states, layer_past, head_mask[i])
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 494, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_transformers/modeling_gpt2.py", line 332, in forward
    output_attn = self.attn(self.ln_1(x), layer_past=layer_past, head_mask=head_mask)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 494, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_transformers/modeling_gpt2.py", line 285, in forward
    x = self.c_attn(x)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 494, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_transformers/modeling_utils.py", line 490, in forward
    x = torch.addmm(self.bias, x.view(-1, x.size(-1)), self.weight)
RuntimeError: CUDA error: invalid configuration argument
```

I found reference to this issue from Pytorch:
https://github.com/pytorch/pytorch/issues/2080
but it is reported as fixed so I'm not sure if this issue belongs to this repo or pytorch.
Further testing showed this was caused by the batch size being too high and the card running out of memory, and providing a misleading error.