RuntimeError: cublas runtime error : resource allocation failed
## 🐛 Bug

<!-- Important information -->

Model I am using Bert:

Language I am using the model on English:

The tasks I am working on is:
* [ ] Finetuned bert model with my own dataset.
* [ ] run_lm_finetuning.py 

## To Reproduce

Steps to reproduce the behavior:

1. I was followesd this issue https://github.com/huggingface/transfer-learning-conv-ai/issues/10
2. i tried to reduced batch_size = 1
3. i tried  `CUDA_LAUNCH_BLOCKING=1`
it is throwing,
`RuntimeError: CUDA error: out of memory`

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

> CUDA_VISIBLE_DEVICES=2 python run_lm_finetuning.py     --output_dir=output     --model_type=roberta     --model_name_or_path=roberta-base     --do_train     --train_data_file=$TRAIN_FILE     --do_eval     --eval_data_file=$TEST_FILE --mlm --per_gpu_train_batch_size 1 --per_gpu_eval_batch_size 1


## Expected behavior


<!-- A clear and concise description of what you expected to happen. -->
```

Traceback (most recent call last):
  File "run_lm_finetuning.py", line 497, in <module>
    main()
  File "run_lm_finetuning.py", line 451, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer)
  File "run_lm_finetuning.py", line 189, in train
    outputs = model(inputs, masked_lm_labels=labels) if args.mlm else model(inputs, labels=labels)
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/transformers/modeling_roberta.py", line 237, in forward
    head_mask=head_mask)
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/transformers/modeling_roberta.py", line 177, in forward
    head_mask=head_mask)
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/transformers/modeling_bert.py", line 625, in forward
    head_mask=head_mask)
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/transformers/modeling_bert.py", line 346, in forward
    layer_outputs = layer_module(hidden_states, attention_mask, head_mask[i])
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/transformers/modeling_bert.py", line 324, in forward
    attention_outputs = self.attention(hidden_states, attention_mask, head_mask)
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/transformers/modeling_bert.py", line 281, in forward
    self_outputs = self.self(input_tensor, attention_mask, head_mask)
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/transformers/modeling_bert.py", line 200, in forward
    mixed_query_layer = self.query(hidden_states)
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 87, in forward
    return F.linear(input, self.weight, self.bias)
  File "/media/user1/storage-1/Ashok_AI/mask_env/lib/python3.6/site-packages/torch/nn/functional.py", line 1371, in linear
    output = input.matmul(weight.t())
RuntimeError: cublas runtime error : resource allocation failed at /pytorch/aten/src/THC/THCGeneral.cpp:216
Epoch:   0%|                                                                                                                                                                                                            | 0/1 [00:00<?, ?it/s]
Iteration:   0%|
```

## Environment

* OS: Linux
* Python version: 3.6
* PyTorch version: 1.2.0
* PyTorch Transformers version: latest
* Using GPU : yes, CUDA 10
* Distributed of parallel setup : yes


What GPU do you have?<|||||>Thanks for your reply and support sir:)

NVIDIA TITAN RTX: 4 × 24 GB GPUs<|||||>Looks like your batch size may be too big?<|||||>Thank you so much for your support sir.

I given batch size = 1. May be the latest branch any issues will be present. I will check out previous master and then i will try sir. <|||||>Hi, I have the same error. Did you get this problem resolved?
<|||||>I have the same error too<|||||>It may be because of this [nn.embedding issue in pytorch](https://github.com/pytorch/pytorch/issues/24838) . I had the same error. See if you have padded correctly.. or have included some invalid token<|||||>Very similar issue with roberta-base (but not bert-base-cased/uncased):

RuntimeError: cublas runtime error : library not initialized at /opt/conda/conda-bld/pytorch_1573049306803/work/aten/src/THC/THCGeneral.cpp:216

I have checked and it isn't a problem with nn.embedding, nor a memory issue.<|||||>> Very similar issue with roberta-base (but not bert-base-cased/uncased):
> 
> RuntimeError: cublas runtime error : library not initialized at /opt/conda/conda-bld/pytorch_1573049306803/work/aten/src/THC/THCGeneral.cpp:216
> 
> I have checked and it isn't a problem with nn.embedding, nor a memory issue.

Very similar issue, when using camembert model which is based on roberta, 
could you solve the issue ? any thoughts about it plz  <|||||>> 
> 
> Very similar issue with roberta-base (but not bert-base-cased/uncased):
> 
> RuntimeError: cublas runtime error : library not initialized at /opt/conda/conda-bld/pytorch_1573049306803/work/aten/src/THC/THCGeneral.cpp:216
> 
> I have checked and it isn't a problem with nn.embedding, nor a memory issue.

@YDYordanov Same with you when using roberta-base, have you resolved it?<|||||>@YDYordanov @Hadjer13 I found the the solution. In my case , my input example has two sentences, so I use `token_type_ids` like I use in Bert, but it turns out that I pass the wrong `token_type_ids` to the `RobertaModel`. According to [the transformers doc](https://huggingface.co/transformers/model_doc/roberta.html#transformers.RobertaTokenizer.create_token_type_ids_from_sequences), **RoBERTa does not make use of token type ids**. So using `[0,0,..0,1,1..1,0,0,..]` as `token_type_ids` for Roberta is wrong, after I change it to all zeros, i.e. `[0,0,...,0,0]`, the error is fixed. Hope it can help someone!<|||||>> @YDYordanov @Hadjer13 I found the the solution. In my case , my input example has two sentences, so I use `token_type_ids` like I use in Bert, but it turns out that I pass the wrong `token_type_ids` to the `RobertaModel`. According to [the transformers doc](https://huggingface.co/transformers/model_doc/roberta.html#transformers.RobertaTokenizer.create_token_type_ids_from_sequences), **RoBERTa does not make use of token type ids**. So using `[0,0,..0,1,1..1,0,0,..]` as `token_type_ids` for Roberta is wrong, after I change it to all zeros, i.e. `[0,0,...,0,0]`, the error is fixed. Hope it can help someone!

thank you,<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>> @YDYordanov @Hadjer13 I found the the solution. In my case , my input example has two sentences, so I use `token_type_ids` like I use in Bert, but it turns out that I pass the wrong `token_type_ids` to the `RobertaModel`. According to [the transformers doc](https://huggingface.co/transformers/model_doc/roberta.html#transformers.RobertaTokenizer.create_token_type_ids_from_sequences), **RoBERTa does not make use of token type ids**. So using `[0,0,..0,1,1..1,0,0,..]` as `token_type_ids` for Roberta is wrong, after I change it to all zeros, i.e. `[0,0,...,0,0]`, the error is fixed. Hope it can help someone!

I have already had this line in my code: 
`transformer_params = {
            'input_ids': input_ids,
            'token_type_ids': (
                segment_ids if args.model == 'bert-base-uncased' else None
            ),
            'attention_mask': attention_mask,
        }`

Still I am getting the error: `RuntimeError: cublas runtime error : resource allocation failed at /pytorch/aten/src/THC/THCGeneral.cpp:216`

Do you have any idea why? my teacher model is `bert-base-uncased` and when I set my student model as `roberta-base`, I am getting this error. 