Cubla Error on DistilBert
# 🐛 Bug
When using DistilBert I get `CUBLAS_STATUS_ALLOC_FAILED` when trying to forward

## Information
Exact error is : `RuntimeError : Cuda Error: CUBLAS_STATUS_ALLOC_FAILED when calling cublasCreate(handle)` and traceback indicates i happens on `output = input.matmul(weight.t())` which is probably not informative but the whole stack is filled with forward on a transformer.

I'm using distilbert-base-multilingual-cased with French

## Environment info
- `transformers` version: 2.5.1
- Platform: Linux-5.4.0-3amd-64-x89_64-with-debian-bullseye-sid
- Python version: 3.7.0
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): 2.1.0 (False)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

After some investigation, I have really no clue on what's happening. This issue is only referred through Tensorflow questions such as [this](https://stackoverflow.com/questions/41117740/tensorflow-crashes-with-cublas-status-alloc-failed) or [this issue](https://github.com/tensorflow/tensorflow/issues/9489) on Tensorflow's Github<|||||>I've got the same issue using the same environment as you. 
I'm using model `"mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es"`
 My error though is at:
```
RuntimeError                              Traceback (most recent call last)
<ipython-input-17-90ab5d6d4393> in <module>
     13         outputs = model(**inputs, labels=labels)
     14         loss, logits = outputs[:2]
---> 15         loss.backward()
     16         
     17 
/opt/conda/lib/python3.7/site-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph)
    193                 products. Defaults to ``False``.
    194         """
--> 195         torch.autograd.backward(self, gradient, retain_graph, create_graph)
    196 
    197     def register_hook(self, hook):

/opt/conda/lib/python3.7/site-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
     97     Variable._execution_engine.run_backward(
     98         tensors, grad_tensors, retain_graph, create_graph,
---> 99         allow_unreachable=True)  # allow_unreachable flag
    100 
    101 
RuntimeError: CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling `cublasCreate(handle)`
```
I've tried most stuff I've found but really nothing seems to work.<|||||>Find similar issue when using regular BERTModel<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Hi, @Ricocotam and @j6e 
I got the same error with torch 1.6.0 GPU with a DistilBert Classification error. Would you advise if and how you fix that?<|||||>So, I encountered the same issue

This might sound stupid, but it happened because I was using the wrong tokenizer. So, for future readers.. check that you really use an appropriate tokenizer ?

However, I agree the error message is rather obscure. I'm not sure exactly *why* it is triggered (I guess the other tokenizer would produce some IDs that do not exist in the distilbert vocabulary ?) , so I don't know if there is an easy fix for that 