`run_language_modeling` fails with community model (BioClinicalBERT)
# 🐛 Bug

The `run_language_modeling.py` fails when using community models.

## Information

Model I am using (Bert, XLNet ...): BioClinical_BERT

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. download wikitext-2 raw
2. run exactly the same 
3.

This code command line yields the following error:
```
python run_language_modeling.py \
    --output_dir=output \
    --model_type=BioClinicalBERT \
    --model_name_or_path=emilyalsentzer/Bio_ClinicalBERT \
    --output_dir=/kaggle/working/model \
    --do_train \
    --line_by_line \
    --train_data_file=wikitext-2-raw/wiki.train.raw \
    --do_eval \
    --eval_data_file=wikitext-2-raw/wiki.valid.raw \
    --num_train_epochs=4 \
    --mlm
```

the following error will appear:
```
  File "run_language_modeling.py", line 782, in <module>
    main()
  File "run_language_modeling.py", line 732, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer)
  File "run_language_modeling.py", line 333, in train
    outputs = model(inputs, masked_lm_labels=labels) if args.mlm else model(inputs, labels=labels)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/transformers/modeling_bert.py", line 987, in forward
    encoder_attention_mask=encoder_attention_mask,
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/transformers/modeling_bert.py", line 790, in forward
    encoder_attention_mask=encoder_extended_attention_mask,
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/transformers/modeling_bert.py", line 407, in forward
    hidden_states, attention_mask, head_mask[i], encoder_hidden_states, encoder_attention_mask
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/transformers/modeling_bert.py", line 368, in forward
    self_attention_outputs = self.attention(hidden_states, attention_mask, head_mask)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/transformers/modeling_bert.py", line 314, in forward
    hidden_states, attention_mask, head_mask, encoder_hidden_states, encoder_attention_mask
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/transformers/modeling_bert.py", line 234, in forward
    attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))
RuntimeError: cublas runtime error : the GPU program failed to execute at /opt/conda/conda-bld/pytorch_1579022034529/work/aten/src/THC/THCBlas.cu:368
```

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
transformers v2.6.0
     
- `transformers` version:
- Platform: Kaggle
- Python version: 3.7
- PyTorch version (GPU?): 1.4.0
- Tensorflow version (GPU?):
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

Seems like scibert is also broken. When I run

```
python run_language_modeling.py \
    --output_dir=output \
    --model_type=scibert \
    --model_name_or_path=allenai/scibert_scivocab_cased \
    --output_dir=/kaggle/working/model \
    --do_train \
    --line_by_line \
    --train_data_file=wikitext-2-raw/wiki.train.raw \
    --do_eval \
    --eval_data_file=wikitext-2-raw/wiki.valid.raw \
    --num_train_epochs=4 \
    --mlm
```

I get this:
```
Epoch:   0%|                                              | 0/4 [00:00<?, ?it/s]
Iteration:   0%|                                       | 0/5942 [00:00<?, ?it/s]
Iteration:   0%|                               | 1/5942 [00:00<33:48,  2.93it/s]
Iteration:   0%|                               | 2/5942 [00:00<29:32,  3.35it/s]
Iteration:   0%|                               | 3/5942 [00:00<25:44,  3.84it/s]
Iteration:   0%|                               | 4/5942 [00:00<24:19,  4.07it/s]
Iteration:   0%|                               | 5/5942 [00:01<22:20,  4.43it/s]Traceback (most recent call last):
  File "run_language_modeling.py", line 782, in <module>
    main()
  File "run_language_modeling.py", line 732, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer)
  File "run_language_modeling.py", line 345, in train
    loss.backward()
  File "/opt/conda/lib/python3.6/site-packages/torch/tensor.py", line 195, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/opt/conda/lib/python3.6/site-packages/torch/autograd/__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: CUDA error: device-side assert triggered
```<|||||>This is because the tokenizers you mention do not have a `tokenizer_config.json` file on the S3. There should be one limiting the maximum length to 512 tokens.

Here it is set to 1e12 because it doesn't detect a maximum length in the configuration.

cc @julien-c <|||||>Yes, in that case you would need to pass a `max_block` arg to the script.

Let us know if it fixes your issue.<|||||>I see, thanks a lot! Here it would be `max_block=512` right (or whatever is the maximum length supported by that model)?<|||||>Yes, that's right!<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>I haven't had the chance to test `max_block=512`, but I hope this solves the problem. If it still persists, I'll re-open this issue.