Albert language model fine tuning not running run_lm_finetuning.py
# ❓ Questions & Help

## Information

Model I am using (Albert(all types)):

Language I am using the model on (English):

The problem arises when using:
* [ ] the official example scripts: (give details below)
the code returns memory allocation problems when run with any version from albert. i tried to reduce the sequence length and batch size to a minum setting, but the issue still arises. my setting and the minimized setting both run normally with bert or roberta, the issue arises only when i change the model to Albert.
an example:
`tcmalloc: large alloc 1951195136 bytes == 0x7f750f664000 @  0x7f76efbf8887 0x7f764c2a1b79 0x7f764c29fb0f 0x7f764c29fc33 0x7f764c26a155 0x7f764c26837e 0x7f764c26bbb1 0x7f764c2606df 0x50a8af 0x50c5b9 0x509d48 0x50aa7d 0x50c5b9 0x508245 0x509642 0x595311 0x5a067e 0x50d966 0x58efc9 0x4c9546 0x5886f4 0x58892e 0x551b81 0x5aa6ec 0x50abb3 0x50c5b9 0x508245 0x50a080 0x50aa7d 0x50c5b9 0x508245`

The tasks I am working on is:
* [ ] my own task or dataset: (give details below)
language model finetuning for albert
## To reproduce

Steps to reproduce the behavior:

1. in run_lm_finetuning add:
` from transformers import (AlbertConfig,
    AlbertForMaskedLM,
    AlbertTokenizer,
)`
2.add to MODEL_CLASSES dictionary:
`    "albert": (AlbertConfig, AlbertForMaskedLM, AlbertTokenizer),`
3. add file text.txt, a similar txt file to the wiki dataset that's mentioned in the docs. 
4.run the finetuning script:
`python transformers/examples/run_lm_finetuning.py \
    --output_dir=output \
    --model_type=albert \
    --model_name_or_path=albert-base-v1 \
    --do_train \
    --train_data_file test.txt \
    --block_size 50 \
    --per_gpu_train_batch_size 2 \
    --max_steps 520000 \
    --weight_decay 0.01 \
    --logging_steps 5000 \
    --mlm`



## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment

* OS: Google colab
* Python version: 3.7
* PyTorch version: 1.3.1
* `transformers` version (or branch): latest
* Using GPU ? yes
* Distributed or parallel setup ? no
* Any other relevant information:

@thomwolf can you give any insights regarding this?<|||||>how much lines in `test.txt`?<|||||>1,041,130 line<|||||>I have a similar issue finetuning the language model with bert. In the end, I had to scale down my training to ~200,000 lines to make it work, which is a very small proportion of my original dataset.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
