RuntimeError: The size of tensor a (30524) must match the size of tensor b (30522) at non-singleton dimension 2 --- run_lm_finetuning.py
## 🐛 Bug


I am using ```run_lm_finetuning.py``` to fine-tune bert-base-uncased on my dataset and I am getting the following error:
I also truncated my dataset to have num examples dividable by the batch_size.
Note that fine-tuning gpt2 on the same dataset works fine.

```
12/30/2019 17:23:28 - INFO - __main__ -   ***** Running training *****
12/30/2019 17:23:28 - INFO - __main__ -     Num examples = 4048
12/30/2019 17:23:28 - INFO - __main__ -     Num Epochs = 3
12/30/2019 17:23:28 - INFO - __main__ -     Instantaneous batch size per GPU = 4
12/30/2019 17:23:28 - INFO - __main__ -     Total train batch size (w. parallel, distributed & accumulation) = 4
12/30/2019 17:23:28 - INFO - __main__ -     Gradient Accumulation steps = 1
12/30/2019 17:23:28 - INFO - __main__ -     Total optimization steps = 3036
Epoch:   0%|                                                                                                                                                                          | 0/3 [00:00<?, ?it/sTraceback (most recent call last):                                                                                                                                                  | 0/1012 [00:00<?, ?it/s]
  File "run_lm_finetuning.py", line 722, in <module>
    main()
  File "run_lm_finetuning.py", line 672, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer)
  File "run_lm_finetuning.py", line 306, in train
    outputs = model(inputs, masked_lm_labels=labels) if args.mlm else model(inputs, labels=labels)
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/transformers/modeling_bert.py", line 990, in forward
    prediction_scores = self.cls(sequence_output)
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/transformers/modeling_bert.py", line 496, in forward
    prediction_scores = self.predictions(sequence_output)
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/anaconda3/envs/torch03/lib/python3.6/site-packages/transformers/modeling_bert.py", line 486, in forward
    hidden_states = self.decoder(hidden_states) + self.bias
RuntimeError: The size of tensor a (30524) must match the size of tensor b (30522) at non-singleton dimension 2
```

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

## Environment

* OS: Ubuntu
* Python version: python3.6
* PyTorch version: 1.3.0
* PyTorch Transformers version (or branch):
* Using GPU ? Yes
* Distributed of parallel setup ? No
* Any other relevant information:


Can you share the full command you are using to run the script?<|||||>> Can you share the full command you are using to run the script?

> Can you share the full command you are using to run the script?

Hi sure, this is the command (basically the same as the document):
```
python run_lm_finetuning.py --output_dir=output --model_type=bert --model_name_or_path=bert-base-uncased --do_train --train_data_file=$TRAIN_FILE --do_eval --eval_data_file=$T
EST_FILE  --mlm --output_dir=$OUTPUT_DIR/bert-fine --num_train_epochs 3
```<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
