Albert pretraining loss not decreasing

I am training Albert from scratch using run_language_modeling.py.
Doing training on 8 V100.p3dn8x.
Launching with this parameters.

```
python transformers/examples/language-modeling/test.py   --train_data_file x.txt  --output_dir albert_model --model_type albert --mlm --config_name test --tokenizer_name test --do_train --line_by_line --learning_rate 0.00088 --num_train_epochs 3 --save_total_limit 50 --save_steps 5000 --per_gpu_train_batch_size 150 --seed 42 --overwrite_output_dir --max_steps 200000 --fp16
```
![image](https://user-images.githubusercontent.com/18630864/83558269-65041100-a530-11ea-965b-f24a76550a28.png)

loss is not decreasing. The above plot contains training with and without warmup steps training curve. loss is stuck on `7.27` in both case.
Also, weird thing is while I am setting the number of epochs 3 but in training its showing 9
```
was: ModuleNotFoundError("No module named 'amp_C'",)
06/02/2020 14:45:14 - INFO - transformers.trainer -   ***** Running training *****
06/02/2020 14:45:14 - INFO - transformers.trainer -     Num examples = 28236463
06/02/2020 14:45:14 - INFO - transformers.trainer -     Num Epochs = 9
06/02/2020 14:45:14 - INFO - transformers.trainer -     Instantaneous batch size per device = 150
06/02/2020 14:45:14 - INFO - transformers.trainer -     Total train batch size (w. parallel, distributed & accumulation) = 1200
06/02/2020 14:45:14 - INFO - transformers.trainer -     Gradient Accumulation steps = 1
06/02/2020 14:45:14 - INFO - transformers.trainer -     Total optimization steps = 200000
``` 
Can anyone suggest what could be reasons for such behavior?

Not certain, but looks like maybe nvidia apex was not installed correctly?

"was: ModuleNotFoundError("No module named 'amp_C'",)"<|||||>thats warning. 
Have followed `pip install -v --no-cache-dir ./` for apex installation. 
Changing LR to 5e-5 reducing the loss.<|||||>@008karan, setting the loss to 5e-5 led your model to convergence?<|||||>@LysandreJik loss is decreasing as of now 

> Also, weird thing is while I am setting the number of epochs 3 but in training its showing 9

can you comment on this?<|||||>I'll have a look.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
