not able to reproduce accuracy at the end of same epoch
# 🐛 Bug

update: Found what is causing this bug

Your optimization [code](https://github.com/huggingface/transformers/blob/master/src/transformers/optimization.py#L70) uses a linear scheduler for learning rate with warmup which is **dependent on the total number of training steps** (== no of batches*total epochs).  This injects randomness (as the total epoch count changes) during every scheduler.step(), which affects the weights of the trained model, which in turn reflects in accuracy....

The effect of this is very minimal in the in-domain setting. However, since I was evaluating on a cross-domain-setting it ended up being heavily amplified. Can you please make the warm up dependent on the current epoch and not on total epochs/step size.

----original post
When am running say for 25 epochs, I wanted to run an eval on dev and test partition at the end of every epoch. So I moved the evaluate() code in trainery.py to the end of all batches. However the accuracy I get at the end of say epoch 1 (in the big 25 epoch run)  is different than what I get when I run the entire code for just 1 epoch. This might be the same answer to [this](https://github.com/huggingface/transformers/issues/5264) issue also
## Information

Model I am using (Bert, XLNet ...):Bert (base-uncased)

Language I am using the model on (English, Chinese ...):English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x ] my own modified scripts: (give details below): the only modification is I call the same evaluate function to after epochs. You can see that [here](https://github.com/mithunpaul08/transformers/blob/master/src/transformers/trainer.py#L1592).

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ x] my own task or dataset: (give details below)
I am training and testing on [FEVER](https://s3-eu-west-1.amazonaws.com/fever.public/wiki-pages.zip) . My code should download it automatically though.

## To reproduce

Steps to reproduce the behavior:

- Kindly please clone my [repo](https://github.com/mithunpaul08/transformers)
- conda create --name huggingface python=3
- source activate huggingface
- pip install -r ./examples/requirements.txt
from home folder (i.e /huggingface/) do:
- cd mithun_scripts/
- ./run_all.sh
(should run i think)

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
