Incorrect training loss scaling factor in examples/run_classifier.py?
In [examples/run_classifier.py](https://github.com/huggingface/pytorch-pretrained-BERT/commit/87b9ec3843f7f9a81253075f92c9e6537ecefe1c), the overall 'loss' is produce as 'tr_loss/global_step' (instead of 'tr_loss/nb_tr_steps'). Is this behavior correct? @mprouveur made the change in this [commit](https://github.com/huggingface/pytorch-pretrained-BERT/commit/87b9ec3843f7f9a81253075f92c9e6537ecefe1c).

I'm wondering as 'global_step' is never reset after a training epoch, while tr_loss is reset every training epoch. So even if 'tr_loss' remains constant, 'loss' will decrease over more training iterations, given the increasingly large denominator ('global_step').

If this is correct, maybe the 'nb_tr_steps' variable should be removed? It looks unused throughout the code currently.

At any rate, it's only a 2-line fix, and it only affects the logging behavior I believe.
You are right Ethan.
I'm refactoring the examples which were a bit rotten, let's include this fix as well.<|||||>Great! Either way the examples are a great starting point :)

I'm also wondering if tensorboard is [only logging](https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/examples/run_classifier.py#L345) the training loss for the last forward pass for a batch (if several are required / when using gradient accumulation)? A fix would be to maintain a variable ```tr_batch_loss``` (similar to ```tr_loss```) for each full training batch (reset after each parameter update) and log that instead.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
