Log metrics during training
# 🚀 Feature request

HF Transformers supports custom metrics at evaluation, but not during training. Many metrics are trivial / fast to collect during training time, so it would be nice to collect them too (e.g. BERT accuracy).  

## Motivation

Collecting these metrics during training helps answer a number of research questions including overfitting, model performance, etc. Integration at this level can also pave the way toward collecting sequence-level information that can be logged using tools like W&B Tables.

## Your contribution

I'm happy to submit a PR. Starting this conversation early to see if there is interest in this feature / understand if there's a reason why this isn't already supported in HF.

I am not sure what you are saying. Metrics are computed during training at each evaluation step/epoch, depending on the strategy you picked.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.