Save models after each epoch
# 🚀 Feature request
In the various training scripts in `examples`, would it be better to checkpoint the model at the end of each epoch, as well as every `save_steps` iterations as specified by the user?

## Motivation
I suppose for language modelling, saving the model after each epoch is not as important, but for anything supervised (and some other applications)  it seems  natural to want checkpoints after each epoch. There are plenty of examples in the literature where one wants to inspect models when they have seen each training example some specific number of times. I have been doing some experiments recently where this was the case (with `run_languagemodeling.py` actually) and found myself having to manually enter the number of iterations per epoch as `save_steps to get the desired checkpoints.

## Your contribution
Would be a simple change to the various training scripts. Happy to do a PR.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>I also need this. Did you figure out a better way of doing it other than counting the number of checkpoints required for an epoch manually?<|||||>> I also need this. Did you figure out a better way of doing it other than counting the number of checkpoints required for an epoch manually?

Without changing the code, no I don't think there's an alternative to counting manually. Looks like it would be a simple matter of repeating [this line](https://github.com/huggingface/transformers/blob/0866669e751bef636fa693b704a28c1fea9a17f3/src/transformers/trainer.py#L521) a few lines further down at the end of the epoch loop.<|||||>not sure what you mean by manually counting..but, if you just add this line before the start of each epoch(i.e [here](https://github.com/huggingface/transformers/blob/0866669e751bef636fa693b704a28c1fea9a17f3/src/transformers/trainer.py#L463)), you can make the model save after each epoch. The len(epoch_iterator) is the number of batches in an epoch. 

`self.args.save_steps = len(epoch_iterator)`<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
