Push to hub training argument not pushing
## Environment info

- `transformers` version: 4.15.0
- Platform: Linux-5.4.144+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.12
- PyTorch version (GPU?): 1.10.0+cu111 (True)
- Tensorflow version (GPU?): 2.7.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help
@sgugger @LysandreJik @lewtun 

## Information

Using `push_to_hub=True` in `TrainingArguments` is not pushing to the Hub. Based on the documentation, if it's `True`, it should push the trained model after training ("Whether or not to upload the trained model to the hub after training").

## To reproduce

Here is a self contained [example notebook](https://colab.research.google.com/drive/1dVUGEAc8JAfuGbGqpOdXBHPsL7rG8bLS#scrollTo=pf8Oz8yx169g).

Most relevant part of code

```python
from transformers import Trainer, TrainingArguments

batch_size = 64
logging_steps = len(emotions_encoded["train"]) // batch_size
model_name = "debug-example"
training_args = TrainingArguments(output_dir=model_name,
                                    num_train_epochs=2,
                                    learning_rate=2e-5,
                                    per_device_train_batch_size=batch_size,
                                    per_device_eval_batch_size=batch_size,
                                    weight_decay=0.01,
                                    evaluation_strategy="epoch",
                                    disable_tqdm=False,
                                    logging_steps=logging_steps,
                                    push_to_hub=True,
                                    log_level="error")

trainer = Trainer(model=model, args=training_args,
                 compute_metrics=compute_metrics,
                 train_dataset=emotions_encoded["train"],
                 eval_dataset=emotions_encoded["validation"],
                 tokenizer=tokenizer)
trainer.train()
```

Output

```
Cloning https://huggingface.co/osanseviero/debug-example into local empty directory.

[500/500 07:28, Epoch 2/2]

Epoch | Training Loss | Validation Loss | Accuracy | F1
-- | -- | -- | -- | --
1 | 0.832700 | 0.306814 | 0.908500 | 0.905449
2 | 0.247700 | 0.219120 | 0.924500 | 0.924758

TrainOutput(global_step=500, training_loss=0.5401657867431641, metrics={'train_runtime': 449.3645, 'train_samples_per_second': 71.212, 'train_steps_per_second': 1.113, 'total_flos': 720342861696000.0, 'train_loss': 0.5401657867431641, 'epoch': 2.0})
```

Doing `cd debug-example && git log` shows there is a local commit that was not pushed, so the local repo is one commit ahead. Doing `trainer.push_to_hub()` afterwards works correctly.

## Expected behavior

The trained model would be pushed to the Hub automatically at the end of training.

That's not the API that was decided. `push_to_hub=True` pushes to the Hub at every save, and you did not pick a `saving_strategy` that triggered a save (it defaults to `"steps"` and there were not enough steps in your training), but it does not push to the Hub at the end of training unless you explicitly asks for it with `trainer.push_to_hub()`<|||||>There was a save. If I do `cd debug-example && git log`, there was a save at step 500. So the model was indeed saved but this did not kicked off a push.
```
commit 9a12f873da1c9c84c2ada3a25a3d1baf917d9e43 (HEAD -> main)
Author: Omar Sanseviero <osanseviero@gmail.com>
Date:   Mon Jan 24 13:36:00 2022 +0000

    Training in progress, step 500

commit 54780a2baf1f8ad714362d890dd1f9e471f0f403 (origin/main, origin/HEAD)
Author: system <system@huggingface.co>
Date:   Mon Jan 24 11:56:59 2022 +0000

    initial commit
```

I don't think [the docstring](https://huggingface.co/docs/transformers/v4.15.0/en/main_classes/trainer#transformers.TrainingArguments) reflects what you're saying about the behaviour. It says it will push after training. Is this something that should be changed?

* push_to_hub (bool, optional, defaults to False) — Whether or not to upload the trained model to the hub after training. If this is activated, and output_dir exists, it needs to be a local clone of the repository to which the Trainer will be pushed.

<|||||>Ok, so there is definitely something wrong going on with the save not being pushed. I don't get why the model was committed but not pushed since it's all done in the same line inside the `Trainer` by calling `Repository.push_to_hub`. The bug is thus probably inside `huggingface_hub`.

For the choice of API, this was mainly because people (for instance all examples) usually do one last evaluation before pushing to the Hub, and if pushing before it, the auto-generate model card does not have that evaluation info (and then the push ends up being rejected because of missing metadata).

We can either:
1. fix the documentation to reflect the reality
2. change the behavior to push at the end of training (with maybe some models not having model cards because the push ends up being rejected if there is no evaluation results)

In all cases, the example scripts will still push a version with the evaluation results afterward.

Think I'm leaning towards 2, but would love to have @LysandreJik and @patrickvonplaten thoughts as well.<|||||>I think in any way we should probably update the docstring because `push_to_hub` pushes the model depending on the `saving_strategy` and I think most people use `"steps"` meaning no? So I think we should state that `push_to_hub` uploads all saved checkpoints during the training to the hub.

In my opinion `push_to_hub` should be very closely tied to saving the model during training in the sense that if `push_to_hub=True` then every time the Trainer saves a file, those updates files should be uploaded, so that the repo online and the local dir are always in sync. <|||||>So, following that logic, the model should not be pushed at the end of training unless the user does `trainer.save_model()` (or `trainer.push_to_hub`) since there is no save at the end of the train method. That works for me too.<|||||>I'm fine with 2), but I wonder if there are use-cases where users don't want to save the model at the end of their training? Fine with me to clarify that `push_to_hub` pushes to hub everytime the model is saved, with an emphasis on the `saving_strategy` and  the possibility to call `trainer.save_model()` at any point to push to the hub.<|||||>I currently have the following code:
training arguments:
```py
train_args = Seq2SeqTrainingArguments(
    overwrite_output_dir=True,
    output_dir="saves/models",
    evaluation_strategy="epoch",
    save_strategy="epoch",
    learning_rate=2e-5,
    per_device_train_batch_size=4,
    per_device_eval_batch_size=4,
    weight_decay=0.01,
    save_total_limit=2,
    num_train_epochs=1,
    predict_with_generate=True,
    logging_steps=len(tokenized_datasets["train"]) // 4,
    hub_strategy="every_save",
    push_to_hub=True,
    hub_model_id=config["MODEL_NAME"],
    hub_private_repo=True,
    hub_token=config["HUGGINGFACE_API_KEY"],
)
```
trainer:
```py
trainer = Seq2SeqTrainer(
    model=model,
    tokenizer=tokenizer,
    args=train_args,
    data_collator=data_collator,
    train_dataset=tokenized_datasets["train"],
    eval_dataset=tokenized_datasets["validation"],
    compute_metrics=compute_metrics
)
```
I then run:
```py
trainer.create_optimizer()
trainer.train()
```
The previous code alone does not save as suggested by the (2.) solution. 
Only when I run the following, it saves to the hub:
```py
trainer.push_to_hub()
```

Any idea why it`s not saving at the end of the epoch? @osanseviero 