save only the best performing checkpoint
# 🚀 Feature request

In the Trainer - Enable an option to save only the best performing checkpoints (rather than the newsest) 

## Motivation

Usually when we train a model we would like to keep only the best performing checkpoints (on the dev set according to the specified metric) rather than the newest checkpoints.


The checkpoints are saved to resume training in case you are interrupted, so saving only the best checkpoints wouldn't work with this.

The `load_best_model_at_end` functionality already keeps track of the best checkpoint during training and reloads it at the end, I think it should cover what you need.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>So，when I  finish training，how can I load the best performing checkpoint ？ @sgugger <|||||> When  I check the `trainer_state.json`  file I found these mssage:
```
  "best_metric": null,
  "best_model_checkpoint": null,
  "epoch": 100.0,
  "global_step": 559300,
  "is_hyper_param_search": false,
  "is_local_process_zero": true,
  "is_world_process_zero": true,
```

as shown above，"best_model_checkpoint"   is null.

<|||||>If you did not specify `--load_best_model_at_end` for your script, you won't get it automatically.