Bug: Finetuning large models resume checkpoint error
When finetuning a large model (e.g. Eleuther 6B), you shard the checkpoints upon saving [here](https://github.com/huggingface/transformers/blob/c79bbc3ba54a81dab2eac13d89f264ed64cb2460/src/transformers/modeling_utils.py#L193). However, upon resuming the checkpoint (and loading the best checkpoint after training), you confirm if there is a valid checkpoint assuming weights are no sharded [here](https://github.com/huggingface/transformers/blob/dced262409177586bb510b6b724c762fb89da0e8/src/transformers/trainer.py#L1196). This causes an error upon resuming training.


Indeed, I saw that yesterday and am working on a fix.<|||||>Should be fixed by the PR mentioned above :-)<|||||>Thanks!!