[trainer] sharded _load_best_model
Looks like a copy-in-paste issue. This code path is probably untested.

@sgugger 
_The documentation is not available anymore as the PR was closed or merged._<|||||>Thank you for explaining why the testing of this path is complicated, Sylvain.

I think I can make it partially tested by using zero3 w/o `"stage3_gather_16bit_weights_on_model_save"` which would make it fall through and at least test that condition. I will be adding these tests here https://github.com/huggingface/transformers/pull/17151
