Fix non-deterministic Megatron-LM checkpoint name
# What does this PR do?

`os.listdir`'s order is not deterministic, which is a problem when querying the first listed file as in the code (`os.listdir(...)[0]`).

This can return a checkpoint name such as `distrib_optim.pt`, which does not include desired information such as the saved arguments originally given to Megatron-LM.

Instead, we try out different file names used by Megatron-LM (`model_rng.pt` was mentioned in other parts of the script; I'm assuming this is for backward-compatibility).

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@pacman100 wrote most of the code in there and made a Twitter post about this functionality, hope you're the right person to tag. :)
_The documentation is not available anymore as the PR was closed or merged._<|||||>Thanks for the review! Actually that's only the case if `--use-distributed-optimizer` is not given! Otherwise an extra file called `distrib_optim.pt` is created on the most recent Megatron-LM commit. :)<|||||>>Otherwise an extra file called distrib_optim.pt is created on the most recent Megatron-LM commit. :)

Cool, thank you for the info!