Cuda rng_state_all is used when saving in distributed mode so same should also be used when loading
When saving in distributed mode this snippet uses `torch.cuda.random.get_rng_state_all()`.

https://github.com/huggingface/transformers/blob/a0e733283930bdb9ae2b1afdc53ec5f2daefb033/src/transformers/trainer.py#L2417-L2421

But while loading `torch.cuda.random.set_rng_state_all()` is not being used for distributed causing issues when resuming training.

https://github.com/huggingface/transformers/blob/a0e733283930bdb9ae2b1afdc53ec5f2daefb033/src/transformers/trainer.py#L2323-L2328


_The documentation is not available anymore as the PR was closed or merged._