TPU needs a rendezvous
When running on TPU, each ordinal needs to call `save_pretrained`. See [original implementation by @jysohn23](https://github.com/huggingface/transformers/pull/3702/files#diff-28e1baa470caa8e6f23b78a356b6bbdfR162).

If we don't do it this way, the trainer hangs when saving the model.