You have examples using Accelerate which is our library for distributed training for all tasks in the Transformers repo 1.
As for your hack, you will need to use the distributed version of the SequentialSampler. You might be better off replacing the sampler for the training dataloader by _get_eval_sampler instead of _get_train_sampler.