[Question] Help needed to understand how torch.distributed.barrier() works
## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
I have been trying to understand how torch.distributed.barrier()  is being used in the examples in this repo. I posted [this stackoverflow question](https://stackoverflow.com/questions/59760328/how-does-torch-distributed-barrier-work). Maybe someone from the huggingface team can help answering it? 
I've provided an answer on Stack Overflow. Please close the issue here on Github. Thanks.<|||||>Thanks @BramVanroy for the detailed answer! 