barrier() requires all processes in your process group to join, so this is incorrect:
if local_rank == 0:
        torch.distributed.barrier()

Remember, all collective APIs of torch.distributed(i.e. not include P2P API: send, recv, isend, irecv), requires all processes in your created process group, either the implicit global group or a sub group created by torch.distributed.new_group, to execute.
Will this solve your problem? Please have a test and respond.