In your main method, you can do this:
    world_size = torch.cuda.device_count()
    backend = 'gloo'
    mp.spawn(init_process, args=(world_size, backend), nprocs=world_size, join=True)

For more details, check out this example 1 in the tutorial.