So, os.environ['CUDA_VISIBLE_DEVICES] and torch.cuda.set_device() are not conflict.
Use CUDA_VISIBLE_DEVICES=0,1 python your_script.py to set all available GPU devices for all processes. In each process, we can also use torch.cuda.set_device() to specify the GPU device for this process.
Is this the correct understanding?