I want to use torch.multiprocessing to accelerate my loop, however there are some errors .
I can’t absolutely understand the shared cuda menmery for subprocess .
Does anyone give some explanations ?
from torch.multiprocessing import Pool

def use_gpu():
    t = []
    for i in range(5):
        time.sleep(1)
        a = torch.randn(1000, 1000).cuda(3)      
        t.append(a)
    return t

if __name__ == "__main__":
    # torch.cuda.set_device(3)
    pool = Pool()
    result = []
    a = time.time()
    for i in range(10):
        result.append(pool.apply_async(use_gpu))
    pool.close()
    pool.join()
    print("cost time :", time.time() - a)

This snippet can work well .
if I used torch.cuda.set_device(3) for allocation gpu, there are some errors as follow .
THCudaCheck FAIL file=/pytorch/torch/lib/THC/THCGeneral.c line=74 error=3 : initialization error
THCudaCheck FAIL file=/pytorch/torch/lib/THC/THCGeneral.c line=74 error=3 : initialization error
THCudaCheck FAIL file=/pytorch/torch/lib/THC/THCGeneral.c line=74 error=3 : initialization error
THCudaCheck FAIL file=/pytorch/torch/lib/THC/THCGeneral.c line=74 error=3 : initialization error
THCudaCheck FAIL file=/pytorch/torch/lib/THC/THCGeneral.c line=74 error=3 : initialization error
THCudaCheck FAIL file=/pytorch/torch/lib/THC/THCGeneral.c line=74 error=3 : initialization error
THCudaCheck FAIL file=/pytorch/torch/lib/THC/THCGeneral.c line=74 error=3 : initialization error
THCudaCheck FAIL file=/pytorch/torch/lib/THC/THCGeneral.c line=74 error=3 : initialization error
THCudaCheck FAIL file=/pytorch/torch/lib/THC/THCGeneral.c line=74 error=3 : initialization error
THCudaCheck FAIL file=/pytorch/torch/lib/THC/THCGeneral.c line=74 error=3 : initialization error


What cause it and how to solve it ?
Thanks in advance .