Hi, finally I found out the problem comes from some function running in  torch.cuda.current_device(), so I work around this by adding the following line to make sure it runs in a context.
with torch.cuda.device(gpu_id):
    #some function must runs in device gpu_id

It works now. Thank you!