Solved by agolynski in post #2 
              
              
                Hi, 
Could you try torch.cuda.set_device() instead, torch.cuda.device is a context manager, also see https://github.com/pytorch/pytorch/issues/1608