the same error means ,I commented the PCI_ORDER line to make cuda order same as nvidia-smi order,and other code unchanged, the torch.rand did create tensor on it specific gpu.But same error occured while I run the trainning code.