Hi,
The torch.cuda.device() context is to change the defaut cuda device so that any cuda-related function will use that device.
We don’t have anything that forces all Tensors to move to cuda. This is mainly because this is a quite expensive operation and the user should be aware of it and avoid moving Tensors back and forth between the CPU and GPU.