Hi David! Thank you for your comment. I also thought of that but the PyTorch documentation got me confused. In particular, in torch.Tensor.cuda — PyTorch 1.9.0 documentation 1 for the device argument, they say:
device (torch.device) – The destination GPU device. Defaults to the current CUDA device.
So I thought that if you don’t specify it the default would be the current CUDA device rather than always  'cuda:0'??