Thank you for such quick reply. The device does mean cuda and cpu.
But
torch.cuda.is_available()

seems like a global configuration detection. There could be situations where different parts of neural networks containing same kinds of modules (say Conv2D) run on different devices which could be controlled in the main function for train and test. In my situation, I would like RNN with custom attention to run on CPU and CNN with same kinds of custom attention to run on GPU in the same network. The tensor’s device could not be decided using some global configuration functions I think.
I am a beginner for pytorch, please correct me if I’m wrong