I was looking at the migration guide 16 and the example:
# at beginning of the script
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

...

# then whenever you get a new Tensor or Module
# this won't copy if they are already on the desired device
input = data.to(device)
model = MyModule(...).to(device)

didn’t seem very device agnostic to me because we are hard coding which device to use. For example, what used to happen in my code is that net.cuda() would use the gpu that environment variable CUDA_VISIBLE_DEVICES was assigned to. But hard coding cuda:0 seems problematic to my code (especially because I use slurm, as my workload manager and I assume it uses that flag). Is the only way to make my code truly device agnostic is by trying to getting the value of the environment variable being used and use this pattern suggested?
What if I am using data parallel, how does things change? Do we still need to specify which device to route things to (data and model)? If we do how does it make sense that data parallel is parallelizing anything for us?