https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html 2
At beginning

device = torch.device(“cuda:0” if torch.cuda.is_available() else “cpu”)

So all the models and data are still just going to the first gpu. Is this a typo, or something deeper going on?