Thanks a lot!
From the tutorial, I understand that I need to Parallelize my model before moving it to device.
Since if I move it model.to(device) it will by default copy it to just 1 GPU regardless how many GPUs are available in torch.cuda.device_count(). Is this right?
I am using the dataloader as follows. Is the implementation correct?
train_loader = DataLoader(dataset = train_set.dataset, 
                               sampler=SubsetRandomSampler(train_set.indices),
                               batch_size = batch_size, num_workers=n_workers)

valid_loader = DataLoader(dataset = valid_set.dataset, 
                               sampler=SubsetRandomSampler(valid_set.indices),
                               batch_size = batch_size, num_workers=n_workers)

test_loader = DataLoader(dataset = test_set, batch_size = 1,
                               shuffle = False, num_workers=n_workers)

Thanks in advance.
Regards,
Pablo