I have a cluster of 10 GPUs. The default code,

    if torch.cuda.device_count() > 1:
        net = nn.DataParallel(net)


Uses 8 GPUs. Does PyTorch have a max 8 GPU policy? Training number workers = 1 at the moment but this behavior doesn’t change when I tried 1, 2 or 10.

image.png875×767 96.1 KB