Hi,
Apologies for the delay in reply.
I’ve attempted to add 2 GPUs to the code via torch.nn.DataParallel.
I get a new error:
TypeError: Can’t broadcast (4, 3, 224, 224) -> (4, 2048, 1, 1)
From my understanding, this is trying to broadcast the images (batch, channel, image size, image size) rather than the extracted features.
Here is my updated (and simplified code):
#where I tell the machine to use cuda
device = torch.device("cuda")
if torch.cuda.device_count() > 1:
  print("Let's use", torch.cuda.device_count(), "GPUs!")


# where i define the model (way before the extraction loop)
resnet50 = models.resnet50(pretrained=True)
resnet50 = torch.nn.DataParallel(resnet50)
resnet50.to(device)
for param in resnet50.parameters():
    param.requires_grad = False


# after defining the dataloaders, feature extraction happens here and appended to hdf5 dataset
for i, (inputs40x, inputs20x, paths40x, paths20x, labels) in enumerate(dataloaders_dict):

                inputs40x = inputs40x.to(device)
                inputs20x = inputs20x.to(device)

                labels = labels.to(device)
                paths = paths40x

                # delete the last fc layer.
                modules = list(resnet50.children())[:-1]
                resnet = nn.Sequential(*modules)
                x40 = resnet(inputs40x)
                x20 = resnet(inputs20x)
   
                x_all = torch.cat([x40, x20], dim=1)

                # torch.Size([1, 2048, 1, 1]) batch, feats, 1l, 1l
                array_40[batch_idx, ...] = x40.cpu()
                array_20[batch_idx, ...] = x20.cpu()
                array_all[batch_idx, ...] = x_all.cpu()
                array_labels[batch_idx, ...] = labels[:].cpu()
                array_paths[batch_idx, ...] = paths
                array_batch_idx[:,...] = batch_idx

Where do I make the changes for torch.nn.DataParallel to work? For example, from another task, I was able to add “module” before .fc. It’s unclear from this task, as I am purely extracting features without doing any training.
Cheers,