Hi,
in the DDP tutorial (https://pytorch.org/tutorials/intermediate/ddp_tutorial.html 5) the following code is shown to split a model onto two GPUs:
class ToyMpModel(nn.Module):
    def __init__(self, dev0, dev1):
        super(ToyMpModel, self).__init__()
        self.dev0 = dev0
        self.dev1 = dev1
        self.net1 = torch.nn.Linear(10, 10).to(dev0)
        self.relu = torch.nn.ReLU()
        self.net2 = torch.nn.Linear(10, 5).to(dev1)

    def forward(self, x):
        x = x.to(self.dev0)
        x = self.relu(self.net1(x))
        x = x.to(self.dev1)
        return self.net2(x)

How can I split a pretrained model (DeeplabV3Resnet101) onto different GPUs?

def getDeepLabV3Resnet101Pretrained(num_of_classes):
    model = models.segmentation.deeplabv3_resnet101(pretrained=1)
  
    # Change number of output classes
    model.classifier[4]  = nn.Conv2d(
    in_channels=256,
    out_channels=num_of_classes,
    kernel_size=1,
    stride=1
    )

   # And now how to put different model parts on different GPUs?
   # Does model.children() help? 

How would you determine where to split?
I would try to calculate the number of parameters for every model layer and then make more or less equal splits.
Would this be a good way?
Thanks!