No, since memory fragmentation and in particular different backends might not yield deterministic results and might also differ based on the device, versions, as well as available memory.
E.g. if you are using torch.backends.cudnn.benchmark = True, cudnn will profile different kernels for your current workload and will select the fastest one, which would fit in the available workspace. This would also mean that different algorithms (and thus memory footprints) might be used, if your device is almost full. The best way would be to use the deterministic mode and run a single training iteration in order to estimate the memory usage.


Since the to() operation is differentiable you can easily apply model sharding via:


class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.module0 = nn.Linear(1, 1).to('cuda:0')
        self.module1 = nn.Linear(1, 1).to('cuda:1')
        self.module2 = nn.Linear(1, 1).cu('cpu') # to('cpu') not necessary here
        
    def forward(self, x):
        x = x.to('cuda:0')
        x = self.module0(x)
        x = x.to('cuda:1')
        x = self.module1(x)
        x = x.to('cpu')
        x = self.module2(x)
        return x