Yea, there is an expansion number in the blocks.  If you look at resnet there is an expansion number.

  
      github.com
  
  
    pytorch/vision/blob/master/torchvision/models/resnet.py#L114 6

    self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3,
                           bias=False)
    self.bn1 = nn.BatchNorm2d(64)
    self.relu = nn.ReLU(inplace=True)
    self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
    self.layer1 = self._make_layer(block, 64, layers[0])
    self.layer2 = self._make_layer(block, 128, layers[1], stride=2)
    self.layer3 = self._make_layer(block, 256, layers[2], stride=2)
    self.layer4 = self._make_layer(block, 512, layers[3], stride=2)
    self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
    self.fc = nn.Linear(512 * block.expansion, num_classes)


    for m in self.modules():
        if isinstance(m, nn.Conv2d):
            nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
        elif isinstance(m, nn.BatchNorm2d):
            nn.init.constant_(m.weight, 1)
            nn.init.constant_(m.bias, 0)


def _make_layer(self, block, planes, blocks, stride=1):
    downsample = None



  
  
    
    
  
  


You can use the following code:
resnet = models.resnet50(pretrained=True)
resnet.conv1 = torch.nn.Conv2d(1,64, kernel_size=(7,7),stride=(2,2),padding=(3,3),bias=False)
resnet.fc = torch.nn.Linear(512 * resnet.layer1[0].expansion,1)

That will work if you use any of the resnets…
Also if you want it to work with any size input, you can use torch.conv.AdaptiveAvgPool2d(1) for the average pooling layer.  As long as the initial input’s size is large enough to go through all the convolutions.