From some points, it works but the loss is high compared to Keras with same structure, what can it be? The output block problem?
(
  ...
  (conv2d_7b): BasicConv2d(
    (conv): Conv2d(2080, 1536, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn): BatchNorm2d(1536, eps=0.001, momentum=0.1, affine=True)
    (relu): ReLU()
  )
  (avgpool_1a): AvgPool2d(kernel_size=8, stride=8, padding=0, ceil_mode=False, count_include_pad=False)
  (last_linear): RetBlock(
    (liner_nose): Linear(in_features=1536, out_features=4, bias=True)
    (liner_tail): Linear(in_features=1536, out_features=5, bias=True)
    (liner_hand): Linear(in_features=1536, out_features=4, bias=True)
    (liner_leg): Linear(in_features=1536, out_features=4 bias=True)
  )
)

My code for output block:
class RetBlock(nn.Module):

    def __init__(self):
        super(RetBlock, self).__init__()
        in_features = 1536
        self.liner_nose = nn.Linear(in_features, 4)
        self.liner_tail = nn.Linear(in_features, 5)
        self.liner_hand = nn.Linear(in_features, 4)
        self.liner_leg = nn.Linear(in_features, 4)

    def forward(self, x):
        x_nose = self.liner_nose(x)
        x_tail = self.liner_tail(x)
        x_hand = self.liner_hand(x)
        x_leg = self.liner_leg(x)
        
        return x_nose, x_tail, x_hand, x_hand, x_leg

model_ft.last_linear = RetBlock()

Any wrong 0.0
Dave