I’m fine-tuning a same custom dataset using Torch and PyTorch based on ResNet 34.


With Torch, I use fb.resnet.torch 11 with learning rate 0.001. After 50 epoch:


top1: 89.267  top5:  97.933


With PyTorch, I use code below for fine-tuing. After 90 epoch:

learning rate = 0.01, top1: 78.500, top5: 94.083
learning rate = 0.001, top1: 74.792, top5: 92.583



You can see that PyTorch fine-tuning result is still not so good as Torch. Fine-tuning ResNet 18 has similar results.
Any suggestion or guidance?
PyTorch code used for fine-tuning:
class FineTuneModel(nn.Module):
    def __init__(self, original_model, arch, num_classes):
        super(FineTuneModel, self).__init__()
            # Everything except the last linear layer
            self.features = nn.Sequential(*list(original_model.children())[:-1])
            self.classifier = nn.Sequential(
                nn.Linear(512, num_classes)
            )

        # Freeze those weights
        for p in self.features.parameters():
            p.requires_grad = False


    def forward(self, x):
        f = self.features(x)
        f = f.view(f.size(0), -1)
        y = self.classifier(f)
        return y