Hi,
That’s a nice tutorial. However, finetuning in PyTorch is about understanding what a computational graph does. So after reading the docs, it is clear that in order to finetune the gradients must not be backpropagated back to the pre-trained weights. So if you’re finetuning off say vgg you can do something like this:
vgg_base = models.vgg16(pretrained=True)
net.feat = nn.Sequential(*list(vgg_base.features.children()))
net.clf = nn.Linear(in_feat_1, out_feat_1)
net.op = nn.Linear(out_feat_1, out_feat_2)

for params in net.feat.parameters():
    params.requires_grad = False 

optim = optim.SGD(chain(net.clf.parameters(), net.op.parameters()), lr=lr, momentum=0.9)

I hope that adding this to your tutorial may help.