I read the above comment after seeing a lot of people suggest on the web to do transfer learning with Sequential (*model.children()) approach. It was still unclear to me whether that approach is the correct one for only finetuning the last layer.  After seeing https://github.com/pytorch/pytorch/issues/15129 8 it made sense that this is not the correct approach, you cannot express resnet by only its .children(). I just wanted to call that out explicitly to save some time.  I ended up just mutating the last layer with  resnet.fc = nn.Linear(num_ftrs, my_num_classes) .