Solved by ptrblck in post #2 
              
              
                In the first code snippet you are setting all requires_grad flags to False except the model.fc.bias, while the second code snippet additionally keeps model.fc.weight.requires_grad as True.