Hi everyone!
I’m trying some experiments in transfer learning using the Alexnet, in particular I tried 3 different modalities for my problem, all of the following have of course the last fully connected layer changed with respect to the number of classes required:

I trained the net with the weight from ImageNet without freezing any layer;
Same as before, but with the convolutional layers frozen, keeping only the classifier part with requires_grad=True;
Same as first row, but with the fully connected layers frozen, keeping only the feature part with requires_grad=True.

Well, to the best of my knowledge, the first point can involve to a previous information loss and achieve a poor result; the second point should be the raccomended situation, in which exploiting the knowledge obtained with ImageNet for the feature extraction, my net learns only the classification weights. Instead, freezing the FCs I guess that again it can involve to a loss of knowledge and the classifier part won’t be well suitable for my task.
Now, the question is… Why do I get almost equal results for both accuracy and loss in the 3 different cases?