I am using AlexNet for TL, extracting the features in different layers, E.g. Fc7:
layer = 'Fc7'
alexNet = models.alexnet(pretrained=True)
new_classifier = nn.Sequential(*list(alexNet.classifier.children())[:-1])
alexNet.classifier = new_classifier
#alexNet.eval()

I noticed the following:


I only see the effect of activating/deactivating the gradient calculation at the output of this last layer (in model.train()), is it because it is required to reach the end of the network to perform the loss calculation and then do the backpropagation?


Using model.eval() the results are the same with and without the gradient activation, why?