Hey Daniel!
Thank you so much for this extremely detailed answer. I just tried it out but unfortunately I am getting this strange message:
File "<ipython-input-3-4a6dcde8a731>", line 11
  nn.Linear(4096, num_classes),
SyntaxError: only named arguments may follow *expression

I am not able to modify the self.classifier by adding these lines:
nn.Linear(4096, num_classes),
nn.Sigmoid()

These two lines above seem to cause the trouble because without these two statements, I am at least able to pick the options 1,2,4,5 I want out of the Alexnet classifier.
Do you know why it is not working because basically I just copied 1 to 1 the code above
Moreover, I wanted to ask you where I am supposed to use these weight freezing and unfreezing statements:
freeze all layers
for param in model2.parameters():
    param.requires_grad = False

unfreeze last fc layer
for layer_idx, param in enumerate(model2.classifier.parameters()):
if layer_idx > 3:
    param.requires_grad = True

within the defintion of my CustomAlexnet class or can this be be done after I have initialized my own net with this statement:
load custom model
model2 = CustomAlexnet(num_classes=10)

Thanks a lot because I was about to give up diving into PyTorch and going over to Keras because there are barely any examples and tutorials doing similar things with PyTorch.
Amine