I suppose you should define a new class that inherits nn.Module to feed your model.classifier module in your custom forward function. Otherwise, please replace model.classifier in your current code with model.fc and run the optimization again.