Sometimes one needs to update the feature extractors too. That is the feature extractor for imagenet might not be optimal for age detection. If you go through fastai course, you will see that they first train the classifier by freezing the feature extractor in first stage. Then in the second stage they unfeeze the whole network and train all the layers.