If you want to freeze more layers, you would have to freeze model.linear as well, since you have already frozen all other layers. Note that this would make the model untrainable, as no parameters require gradients anymore.