Let’s take your first example of altering the number of classes. Since the weights on the head of the network is not trained (initialized randomly), shouldn’t gradient descent be smart to modify the parameters in those new layers to adapt accordingly.