Aliseyfi:

“does simply changing the number of labels mean that we have changed the classifier head?!”


=> yes. If you change the number of output neurons, then you’ll get a new linear layer whose weights and bias are randomly initialized.
Also, not sure why you have _num_labels, you should only have num_labels.