olaffson:


My question: which model is actually used for classification? Is it a logistic model (with uses as input the CLS representation)?



One typically just places a linear layer (nn.Linear) on top of the final hidden state of the [CLS] token. In other words, a linear classifier is sufficient. It converts the final hidden state vector into a vector that represents the classes.



 olaffson:

In the case of several classes (say bad, neutral, good) the usual methodology in machine learning is to train several one-vs-all classifiers and then predict the label with most votes. Is this what is happening under the hood with huggingface ?


For HuggingFace models, one typically only learns a single classifier on top of the base model, for binary, multi-class and multi-label classification.